Allow null-EID requests & responses with extended addressing

[fxxJuses]

Description:

When the BMC is configured as an MCTP Endpoint and receives an Endpoint Discovery request (with a target EID of 0xFF) from the Bus Owner, the mctpd userspace daemon attempts to send a response using physical addressing.

Our analysis of the Linux kernel's MCTP protocol stack (net/mctp/) reveals a conflict that prevents this response from being routed:

1. EID Registration Restriction: The kernel's mctp_address_unicast() function (used by mctp_rtm_newaddr for EID registration via Netlink) does not permit 0x00 (MCTP_ADDR_NULL) or 0xFF (MCTP_ADDR_ANY) EIDs to be registered as local addresses to an mctp_dev. This means that if the mctp_dev corresponding to the BMC's interface has not yet been assigned a unicast EID (0x01-0xFE), its addrs list will be empty, and num_addrs will be 0.

2. Blocking Outbound Traffic: The mctp_local_output() function (in route.c), which is responsible for processing all outbound MCTP packets, includes a critical check:

   if (rt->dev->num_addrs == 0) {
       rc = -EHOSTUNREACH;
   } else {
       saddr = rt->dev->addrs[0];
       rc = 0;
   }
   if (rc)
       goto out_release;

   If rt->dev->num_addrs is 0 (as described in point 1), rc is set to -EHOSTUNREACH, and the function immediately exits, preventing the packet from being sent.

3. mctpd's Intent: The mctpd daemon (specifically handle_control_endpoint_discovery and endpoint_query_phys) intends to send a response. endpoint_query_phys even contains a comment: "The kernel mctp stack has special handling for eid=0 to make sure we can recv a response on the socket, so it's important to set eid=0 here in the request." This suggests an expectation of kernel support for 0x00 EID handling, possibly for source EIDs in discovery responses. However, our analysis indicates this handling is not present for outbound traffic when no valid EIDs are registered.

Consequence:
As a result, if a BMC (acting as an Endpoint) has no assigned EID, it cannot send an Endpoint Discovery response with a 0x00 source EID (or any source EID) back to the Bus Owner, because the kernel stack blocks the transmission. This hinders the successful completion of the Endpoint Discovery process from the Endpoint's perspective.

Proposed Question for CodeConstruct:
What is the recommended solution or workflow for a BMC in an Endpoint role to successfully respond to Endpoint Discovery requests when its mctp_dev has no assigned EID, given the current Linux kernel MCTP stack's behavior? Are there specific configurations, a sequence of Netlink commands, or a userspace workaround to enable this scenario?

[jk-ozlabs]

This is fundamentally the same limitation discussed in #8.

endpoint_query_phys even contains a comment: "The kernel mctp stack has special handling for eid=0 to make sure we can recv a response on the socket, so it's important to set eid=0 here in the request."
This suggests an expectation of kernel support for 0x00 EID handling, possibly for source EIDs in discovery responses.

That comment is in reference to a destination EID, so is unrelated to the case you're referring to here, where there is no local EID assigned.

What is the recommended solution or workflow for a BMC in an Endpoint role to successfully respond to Endpoint Discovery requests when its mctp_dev has no assigned EID, given the current Linux kernel MCTP stack's behavior?

The solution would be to allow messaging with no local EID assigned, by relaxing the rt->dev->addrs requirement, while checking for other assumptions on the presence of an address - like ensuring that the tag/key has been registered correctly for the corresponding response handling.

[fxxJuses]

Thank you very much for your detailed response. I clearly understand the proposed solution to relax the requirement in the kernel.

However, I'm wondering if there are any existing patches that implement this relaxation. I understand that the OpenBMC project utilizes the mctp-pcie-vdm driver, and BMCs often act in an Endpoint role in that context. It seems highly probable that they would have encountered this exact issue.

Therefore, I'd like to inquire if there's an existing solution or patch from the OpenBMC community or elsewhere that addresses this, so I don't have to duplicate the effort. Any pointers would be greatly appreciated.

[jk-ozlabs]

Not that I know of, no.

[fxxJuses]

Thank you so much!

[jk-ozlabs]

Unless maybe @khangng-ampere has done anything in this area?

[fxxJuses]

Yeah! I got it.

[fxxJuses]

Yes, the ampere-linux implementation has successfully added this functionality with two key modifications:

Relaxed routing restrictions for unassigned EIDs: They've modified the mctp_local_output function in route.c to allow messages to be sent even when the device has no EID configured (rt->dev->num_addrs == 0), but only in the context of extended addressing mode (ext_rt == true). In this scenario, the source EID is set to 1MCTP_ADDR_NULL (0x001) and the operation succeeds rather than returning an error.

Direct message handling without RTN_LOCAL routes: Since no EID is configured, there's no automatically created RTN_LOCAL route. This means that mctp_pkttype_receive can't find a suitable RTN_LOCAL route to deliver MCTP messages from the driver to the application. Their solution is to modify this function to bypass the regular route lookup when the destination EID is MCTP_ADDR_NULL (0x00) and the packet is physically addressed to the device (skb->pkt_type == PACKET_HOST). In this case, they directly call mctp_route_input to deliver the message to the application.
These changes allow an Endpoint to properly handle Discovery requests without having a pre-assigned EID, which is essential for the initial device discovery phase in MCTP networks.

For reference, you can examine their implementation in detail here: https://github.com/ampere-openbmc/linux/blob/ampere/net/mctp/route.c

[jk-ozlabs]

Oh yeah, I did that a while ago. It would need some changes for the newer dst-based routing though.

[fxxJuses]

Thank you for your solution. If your modifications are merged into the main repository or synchronized with the upstream Linux community code, I would be very happy to use your official version directly. Currently, we can only implement Ampere's solution by applying patches to the kernel, which may lead to maintenance challenges in the future. We would much prefer to see this functionality become part of the standard Linux MCTP implementation.

[jk-ozlabs]

OK, perhaps I should rework that for current MCTP core, then if you are able to test, I can propose a merge to upstream.

[fxxJuses]

Of course, it is an honour to do so

[jk-ozlabs]

I will re-use this issue to track that then: reopening and editing the title.