| Document Version 21 |  | |
LINX Protocols | ||
| Prev | 10. Enea LINX RapidIO Connection Manager | Next |
|---|---|---|
RapidIO is considered as a reliable media, so there is no need for a reliability protocol. RapidIO hardware are however not reliable to the full extent - packets can be received in a different order than they were sent. This requires sequence numbers on all data packets that are sent with the protocol described herein.
If a malformed packet is received, the RapidIO Connection Manager resets the connection and informs the RLNH.
When the RapidIO Connection Manager encounters problems which prevents delivery of a message or part of a message it must reset the connection. The RapidIO CM notifies the RLNH when the peer replies with RESET or, if the peer has crashed, the Connection Supervision timer fires.
All Enea LINX RapidIO packets starts with the common rio_frame header. The rio_frame header allows for other protocols to coexist with this protocol.
The protocol number is 0x22cf (8911) for version 2 of the Enea LINX RapidIO Connection Manager Protocol. It was 0x8911 for version 1.
The rio_frame header is followed by either a CONN, HEARTBEAT or UDATA header. There are three types of CONN headers, one HEARTBEAT header and five types of UDATA headers. Some fields are common for all nine headers. The common fields are type and dst_port:
The type is used to identify the type of the packet. The dst_port is used to identify the destination port of the packet. In multi-core processors that share the same RapidIO device, the dst_port can be used by the hardware to identify the destination core of the packet.
The following headers types are used in the Connection Establishment Algorithm:
Table 10.3 RapidIO Connection Manager Connect Headers
| Header type | Value | Definition |
| RIO_CONN_REQ | 0x81 | Connect request. Used to request connection establishment and its settings |
| RIO_CONN_ACK | 0x82 | Connect acknowledgement. Used to acknowledge the connection request and its settings. |
| RIO_CONN_RESET | 0x83 | Reset. Used to cancel the connection. |
The connection algorithm is quite straightforward.
Let X and Y be two nodes in the same rapidio network. Let a connection be created on X towards Y. X will start to periodically send RIO_CONN_REQ to Y. When a connection is created on Y towards X, it will do the same.
When Y (or X) receives a RIO_CONN_REQ, it will respond with a RIO_CONN_ACK. When X receives a correct RIO_CONN_ACK, it will consider the connection as established and respond with its RIO_CONN_ACK. When Y receives this RIO_CONN_ACK, it will consider the connection as established, it may also send a RIO_CONN_ACK to X again, which will be dropped by X.
The last transmission of a RIO_CONN_ACK will seem unnecessary, but it is a consequence of the state machine used in the implementation. The important thing to notice here is that a correct RIO_CONN_ACK is dropped by a peer that considers the connection to be established.
A correct RIO_CONN_ACK is a RIO_CONN_ACK with the correct negotiated settings for the connection. The RIO_CONN_REQ has fields with requests for the mtu and the heartbeat timeout for the connection. These values are set by a logic expression for each field. The result of this calculation is sent in the RIO_CONN_ACK. The lowest configured mtu is used, while the largest configured heartbeat timeout is used. If this calculation mismatches, the RIO_CONN_ACK is regarded as incorrect, and then treated as a RIO_CONN_REQ instead!
RIO_CONN_REQ
A generation value for this connection. Used to distinguish between new and old connections.
The MTU that the sender of the RIO_CONN_REQ wants to use for this connection.
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
This field is not used in this header.
The heartbeat timeout that the sender of the RIO_CONN_REQ wants to use for this connection. The value is written in hundreds of msec. A value of 5 yields a tmo of 500ms.
The device id of the sender.
The port of the sender.
Table 10.5 RIO_CONN_ACK Header
| 0 | 1 | 2 | 3 |
| type | generation | mtu_ack | |
| dst_port | generation_ack | hb_tmo_ack | |
| sender | src_port | ||
| my_cid | |||
RIO_CONN_ADD
A generation value for this connection. Used to distinguish between new and old connections. The connection generation value of the sender.
The MTU calculated by the sending peer.
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
This field is used to acknowledge the generation field sent from the peer in the RIO_CONN_REQ.
The heartbeat timeout calculated by the sending peer. The value is written in hundreds of msec. A value of 5 yields a tmo of 500ms.
The device id of the sender.
The port of the sender.
The cid (connection id) that is to be used in udata and heartbeat headers for this connection. It is uniqe for each connection and allows the receiver to quickly identify the connection.
RIO_CONN_RESET
A generation value for this connection. Used to distinguish between new and old connections.
This may contain the MTU of the connection. Disregarded field.
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
This field is not used in this header.
The device id of the sender.
The port of the sender.
Data is sent reliable on RapidIO. But some drivers may not guarantee that the order of the incoming packets is the same as the order they were sent in. Therefore, the user data packets need sequence numbers. The receiver implements a reordering queue to ensure that all messages are delivered in order.
Table 10.7 RapidIO Connection Manager User Data Headers
| Header type | Value | Definition |
| RIO_SINGLE | 0x1 | Used when the entire messages plus header can fit within the MTU |
| RIO_FRAG_START | 0x2 | The first fragment when sending with simple fragmentation |
| RIO_FRAG | 0x3 | Remaining fragments for both types of fragmentation |
| RIO_PATCH_START | 0x4 | The first fragment when receiving with improved fragmentation |
| RIO_PATCH | 0x5 | Contains patches for the ‘holes’ created by the improved fragmentation |
Messages can be smaller or larger than the max amount of
payload that can be sent in a single packet. When the messages are
smaller, the RIO_SINGLE header is used. If otherwise, there are two ways
of sending fragmented messages.
The simple fragmentation is implemented using two headers, RIO_FRAG_START and RIO_FRAG. A RIO_FRAG_START header is trailed by the needed amount of RIO_FRAG headers to complete the message.
The other way of sending/receiving fragmented messages is an optimization done in the OSEck Operating System. The Linux implementation of the RapidIO Connection Manager only has support of receiving such messages. This way is implemented using three headers, RIO_PATCH_START, RIO_FRAG and RIO_PATCH. The RIO_PATCH_START is trailed by the necessary amount of RIO_FRAG headers and then finally trailed by the necessary amount of RIO_PATCH headers. The scenario for using these headers is as follows:
The cm needs to send a message larger than mtu over the connection. To avoid a lot of memcpys, the cm merely overwrites some data of the message with RIO_PATCH_START and RIO_FRAG headers after moving that data to RIO_PATCH packets. It will then transmit every fragment of the message, which then consists of subsequent data packets. Finally, it transmits all RIO_PATCH packets that the peer needs to patch (repair) the overwritten data in the message with.
RIO_SINGLE
A message identifier for this message.
The sequence number for this packet
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
The connection id that was submitted in the RIO_CONN_ACK header.
The device id of the sender.
The port of the sender.
The source link adress of the message
The destination link adress of the message.
The total size of the message.
RIO_FRAG_START
A message identifier for this message.
The sequence number for this packet
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
The connection id that was submitted in the RIO_CONN_ACK header.
The device id of the sender.
The port of the sender.
The source link adress of the message
The destination link adress of the message.
The total size of the message.
RIO_FRAG
A message identifier for this message.
The sequence number for this packet
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device. When multiple connections share a device ID they must have different ports.
The connection id that was submitted in the RIO_CONN_ACK header.
The device id of the sender.
The port of the sender.
Table 10.11 RIO_PATCH_START Header
| 0 | 1 | 2 | 3 |
| type | msgid | seqno | |
| dst_port | dst_cid | ||
| sender | src_port | ||
| src | |||
| dst | |||
| payl_size | |||
| count_frag | count_patch | ||
RIO_FRAG_START
A message identifier for this message.
The sequence number for this packet
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device.
The connection id that was submitted in the RIO_CONN_ACK header.
The device id of the sender.
The port of the sender.
The source link adress of the message
The destination link adress of the message.
The total size of the message.
Number of RIO_FRAG fragments that will trail this header
Number of RIO_PATCH packets that will trail the fragments
RIO_PATCH
A message identifier for this message.
The sequence number for this packet
This is the user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device.
The connection id that was submitted in the RIO_CONN_ACK header.
This is the device id of the sender.
This is the port of the sender.
Supervision of the connection is needed in order to detect if a peer has ‘died’.
Table 10.13 RapidIO Connection Manager Connection Supervision Header
| Header type | Value | Definition |
| RIO_HEARTBEAT | 0x6 | Sent periodically to indicate that the sender is alive |
The heartbeat packet is sent every negotiated hb_tmo * 100 ms.
When a node has not received a heartbeat in three periods, the
connection is considered timed out and has to be reestablished.
RIO_HEARTBEAT
pad
Not used.
The user defined destination port of the connection. This field is used by some RapidIO hardware to distinguish between destination cores that shares the same device.
The connection id that was submitted in the RIO_CONN_ACK header.
The device id of the sender.
The port of the sender.