Robot Middleware & DDS: The Ultimate Guide

Every robot with more than one process has a middleware problem. A LiDAR driver produces scans, a SLAM node wants them, a planner wants the map SLAM produces, and a motor bridge wants the velocity command the planner emits. Something has to carry those messages between processes and machines, find the endpoints without hard-coding addresses, turn C++ and Python structs into bytes and back, and do it fast enough that the control loop does not starve. That something is the middleware, and on most modern robots it is DDS sitting under ROS 2.

Middleware is invisible when it works and baffling when it does not. Two nodes on the same machine refuse to see each other, a camera stream silently delivers nothing, a fleet of twenty robots pins a CPU core doing discovery, and the logs say nothing useful because the delivery contract that was violated was never printed anywhere. The people who ship robots learn the middleware layer on purpose, because the alternative is learning it at 2 a.m. during a field test.

This guide covers what robot middleware actually does, why ROS 2 was built on the OMG Data Distribution Service, the DDS concepts that matter (participants, topics, domains, the RTPS wire protocol, and the Quality of Service policies that govern delivery), the real-time and shared-memory story including iceoryx, the alternatives (Zenoh, MQTT, gRPC, LCM) and where each earns its place, and the security layer (DDS-Security and SROS 2) that turns an open LAN broadcast into an authenticated, encrypted graph. Real specifics throughout: RTPS 2.5, the RxO compatibility model, Fast DDS and Cyclone DDS, rmw_zenoh, and the tuning knobs that decide whether a depth cloud arrives whole.

The take: Robot middleware is a publish/subscribe data bus plus discovery, serialization, and a delivery contract called QoS, and DDS is the industrial-grade implementation ROS 2 standardized on. Learn the four ideas that carry the whole system (participants and topics, the RTPS wire format, the RxO QoS matching rule, and where shared memory takes over from the network) and most middleware mysteries become ten-minute debugging sessions instead of lost days. The transport is pluggable; the concepts stay the same whichever one you pick.

Companion reading: ROS 2: the ultimate guide, real-time control systems, robot cybersecurity, edge AI & robot compute, and robot networking: EtherCAT & TSN.

Table of contents

1. Key takeaways
2. What robot middleware does
3. Why ROS 2 chose DDS
4. DDS core concepts
5. RTPS: the wire protocol
6. Discovery & domains
7. Quality of Service in depth
8. Serialization & message formats
9. Shared memory & zero-copy
10. The alternatives: Zenoh, MQTT, gRPC, LCM
11. Real-time & determinism
12. Middleware security: DDS-Security & SROS 2
13. Tuning & production checklist
14. Frequently asked questions

What robot middleware does

Middleware is the software layer that lets independent programs on a robot exchange data without knowing about each other's location, language, or lifecycle. Strip away the branding and every robot middleware does the same four jobs.

Publish/subscribe messaging. The dominant pattern in robotics is anonymous, many-to-many pub/sub. A publisher writes typed messages to a named channel (a topic), and any number of subscribers read from it. The publisher does not know who listens, subscribers do not know who publishes, and either side can appear or vanish without breaking the other. This decoupling is why you can start a logger against a running robot, or swap a perception node, without touching the rest of the graph. Streaming data (sensor scans, odometry, transforms, images) flows this way because it is periodic, high-rate, and has many consumers.

Discovery. Before two endpoints can talk, they have to find each other. Discovery is the process by which a new participant announces itself and learns who else exists, what topics they offer, and with what delivery contract. Centralized designs use a broker or a master registry; decentralized designs like DDS have every participant announce over multicast and match peer to peer. Discovery is where the "no configuration" magic comes from, and also where large graphs get expensive.

Serialization. A message in memory is a C++ or Python object with pointers, alignment, and platform-specific layout. To cross a process boundary it has to become a flat, self-describing or schema-agreed byte stream, then be reconstructed on the other side. Serialization (also called marshalling) does this. DDS uses the OMG Common Data Representation (CDR); other systems use Protocol Buffers, FlatBuffers, MessagePack, or a hand-rolled format. The serializer's speed and whether it can be deserialized in place both matter for high-rate data.

Transport abstraction. The same publish call should work whether the subscriber is a function away, a process away over loopback, or a machine away over Wi-Fi. Transport abstraction hides the mechanism. The middleware picks shared memory for intra-host, UDP or TCP for inter-host, and increasingly a routed overlay for WAN. Your code publishes; the middleware routes.

Rule of thumb: if you can describe your data flow as "producers write named streams, consumers read them, and neither should care who the other is," you want pub/sub middleware. If it is "call this specific function on that specific server and wait for the answer," you want a request/response system like gRPC. Most robots need both, with pub/sub carrying the bulk.

The reason this layer exists as a distinct concern is reuse. A sensor_msgs/LaserScan from a Hokuyo and one from an Ouster look identical to a SLAM node because the middleware standardizes the message and the transport. That standardization is the entire value proposition, and it is why the middleware choice ripples through everything above it.

Why ROS 2 chose DDS

ROS 1 shipped its own middleware: a custom transport called TCPROS/UDPROS with a central roscore master that every node registered with to find every other node. It worked for a decade of research and had three problems that kept it out of products. The master was a single point of failure. There was no delivery contract beyond "TCP, reliable, in order." And there was no security, no real multi-robot story, and no path to microcontrollers or hard networks.

When the ROS 2 team designed the replacement around 2014, they made a deliberate call: rather than build another bespoke transport, adopt an existing industrial standard. They chose the OMG Data Distribution Service. The 2014 design article "ROS on DDS" laid out the reasoning, and the reasons still hold.

DDS was already a mature standard. DDS is an Object Management Group specification with roots in aerospace, defense, air traffic control, financial trading, and naval combat systems. It had multiple independent implementations, a published wire protocol (RTPS), and years of hardening in systems where a dropped message is a serious event. ROS 2 inherited that instead of reinventing it.

Decentralized discovery, no master. DDS discovery is peer to peer. There is no central registry to start, to crash, or to become a bottleneck. Kill any node and the rest keep talking. This single property is why fault-tolerant and multi-robot systems became practical in ROS 2.

A real delivery contract. DDS ships Quality of Service as a first-class concept. You can say "this stream is best-effort, drop is fine" for a camera and "this stream is reliable and latched" for a map, per topic, declaratively. ROS 1 had one behavior for everything.

Data-centricity. DDS is a data-centric middleware, meaning it understands topics as typed data spaces with keys and history, well beyond opaque byte pipes. That model supports late-joiner delivery, per-instance history, and content filtering that a plain message queue does not.

ROS 2 does not expose DDS directly. It defines an abstract middleware interface, the RMW (ROS MiddleWare) layer, and plugs a DDS vendor in behind it. Your code calls rclpy or rclcpp, which call the common rcl C library, which calls the rmw interface, which calls the DDS implementation. Swap the implementation with one environment variable:

export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp   # or rmw_fastrtps_cpp, rmw_zenoh_cpp

The layering is the point. It cost the ROS 2 team a leaky abstraction (QoS and discovery behavior bleed through from DDS, and a bug can hide in any of five stacked libraries), and it bought a robotics community that does not maintain its own network stack and can pick the transport that fits the deployment.

DDS core concepts

DDS has its own vocabulary, and the words map onto ROS 2 concepts with a small translation. Learning both saves confusion when you drop below ROS 2 to debug.

DomainParticipant. The entry point. A participant is a single membership in a DDS domain, roughly one process's presence on the bus. In ROS 2 a process (often one node, sometimes several composed together) maps to a participant. Participants are relatively heavyweight; they own the discovery machinery, so creating hundreds of them is a known way to melt discovery.

Topic. A named, typed data channel, exactly the ROS 2 topic. In DDS a topic has a name and a registered data type, and it is the rendezvous point that publishers and subscribers match on.

DataWriter and DataReader. The DDS objects that actually publish and subscribe. A DataWriter writes samples of a topic's type; a DataReader receives them. In ROS 2 these live inside the publisher and subscription objects you create. Writers and readers carry QoS, and it is their QoS that has to be compatible for data to flow.

Publisher and Subscriber. In raw DDS these are containers that group DataWriters and DataReaders and can apply shared policies. ROS 2 mostly hides them, which is why ROS people say "publisher" to mean what DDS calls a DataWriter. The vocabulary clash trips up everyone reading DDS docs for the first time.

Samples and instances. A sample is one published message. DDS also supports keyed topics, where a key field partitions a topic into instances (think one instance per tracked object), each with its own history and lifecycle. ROS 2 uses keys sparingly, but they exist under the hood.

QoS policies. The set of per-entity policies that govern delivery: reliability, durability, history, deadline, liveliness, and more. These are the knobs that decide whether and how a sample reaches a reader. They get their own section because they cause most of the pain.

The mental model to hold: a running DDS system is a set of participants, each hosting readers and writers on shared topics, connected by peer-to-peer discovery, with every reader/writer pair governed by a negotiated QoS contract. Everything else is detail on top of that.

RTPS: the wire protocol

DDS the API is one standard; DDS-RTPS the wire protocol is a separate one, and it is the reason interoperability exists. RTPS (Real-Time Publish-Subscribe) is the OMG specification that fixes the exact bytes on the network: message headers, submessage types, the sequence-number scheme, the heartbeat and acknowledgment handshake for reliable delivery, and the discovery data format. As of 2026 the current published version is RTPS 2.5.

Because RTPS is standardized, a Fast DDS publisher and a Cyclone DDS subscriber can talk even though eProsima and Eclipse wrote unrelated code. They agree on the packets. This is the property that lets a ROS 2 fleet mix vendors, or interoperate with a non-ROS DDS system on the same bus.

RTPS runs over an unreliable transport, normally UDP, and builds its own reliability on top when a writer and reader ask for it. The core of reliable RTPS is a sliding window of sequence numbers plus two control submessages. The writer periodically sends a Heartbeat announcing the range of sequence numbers it holds. The reader replies with an AckNack saying what it has and what it is missing. The writer then resends (a Gap or Data submessage) the missing samples. Best-effort delivery skips this handshake entirely: the writer fires Data submessages and never retransmits.

A few consequences fall out of this design that matter in practice.

Large messages get fragmented. A single point cloud far exceeds a UDP datagram, so RTPS splits it into DATA_FRAG submessages that the reader reassembles. If any fragment is lost and the QoS is best-effort, the whole sample is lost with no error, because a partial sample cannot be reconstructed. This is why raising kernel socket buffers matters so much for high-rate large data: a full receive buffer drops the tail fragments and you silently lose whole frames.

Reliable delivery costs round trips. The Heartbeat/AckNack exchange adds latency and CPU, especially under loss. Best-effort trades guaranteed delivery for lower and more predictable latency, which is exactly the trade a high-rate sensor wants.

Discovery itself rides on RTPS. The endpoint announcements are just samples on well-known built-in topics, which is why discovery traffic scales with the number of endpoints and shows up as steady background network load.

War story: a team streamed organized VGA depth clouds (roughly 4 to 5 MB per frame) over default DDS and saw the effective frame rate sag from 30 Hz to a jittery 12 with no error anywhere. The clouds were fragmenting into thousands of DATA_FRAG submessages, the default kernel receive buffer of a few hundred KB filled before the subscriber drained it, and the tail fragments were dropped so the samples never reassembled. Raising net.core.rmem_max to 64 MB and matching the DDS reader buffer restored the full rate. Nothing in the RTPS layer complained; a dropped fragment is a normal event, and best-effort means it stays dropped.

Discovery & domains

Discovery is how participants find each other with no central registry, and it is both DDS's best trick and its most common scaling wall.

Domains are the top-level isolation boundary. Every participant joins a domain identified by an integer, the ROS_DOMAIN_ID (0 to 232, default 0). Participants in different domains cannot see each other, full stop. The domain ID also maps to specific UDP ports, so two domains do not even share sockets. This is how you run two robots, or two engineers' dev machines, on one physical LAN without cross-talk. The first thing to check when a colleague's nodes appear in your ros2 node list is whether you are both on domain 0.

Simple discovery is the default, and it works in two phases. The Participant Discovery Phase (PDP) has every participant periodically multicast an announcement of its existence. Once two participants know about each other, the Endpoint Discovery Phase (EDP) exchanges the details of their readers and writers, including full QoS, over unicast. Only after EDP completes and the QoS proves compatible does data flow.

The cost is the scaling law. Simple discovery matches every participant with every other participant, and every reader with every compatible remote writer. For N participants that is O(N²) matching work, and each match exchanges the complete QoS of every endpoint. Doubling the node count roughly quadruples the discovery cost. A graph that is invisible at 20 nodes can pin a core at 200, and the symptom is a steady baseline CPU load with nothing publishing, or a ros2 node list that takes seconds.

Two architectural fixes exist, and both collapse the mesh to something linear.

Discovery Server (Fast DDS) introduces a broker that participants register with, turning the O(N²) mesh into O(N) client-to-server relationships. You run one or more servers and point clients at them. It also lets discovery cross network segments where multicast is blocked, which is common on managed enterprise switches.

Router model (Zenoh) does the same at the protocol level. Zenoh routers relay declarations and data, so participants talk to a router rather than to every peer, and the router handles the matching. This also gives clean WAN and multi-robot behavior, covered below.

Rule of thumb: if your graph is under about 50 participants on a clean LAN with multicast working, simple discovery is fine and needs no thought. Past that, or on any network where multicast is filtered, plan for a discovery server or a router before discovery becomes the bottleneck. It is an architecture decision, not a QoS knob.

Multicast is the hidden dependency. Simple discovery announcements go out over UDP multicast, and plenty of environments break it: managed switches with IGMP snooping misconfigured, Wi-Fi access points that drop multicast, VPNs, and container networks. When discovery fails on a network that "should work," suspect multicast first.

Quality of Service in depth

QoS is the delivery contract, and it is the concept that separates people who took a middleware course from people who ship. Each reader and writer declares a set of policies, and the middleware forms a connection only when they are compatible. Get the contract wrong and messages silently do not flow.

The policies you touch most:

Reliability. RELIABLE means the writer retransmits (via the Heartbeat/AckNack handshake) until delivery is confirmed. BEST_EFFORT means fire and forget, no retransmit. Commands, transforms, and maps want reliable; high-rate sensor streams usually want best-effort because the next sample is milliseconds away and buffering a late one is worse than dropping it.

Durability. VOLATILE delivers only to subscribers present when the sample is published. TRANSIENT_LOCAL has the writer keep the last N samples and deliver them to late-joining readers. This is how "latched" data works: a map, a robot description, or a static transform published once at startup still reaches a node that connects a minute later.

History. KEEP_LAST with depth N keeps the most recent N samples for delivery. KEEP_ALL keeps everything up to resource limits. Depth trades memory and staleness against the ability to catch up a briefly slow reader.

Deadline. The maximum expected gap between samples on a topic. If it is violated, both sides get a callback, which is a clean way to detect a dead sensor. Size it with headroom: a deadline equal to the nominal period false-trips constantly on ordinary scheduling jitter, so deadline ≈ k / f with k in the 1.5 to 3 range works. A 10 Hz LiDAR (100 ms period) wants roughly a 200 to 250 ms deadline.

Liveliness. A heartbeat contract that declares a writer dead if it stops asserting liveliness within a lease duration. Use it to detect a node that froze without exiting.

The rule that governs whether a connection forms is the Request versus Offered (RxO) model from the OMG DDS spec. Order each policy's values by strength, and a connection forms only if, for every policy at once, what the writer offers is at least as strong as what the reader requests.

Policy	Weaker ... stronger	RxO rule
Reliability	BEST_EFFORT < RELIABLE	offered ≥ requested
Durability	VOLATILE < TRANSIENT_LOCAL < TRANSIENT < PERSISTENT	offered ≥ requested
Deadline	larger period < smaller period	offered period ≤ requested period
Liveliness	AUTOMATIC < MANUAL_BY_PARTICIPANT < MANUAL_BY_TOPIC	offered ≥ requested

The practical consequence: a subscriber requesting RELIABLE will not connect to a BEST_EFFORT publisher, because it asks for more than the publisher offers. A BEST_EFFORT subscriber will connect to a RELIABLE publisher, because it asks for less. The trap is that an incompatible pair produces no connection and no error, silent by design. When messages do not arrive, run ros2 topic info /topic -v and compare the QoS on both ends before touching anything else.

ROS 2 ships named profiles so you rarely hand-build these:

Profile	Reliability	Durability	History	Use for
Default	RELIABLE	VOLATILE	KEEP_LAST 10	General topics, commands
Sensor data	BEST_EFFORT	VOLATILE	KEEP_LAST 5	LiDAR, camera, IMU at rate
Services	RELIABLE	VOLATILE	KEEP_LAST 10	RPC-style calls
Parameters	RELIABLE	VOLATILE	KEEP_LAST 1000	Parameter events
TF static	RELIABLE	TRANSIENT_LOCAL	KEEP_LAST 1	Static transforms, latched

Selecting a profile in rclpy:

from rclpy.qos import QoSProfile, ReliabilityPolicy, DurabilityPolicy, HistoryPolicy

# A camera at 30 Hz: drop is fine, latency matters.
sensor_qos = QoSProfile(
    reliability=ReliabilityPolicy.BEST_EFFORT,
    durability=DurabilityPolicy.VOLATILE,
    history=HistoryPolicy.KEEP_LAST,
    depth=5,
)
self.create_subscription(Image, "/camera/image_raw", self.cb, sensor_qos)

# A latched map: a node that joins late must still receive it.
map_qos = QoSProfile(
    reliability=ReliabilityPolicy.RELIABLE,
    durability=DurabilityPolicy.TRANSIENT_LOCAL,
    history=HistoryPolicy.KEEP_LAST,
    depth=1,
)
self.create_publisher(OccupancyGrid, "/map", map_qos)

The single most useful habit is to match the publisher's profile. When you subscribe to a vendor camera and get nothing, the vendor almost certainly published best-effort and you defaulted to reliable. Switch to the sensor profile and the stream appears.

Serialization & message formats

Serialization is the quiet cost center of middleware. Every message that crosses a process boundary gets flattened to bytes and rebuilt, and at high rate on big data that work shows up in your CPU budget.

DDS uses the OMG Common Data Representation (CDR), a compact binary format that lays out fields in declaration order with defined alignment. ROS 2 message types (the .msg files) are compiled into type-support code that serializes to CDR. CDR is fast and compact, and it is what ros2 bag's .mcap files store, which is why a recorded topic can be replayed into an unrelated subscriber later.

The formats you will meet across the middleware landscape:

Format	Used by	Character
CDR	DDS / ROS 2	Compact binary, schema agreed out of band, fast
Protocol Buffers	gRPC, many services	Schema-driven, versionable, wide language support
FlatBuffers / Cap'n Proto	zero-copy pipelines	Read fields without a parse step, ideal for large messages
MessagePack / JSON	telemetry, MQTT payloads	Self-describing, human-friendly, slower and larger
LCM types	LCM	Simple generated structs, lean

The property that matters for high-rate robotics is whether you can deserialize in place. A classic serializer (CDR, Protobuf) copies bytes into a fresh object, which for a 5 MB image is a real cost. Zero-copy formats like FlatBuffers let you read fields directly out of the received buffer without a parse, and true zero-copy transports (below) let the subscriber read the writer's own buffer with no copy at all. When you profile a perception pipeline and find CPU going to memcpy, serialization and copies are usually the reason, and the fix is a zero-copy path rather than a faster CPU.

Type versioning is the other practical issue. CDR agrees the schema out of band, so a publisher and subscriber built against different versions of a message will misinterpret bytes. This is why ROS 2 ties message definitions to a distribution and why mixing packages built against different message versions produces garbage fields rather than a clean error. Protobuf handles this better with field tags and optional fields, one reason service APIs that need to evolve independently often use gRPC.

Shared memory & zero-copy

The moment two nodes on the same machine exchange large messages, the network transport becomes the bottleneck, and the fix is to stop using the network at all.

Consider a camera node publishing 1080p images to three subscribers on the same host. Over a UDP loopback transport, each image is serialized, copied into a kernel socket buffer, copied out three times, and deserialized three times. That is a lot of memory bandwidth for data that never left the machine. Shared-memory transport replaces the socket with a shared-memory segment: the writer places the sample in shared memory once, and readers access it there.

Shared-memory transport (Fast DDS and Cyclone DDS both offer it) keeps the serialize step but skips the loopback network, moving big intra-host messages far more cheaply. It engages automatically for participants that discover each other on the same host.

True zero-copy goes further and removes the copy and often the serialize step entirely. The writer allocates the message directly in a shared-memory pool (a "loaned" message), fills it in place, and publishes a reference. Readers see the same physical bytes. Nothing is copied and, for fixed-size plain data, nothing is serialized. For a 5 MB point cloud to several subscribers this is the difference between saturating memory bandwidth and near-free delivery.

iceoryx is the shared-memory backbone here. Eclipse iceoryx (and its successor iceoryx2) is a zero-copy inter-process communication library built for exactly this. It uses a lock-free shared-memory design with a small RouDi daemon that manages the memory pools and discovery, and it delivers messages between processes in single-digit microseconds regardless of message size, because it passes a pointer rather than the payload. Cyclone DDS integrates iceoryx as a transport (the cyclonedds iceoryx binding), and ROS 2's loaned-message API surfaces zero-copy to application code when the underlying RMW and message type support it.

The constraints are real. Zero-copy needs fixed-size messages (no variable-length arrays or strings in the type, because the pool allocates fixed slots), it is intra-host only (shared memory does not cross machines), and it requires the loaned-message API path through your code. Variable-size messages fall back to shared-memory-copy or the network. So the big win applies cleanly to fixed-layout data like images and structured point clouds, and less cleanly to variable-length messages.

Rule of thumb: if a message is over roughly 1 MB and its subscribers are on the same host, you want shared memory, and if it is also fixed-size and on a hot path, you want true zero-copy through the loaned-message API. Below that size the copy cost is noise and the plain transport is simpler. See edge AI & robot compute for how this interacts with GPU pipelines, where you also want to avoid copying frames off and onto the device.

The alternatives: Zenoh, MQTT, gRPC, LCM

DDS is the ROS 2 default, and it is not the only middleware a robot uses. Each alternative fits a shape DDS handles less well, and real systems mix them.

Zenoh (Eclipse) is the rising star and increasingly a first-class ROS 2 option via rmw_zenoh_cpp. It is a pub/sub, store, and query protocol built around a router model that sidesteps DDS's O(N²) discovery and behaves well over WAN, cellular, and lossy links. Where classic DDS assumes a well-behaved LAN with multicast, Zenoh routes explicitly and scales to multi-robot and cloud-to-edge topologies without the discovery meltdown. It also has a much smaller footprint, which suits constrained devices. In 2026 rmw_zenoh is an officially supported RMW, and it is the answer when your pain is large graphs, flaky networks, or robots talking across sites. A zenoh-bridge-dds also lets a Zenoh backbone carry DDS traffic between islands.

MQTT is a lightweight publish/subscribe protocol built for telemetry over unreliable networks to a central broker, and it dominates IoT. It is broker-centric (every message goes through a broker, unlike DDS's peer-to-peer model), which makes it a poor fit for the high-rate intra-robot bus but an excellent fit for shipping robot telemetry and receiving commands over the internet. A common architecture runs DDS/ROS 2 inside the robot and an MQTT client that bridges selected topics up to a cloud broker for fleet monitoring. MQTT's QoS levels (0 fire-and-forget, 1 at-least-once, 2 exactly-once) are coarser than DDS's, which is fine for telemetry.

gRPC is a request/response RPC framework over HTTP/2 with Protocol Buffers, from Google. It is the right tool for service APIs: "call this method on that server, get a typed response," with streaming variants. Robots use it for the parts that are genuinely client/server rather than pub/sub: a web or mobile app talking to the robot's control API, a cloud service the robot queries, inter-service calls in a backend. It does not replace the sensor bus (it has no peer-to-peer discovery and no many-to-many streaming in the pub/sub sense), and it complements it where the interaction is a call and a reply.

LCM (Lightweight Communications and Marshalling), from MIT's DARPA Urban Challenge work, is a lean pub/sub library over UDP multicast with a simple type-generation system. It has no QoS, no reliability, and no discovery beyond multicast, which is exactly why some teams like it: it is small, predictable, easy to log and replay, and has no configuration surface. It shows up in research vehicles and legacy autonomy stacks where the DDS machinery is more than the project wants. It is a reasonable choice for a closed, known set of nodes on a trusted LAN, and it gives you none of the security or delivery-contract features DDS provides.

Middleware	Pattern	Discovery	Best fit
DDS	Pub/sub, data-centric	Peer-to-peer (multicast)	The intra-robot bus, ROS 2 default
Zenoh	Pub/sub + query	Router model	Multi-robot, WAN, constrained devices
MQTT	Pub/sub	Central broker	Cloud telemetry and command
gRPC	Request/response RPC	Explicit endpoints	Service APIs, app-to-robot, backend
LCM	Pub/sub	UDP multicast	Lean research stacks, easy logging

The takeaway is that "robot middleware" is rarely one thing on a shipping product. DDS or Zenoh carries the internal graph, MQTT lifts telemetry to the cloud, and gRPC serves the control API. Picking each by the shape of its traffic beats forcing everything through one bus.

Real-time & determinism

Middleware sits between the application and the network, and both add latency with a tail. Being precise about what the middleware can and cannot promise saves a class of field failures.

Pub/sub over DDS is soft real-time. Delivery latency is low and usually predictable, but it is not bounded in the hard sense. Discovery traffic, retransmissions under loss, kernel scheduling, socket buffer contention, and (in Python) garbage collection all add jitter. The number that ends careers is the worst case, not the mean. A camera stream that averages 3 ms of transport latency can spike to tens of milliseconds when a second robot joins the domain and floods discovery, and the average still looks fine.

Reliable QoS trades latency for delivery. The Heartbeat/AckNack handshake and retransmission mean a reliable stream under packet loss has a longer and less predictable tail than a best-effort one. For a control setpoint that must arrive, this is the right trade. For a high-rate sensor where the next sample is imminent, best-effort gives the tighter, more predictable latency you actually want. Choosing QoS is choosing your latency distribution.

Shared memory is the low-jitter path. iceoryx-class transports deliver in single-digit microseconds with tiny variance because they pass a pointer and never touch the network stack or the scheduler's networking path. When intra-host determinism matters, zero-copy shared memory is far more predictable than any network transport.

The hard loop belongs below the middleware. The kHz current loop that commutates a motor, and often the 1 kHz joint loop, should not close through the DDS graph on general-purpose Linux. Those live in drive firmware or a tuned control layer, with the graph carrying velocity or position setpoints at tens to hundreds of Hz. Architectures that route a 1 kHz balance loop through DDS work on the bench and miss deadlines in the field when discovery, a Wi-Fi roam, and an allocation line up on the same tick. The real-time control systems guide covers where each loop belongs, and robot networking: EtherCAT & TSN covers the deterministic fieldbus layer that carries the truly hard traffic.

If you need determinism inside the middleware layer, the levers are the same ones that make any Linux workload real-time: a PREEMPT_RT kernel (mainlined as of Linux 6.12), isolated CPUs, SCHED_FIFO priorities, locked memory (mlockall), pre-allocated messages so the hot path never calls malloc, and a transport tuned for latency (Cyclone DDS or Fast DDS over shared memory). Measure with cyclictest and application-level latency tracing rather than trusting the average. On a stock kernel under load, worst-case wakeup latency runs into the high hundreds of microseconds to low milliseconds; on PREEMPT_RT with isolated cores it usually holds under about 100 microseconds. The middleware inherits whatever the OS gives it.

Middleware security: DDS-Security & SROS 2

By default, every topic on a DDS graph is readable and writable by anyone who can reach the network. There is no authentication, no encryption, and no access control. On a trusted lab LAN that is fine. On anything that touches an untrusted network it is a serious exposure: an attacker on the segment can subscribe to camera feeds, inject velocity commands, or flood discovery. Robot security starts with closing this.

DDS-Security is the OMG specification that adds security as a set of pluggable service plugins to DDS, and it maps cleanly onto the five things a secure bus needs.

• Authentication. Participants prove identity with X.509 certificates signed by a shared certificate authority. A participant with no valid cert cannot join.
• Access control. A signed permissions file (governance and permissions documents) declares which participant may publish or subscribe to which topics. A node with a valid identity still cannot touch a topic it is not permitted to.
• Cryptography. Traffic is encrypted and authenticated on the wire (commonly AES-GCM), so a sniffer sees ciphertext and cannot forge or replay messages.
• Logging and tagging. Auditable security events and data-tagging round out the plugin set.

The whole scheme is certificate-managed, which is the operational cost. You run a certificate authority, issue per-participant identity certs, sign the governance and permissions files, and distribute and rotate keys. That key management is the real work; turning the feature on is a configuration change, keeping it running is a PKI.

SROS 2 is the ROS 2 tooling that wraps DDS-Security in commands a robotics team can actually use. It generates the keystore, creates identity and permissions for each node, and packages the artifacts so a launch can bring up a secured graph:

ros2 security create_keystore demo_keystore
ros2 security create_enclave demo_keystore /talker_listener/talker
ros2 security create_enclave demo_keystore /talker_listener/listener
export ROS_SECURITY_KEYSTORE=$PWD/demo_keystore
export ROS_SECURITY_ENABLE=true
export ROS_SECURITY_STRATEGY=Enforce

With Enforce, a node without valid credentials for a topic simply cannot communicate on it, and the enclave model gives each node its own identity and permission set.

The cost side is honest: DDS-Security adds CPU for the crypto and latency for the handshakes and per-message authentication, and it adds the PKI operational burden. The rule is to turn it on for anything that leaves a trusted network and to budget for key management as a real ongoing task. The broader threat model (network segmentation, secure boot, update signing, physical access) lives in the robot cybersecurity guide; DDS-Security secures the message bus, which is one layer of several.

Rule of thumb: treat an unsecured DDS graph like an unsecured database. It is fine behind a firewall on a trusted segment and it is a liability the moment it can be reached from anywhere else. If the robot has a radio and leaves the building, secure the bus.

Tuning & production checklist

Out-of-the-box middleware settings are tuned for correctness on a small graph on a clean network. A shipping robot needs deliberate configuration. The adjustments that matter most, roughly in order of how often they bite:

Raise OS socket buffers. Increase net.core.rmem_max and net.core.wmem_max to tens of MB (64 MB is a common target) and configure the DDS reader/writer buffers to match. This is the fix for silent frame loss on large messages, and it is arithmetic, not superstition: a full receive buffer drops fragments and whole samples never reassemble.

Enable shared memory for intra-host traffic. Turn on the shared-memory transport (Fast DDS or Cyclone DDS) so co-located nodes stop round-tripping big messages through loopback, and use loaned messages for zero-copy on fixed-size hot-path data.

Isolate graphs by domain. Give each robot and each dev machine its own ROS_DOMAIN_ID. This prevents cross-talk and cuts discovery load, since a participant only matches within its domain.

Plan discovery for scale. Past roughly 50 participants, or on any network where multicast is filtered, deploy a Fast DDS Discovery Server or move to rmw_zenoh. Do this before discovery becomes the bottleneck, because it is an architecture change.

Right-size QoS. Match publisher and subscriber profiles (the RxO rule), use best-effort with a shallow history for high-rate sensors, reliable and TRANSIENT_LOCAL for latched configuration, and set deadlines with headroom to catch dead sensors without false trips. Do not run KEEP_ALL on an image topic unless you enjoy buffering megabytes of stale frames.

Pick the RMW on purpose. Stay on the distribution default (Fast DDS) unless you have a reason. Move to Cyclone DDS for lean, predictable latency and simple config on a single robot; move to Zenoh for multi-robot, WAN, or lossy links. Change it with one environment variable and re-test, because QoS and discovery behavior differ subtly between vendors.

Secure anything that leaves a trusted network. Turn on SROS 2 / DDS-Security, and stand up the certificate authority and key rotation process before you need them.

Instrument the middleware. Use ros2 topic info -v to compare QoS, ros2 topic hz and ros2 topic bw to watch rate and bandwidth, and vendor tools (Fast DDS Monitor, Cyclone's tracing) to see discovery and retransmissions. When something is wrong, the graph will usually tell you if you ask it.

The honest summary: the middleware layer is the part of a robot software stack most likely to fail silently and least likely to be understood, and a modest amount of upfront configuration (buffers, domains, QoS, discovery, security) removes most of the field surprises. Spend a day on it before the robot leaves the lab, not a night after it comes back.

Frequently asked questions

What is robot middleware in one sentence? It is the software layer that lets a robot's independent programs exchange typed data without knowing each other's location or language, by providing publish/subscribe messaging, automatic discovery, serialization, and transport abstraction. On most modern robots that layer is DDS under ROS 2.

Is DDS the same as ROS 2? No. DDS is a general-purpose data-distribution standard from the Object Management Group, used well beyond robotics. ROS 2 is built on top of DDS through a pluggable RMW interface, adding robotics conventions, message types, and tooling. You can run DDS with no ROS, and ROS 2 can run on non-DDS middleware like Zenoh.

Why do my messages not arrive even though the publisher is running? The overwhelmingly likely cause is a QoS mismatch. A subscriber requesting RELIABLE will not connect to a BEST_EFFORT publisher, and the failure is silent by design. Run ros2 topic info /your_topic -v and compare the reliability and durability on both ends. Sensor topics are usually best-effort, so subscribe with the sensor profile.

What is RTPS and why does it matter? RTPS (Real-Time Publish-Subscribe) is the standardized wire protocol under DDS. It fixes the exact packets on the network, including the heartbeat and acknowledgment handshake for reliable delivery, so implementations from different vendors interoperate. It is why a Fast DDS publisher and a Cyclone DDS subscriber can talk despite sharing no code.

Fast DDS, Cyclone DDS, or Zenoh: which should I use? Start with the distribution default (Fast DDS). Switch to Cyclone DDS for lean, predictable latency and simple tuning on a single robot. Use rmw_zenoh for multi-robot, WAN, cellular, or lossy networks where classic DDS discovery struggles. Change it with the RMW_IMPLEMENTATION environment variable and re-test.

When should I use MQTT or gRPC instead of DDS? Use MQTT to ship telemetry and receive commands over the internet through a central broker; it suits cloud fleet monitoring, not the high-rate intra-robot bus. Use gRPC for request/response service APIs, such as an app or cloud service calling the robot's control methods. Keep DDS or Zenoh for the internal sensor and control graph, and bridge selected topics out.

What is zero-copy and when do I need it? Zero-copy delivery lets a subscriber read the publisher's own buffer with no memory copy, and often no serialization, using shared memory. You need it when co-located nodes exchange large, fixed-size messages at high rate (images, structured point clouds), where copying dominates the CPU cost. iceoryx provides the shared-memory backbone, and ROS 2's loaned-message API exposes it. It is intra-host only and needs fixed-size types.

Why does my robot fleet slow down as I add nodes? Default DDS simple discovery matches every participant with every other, so cost scales as O(N²) in the number of participants. Doubling the nodes roughly quadruples discovery work, which shows up as steady background CPU and slow ros2 node list. The fix is a Fast DDS Discovery Server or a Zenoh router, which collapse the mesh to a linear client-to-server model.

Is DDS secure by default? No. By default every topic is readable and writable by anyone on the network. DDS-Security (exposed as SROS 2) adds certificate-based authentication, per-topic access control, and on-the-wire encryption, but it is opt-in and requires running a certificate authority and managing keys. Turn it on for any robot that touches an untrusted network.

Can I close a real-time control loop over DDS? Only soft real-time loops at tens to hundreds of Hz, and only with care (PREEMPT_RT kernel, isolated cores, best-effort or shared-memory transport, no allocation in the loop). The kHz current loop and usually the 1 kHz joint loop belong below the middleware in drive firmware or a deterministic control layer, with the DDS graph carrying setpoints and telemetry.