Protocol Support

Protocol Support

Beyond HTTP — IoT, industrial, and data streaming protocols for FIWARE 2.0

The Case

Why Multi-Protocol Matters

FIWARE is a platform for smart cities, smart industry, and IoT. The real world doesn't speak HTTP. Sensors talk MQTT and CoAP. Factory floors run OPC-UA, Modbus, and PROFINET. Building automation uses BACnet and KNX. High-performance data distribution uses DDS and Kafka. Real-time web applications need WebSockets and SSE.

Current FIWARE handles this through IoT Agents — separate Node.js processes, each speaking one protocol and translating to HTTP/REST for the broker. It works, but every protocol is a separate deployment, a separate process, a separate 100 MB of RAM, and a separate point of failure.

FIWARE 2.0 takes a different approach: fw-libs for protocols. Each protocol gets a C library — either a new fw-lib built from scratch or a thin wrapper around a mature C library. The unified IoT agent (FiWorks Multi-Agent) loads protocol modules as plugins. The broker can speak protocols directly when it makes sense. Application programmers get native C libraries for any protocol they need.

Sensor (MQTT) kmqtt FiWorks Multi-Agent FiWorks Broker
PLC (Modbus) kmodbus FiWorks Multi-Agent FiWorks Broker
Browser (WebSocket) kws FiWorks Broker kws Browser (live updates)

No more translating everything to HTTP. Each protocol speaks natively to the platform.

IoT Protocols

Sensor & Device Connectivity

These are the workhorses of IoT — the protocols that connect millions of sensors, actuators, and edge devices to the platform. They are the primary intake for data in any FIWARE deployment.

MQTT / MQTTS

TCP / TLS Priority: Critical fw-lib: kmqtt

The dominant IoT messaging protocol. Publish/subscribe model, persistent sessions, QoS levels (0 = fire-and-forget, 1 = at-least-once, 2 = exactly-once), last-will messages, retained messages. MQTTS adds TLS encryption — mandatory for any production deployment.

Used by AWS IoT Core, Azure IoT Hub, Google Cloud IoT, TTN (LoRaWAN), and the majority of FIWARE IoT deployments today (via the iotagent-json and iotagent-ul agents). MQTT 5.0 adds shared subscriptions, message expiry, topic aliases, and user properties.

FeatureDetail
TransportTCP (1883), TLS (8883), WebSocket (optional)
VersionsMQTT 3.1.1 (ubiquitous), MQTT 5.0 (modern)
QoS levels0 (fire-and-forget), 1 (at-least-once), 2 (exactly-once)
C librariesmosquitto (Eclipse, mature, widely used), paho.mqtt.c (Eclipse, async), NanoMQ (EMQ, lightweight)
ApproachCustom fw-lib (kmqtt) — MQTT is simple enough to implement from scratch with full control over memory (fwAlloc), I/O (epoll), and TLS (OpenSSL). No dependency on libmosquitto.
Effort3–4 weeks — MQTT 3.1.1 core + MQTTS + QoS 0/1. +1 week for QoS 2, MQTT 5.0 features.

MQTT is the #1 protocol for FiWorks Multi-Agent. A native fw-lib means zero external dependencies, full integration with fwAlloc/fwHttp event loops, and the ability to handle MQTT and HTTP in the same thread without context switching.

CoAP / CoAPS

UDP / DTLS Priority: High fw-lib: kcoap

The Constrained Application Protocol — essentially HTTP semantics (GET, PUT, POST, DELETE) over UDP, designed for constrained devices with limited RAM, CPU, and battery. CoAP adds observe (server-push notifications), block transfer (chunked messages for constrained links), and resource discovery (.well-known/core).

CoAPS uses DTLS (TLS over UDP) for encryption. CoAP is the IETF's answer to "HTTP is too heavy for IoT" — and it's the wire protocol for LwM2M (OMA Lightweight M2M), used in cellular IoT, smart metering, and asset tracking.

FeatureDetail
TransportUDP (5683), DTLS (5684)
Message modelRequest/response (like HTTP) + observe (server-push), 4-byte header
Key featuresConfirmable/non-confirmable messages, block-wise transfer, resource discovery
C librarieslibcoap (mature, reference implementation), Eclipse Californium (Java, reference for CoAP), microcoap (tiny, embedded)
ApproachCustom fw-lib (kcoap) — CoAP is minimal (4-byte base header, simple state machine). Perfect for a fw-lib: UDP socket, fwAlloc for message buffers, epoll for async I/O.
Effort2–3 weeks — CoAP core + observe + block transfer. +1 week for DTLS (OpenSSL).

Critical for LwM2M device management and constrained sensor networks. CoAP's 4-byte header vs HTTP's ~300 bytes makes it feasible for devices with 10 KB of RAM.

AMQP 1.0

TCP / TLS Priority: Medium Wraps: qpid-proton

The Advanced Message Queuing Protocol — enterprise-grade messaging with guaranteed delivery, transactional messaging, flow control, and sophisticated routing. Used by Azure Service Bus, Azure IoT Hub (AMQP mode), RabbitMQ, Apache ActiveMQ.

AMQP 1.0 is significantly more complex than MQTT: connection multiplexing (sessions and links), flow control (credit-based), message settlements (accepted, rejected, released, modified), and a rich type system (composite types, described types, arrays). Not worth building from scratch.

FeatureDetail
TransportTCP (5672), TLS (5671)
Message modelPeer-to-peer links with credit-based flow control, multi-session multiplexing
C librariesApache Qpid Proton-C (reference implementation, mature, async, well-maintained)
ApproachThin fw-lib wrapper around qpid-proton — AMQP is too complex for a from-scratch implementation. Wrap Proton-C with a fwAlloc-friendly API.
Effort1–2 weeks — wrapper + integration with FiWorks Multi-Agent event loop.

WebSockets

TCP / TLS (HTTP upgrade) Priority: Critical fw-lib: kws

Full-duplex bidirectional communication over a single TCP connection, initiated via HTTP Upgrade. The standard for real-time web applications — live dashboards, real-time notifications, chat, collaborative editing. Also used as a transport layer for MQTT-over-WebSocket (browser MQTT clients) and STOMP.

For FIWARE 2.0, WebSockets enable live entity subscriptions from browsers — instead of the FiWorks Broker pushing notifications via HTTP POST to a callback URL, a web dashboard can open a WebSocket and receive entity updates in real time. This eliminates the need for a separate notification receiver service.

FeatureDetail
TransportTCP (HTTP Upgrade on port 80/443), TLS (WSS)
FramingMinimal: 2–14 byte header (opcode, mask, length), binary and text frames
Key featuresFull-duplex, ping/pong keepalive, fragmentation, per-message compression (RFC 7692)
C librarieslibwebsockets (mature, feature-rich, complex API), wslay (lightweight, frame-level only)
ApproachCustom fw-lib (kws) — WebSocket framing is simple (RFC 6455). Build on top of fwHttp's TCP/TLS infrastructure. The HTTP Upgrade handshake is trivial; the frame codec is ~200 lines of C.
Effort1–2 weeks — frame codec + integration with fwHttp's epoll loop. Ping/pong, binary/text frames, WSS via existing TLS.

WebSockets are essential for modern FIWARE deployments. Every smart city dashboard, every real-time monitoring application, every browser-based IoT console needs them. Also serves as the transport for MQTT-over-WebSocket, enabling browser-based MQTT clients.

Industrial Protocols

Factory Floor & Building Automation

Smart industry ("Industry 4.0") and smart buildings are major FIWARE verticals. These protocols connect to PLCs, SCADA systems, building management systems, and industrial sensors. They speak a different language from IoT — register-based addressing, real-time constraints, decades-old standards. But they all need to get their data into the NGSI-LD world.

OPC-UA

TCP / TLS Priority: Critical Wraps: open62541

The Open Platform Communications Unified Architecture — the modern standard for industrial communication. OPC-UA provides a rich information model (nodes, references, data types), discovery, security (X.509 certificates, encrypted channels), subscriptions (monitored items with sampling intervals), and historical data access.

OPC-UA is the lingua franca of Industry 4.0. Every major PLC vendor (Siemens, Beckhoff, Rockwell, ABB) exposes OPC-UA servers. FIWARE's current IoT Agent for OPC-UA (Node.js) uses the node-opcua library and is one of the most requested agents for industrial deployments.

FeatureDetail
TransportTCP (4840), TLS, HTTPS (alternative)
Information modelAddress space with nodes, references, data types, methods, events
Key featuresDiscovery, subscriptions, historical access, alarms & conditions, X.509 security
C librariesopen62541 (Mozilla Public License, actively maintained, production-grade, single-file distribution available)
ApproachThin fw-lib wrapper around open62541 — OPC-UA is extremely complex (1400+ page spec). open62541 is an excellent C library; wrap it with a FiWorks Multi-Agent-friendly API for node browsing, subscription management, and value reading.
Effort2–3 weeks — wrapper for client mode (browse, read, subscribe, write). +2 weeks for server mode (expose NGSI-LD entities as OPC-UA nodes).

OPC-UA is the #1 protocol for industrial FIWARE. Wrapping open62541 rather than reimplementing ensures compliance with the massive spec while keeping the fw-libs integration clean.

Modbus TCP

TCP Priority: High fw-lib: kmodbus

The oldest and simplest industrial protocol still in active use. Modbus TCP wraps the original Modbus RTU (serial) protocol in TCP frames. It reads/writes registers (16-bit words) and coils (bits) from PLCs, sensors, and actuators. No discovery, no security, no subscriptions — just raw register access. Simple, universal, everywhere.

Used in power meters, VFDs (variable frequency drives), water treatment, HVAC controllers, solar inverters — anywhere there's an industrial device older than 10 years or cheaper than $100.

FeatureDetail
TransportTCP (502), serial (RTU/ASCII via RS-485)
Data modelCoils (bits), discrete inputs, holding registers (16-bit), input registers
Protocol complexityMinimal — ~10 function codes, fixed packet format, no negotiation
C librarieslibmodbus (mature, well-maintained, LGPL)
ApproachCustom fw-lib (kmodbus) — Modbus TCP is trivially simple: a 7-byte MBAP header + function code + data. Perfect for a fw-lib: ~500 lines of C for the full client.
Effort1 week — TCP client, read/write coils, read/write registers, unit-id addressing, error handling.

BACnet

UDP / IP Priority: Medium Wraps: bacnet-stack

Building Automation and Control Networks — the standard for building management systems (BMS). HVAC, lighting, fire alarms, elevators, access control. BACnet defines object types (analog-input, binary-output, schedule, trend-log, etc.) and services (ReadProperty, WriteProperty, SubscribeCOV, WhoIs/IAm).

BACnet/IP runs over UDP with broadcast discovery. It's essential for smart buildings — any FIWARE deployment in a smart campus, hospital, or commercial building will encounter BACnet devices.

FeatureDetail
TransportUDP/IP (47808), MS/TP (RS-485), Ethernet
Data modelObject-oriented: analog/binary inputs/outputs, schedules, trend logs, alarms
Key featuresChange-of-Value (COV) subscriptions, device discovery (WhoIs/IAm), scheduling
C librariesbacnet-stack (Steve Karg, open source, widely used), libbacnet
ApproachThin fw-lib wrapper around bacnet-stack — BACnet's ASN.1-derived encoding and extensive object model make reimplementation impractical. bacnet-stack is solid and well-tested.
Effort2 weeks — wrapper for client operations: device discovery, ReadProperty, WritePropery, SubscribeCOV.

PROFINET

Ethernet (Layer 2) Priority: Low (Specialized) Wraps: p-net

Siemens' industrial Ethernet protocol for real-time I/O in factory automation. PROFINET RT (real-time) runs at Layer 2 with cycle times of 1–10ms; PROFINET IRT (isochronous real-time) achieves sub-microsecond jitter for motion control. PROFINET operates at the Ethernet frame level, below TCP/IP.

FeatureDetail
TransportRaw Ethernet frames (EtherType 0x8892), LLDP for discovery
VariantsPROFINET RT (software, 1–10ms), PROFINET IRT (hardware, <1μs jitter)
C librariesp-net (rt-labs, open source PROFINET device stack), Siemens PROFINET SDK (proprietary)
ApproachWrapper around p-net for reading cyclic I/O data and alarm events.
Effort2–3 weeks — highly specialized, depends on deployment requirements.

Relevant only for Siemens-heavy factory floors. Most FIWARE industrial deployments use OPC-UA as the gateway to PROFINET controllers. Direct PROFINET support is a differentiator for edge deployments co-located with the PLC.

EtherNet/IP

TCP + UDP Priority: Low (Specialized) Wraps: OpENer

Rockwell/Allen-Bradley's industrial protocol, dominant in North American manufacturing. Uses CIP (Common Industrial Protocol) over TCP (explicit messaging) and UDP (implicit I/O). Same CIP layer as DeviceNet and ControlNet.

FeatureDetail
TransportTCP (44818) for configuration, UDP for cyclic I/O
C librariesOpENer (open source EtherNet/IP adapter stack)
ApproachWrapper around OpENer for CIP reads and implicit messaging.
Effort2–3 weeks — specialized, lower priority than OPC-UA.

Like PROFINET, most deployments will access EtherNet/IP controllers via OPC-UA. Direct support is for edge co-location scenarios.

KNX

UDP / IP (multicast) Priority: Medium fw-lib: kknx

The European standard for home and building automation. KNX connects light switches, blinds, HVAC valves, presence sensors, and energy meters over twisted pair (TP), powerline (PL), RF, or IP. KNXnet/IP tunnels KNX telegrams over UDP multicast, enabling integration from a standard network.

FeatureDetail
TransportKNXnet/IP: UDP (3671), multicast for discovery and routing
Data modelGroup addresses (e.g., 1/2/3 = "Living Room / Lights / Dimmer"), standardized datapoint types (DPTs)
C librariesknxd (Linux daemon, open source), custom implementations common
ApproachCustom fw-lib (kknx) — KNXnet/IP tunneling and routing is relatively simple: UDP multicast + fixed-format telegrams. The protocol is well-documented.
Effort2 weeks — KNXnet/IP tunneling client, group read/write, group response handling, DPT decoding for common types.

Essential for European smart building deployments. KNX installations are estimated at 500 million+ devices worldwide.

Data & Streaming

High-Performance Data Distribution

Beyond device connectivity, FIWARE needs to move large volumes of data between services, to analytics engines, and to subscribers. These protocols handle high-throughput, low-latency, and real-time data distribution at scale.

DDS (Data Distribution Service)

UDP multicast / TCP Priority: High Wraps: Fast-DDS / Cyclone DDS

The OMG Data Distribution Service — a middleware for real-time publish/subscribe data distribution. DDS provides automatic discovery (no broker), QoS policies (reliability, durability, deadline, liveliness, ownership), content-based filtering, and deterministic latency. Used in autonomous vehicles, defense, aerospace, robotics (ROS 2), financial trading, and smart grids.

DDS is already part of the FIWARE ecosystem via the FIWARE DDS Enabler (based on eProsima Fast-DDS). FIWARE 2.0 deepens this integration — DDS topics can map directly to NGSI-LD entities, enabling FIWARE to act as a bridge between DDS domains and the NGSI-LD world.

FeatureDetail
TransportRTPS (Real-Time Publish-Subscribe) over UDP multicast, TCP, shared memory
DiscoveryAutomatic peer discovery (SPDP + SEDP), no broker needed
QoS22 QoS policies: reliability, durability, deadline, lifespan, history depth, ownership, etc.
C/C++ librariesFast-DDS (eProsima, Apache 2.0), Cyclone DDS (Eclipse, C, lightweight), RTI Connext (commercial, dominant in defense/aero)
ApproachThin wrapper around Fast-DDS or Cyclone DDS — DDS is enormously complex (RTPS wire protocol, IDL type system, QoS negotiation). Use the existing FIWARE DDS stack at /opt/Fast-DDS.
Effort1–2 weeks — build on existing FIWARE DDS Enabler work. Bridge DDS topics to NGSI-LD entities.

Apache Kafka

TCP Priority: High Wraps: librdkafka

The de facto standard for event streaming at scale. Kafka provides persistent, ordered, replayable event logs. In FIWARE, Kafka serves as a notification backbone — instead of the FiWorks Broker pushing HTTP notifications to each subscriber (which fails if the subscriber is down), events are published to Kafka topics. Subscribers consume at their own pace, with replay capability.

FeatureDetail
TransportTCP (9092), TLS (9093), SASL authentication
Key featuresPersistent log, consumer groups, partitioning, exactly-once semantics, compaction
C librarieslibrdkafka (Confluent, extremely mature, production-grade, high-performance)
ApproachThin fw-lib wrapper around librdkafka — Kafka's wire protocol is complex (API versioning, partition assignment, consumer group coordination). librdkafka is the gold standard.
Effort1 week — producer/consumer wrapper with fwAlloc-friendly API, integration with FiWorks Broker notification engine.

Kafka transforms FIWARE from a push-only notification model to a durable event stream. Subscribers can replay history, new subscribers get backfill, and the broker doesn't need to manage notification retry queues.

gRPC

HTTP/2 Priority: Medium Wraps: grpc-c or custom

Google's high-performance RPC framework based on HTTP/2 and Protocol Buffers. gRPC provides bidirectional streaming, flow control, connection multiplexing, and automatic code generation from .proto files. Used by Kubernetes, Envoy, gRPC-Web, and many cloud-native services.

For FIWARE 2.0, gRPC offers an alternative API surface for performance-sensitive clients — especially in cloud-native environments where gRPC is the expected wire protocol. NGSI-LD operations could be exposed as gRPC services alongside the REST API.

FeatureDetail
TransportHTTP/2 (multiplexed streams, header compression)
SerializationProtocol Buffers (binary, schema-driven, backwards-compatible)
StreamingUnary, server-streaming, client-streaming, bidirectional
C librariesgrpc-c (official C-core), nanopb (protobuf for embedded)
ApproachThin wrapper around grpc-c core for an alternative NGSI-LD API, or custom HTTP/2 + protobuf if fwHttp adds HTTP/2 support.
Effort3–4 weeks — .proto definitions for NGSI-LD operations + integration. HTTP/2 support in fwHttp is a prerequisite.

Server-Sent Events (SSE)

HTTP (long-lived) Priority: High fw-lib: built into fwHttp

SSE is a simple, browser-native mechanism for server-to-client push over a long-lived HTTP connection. No upgrade handshake, no framing complexity — just text/event-stream with data: fields. The browser's EventSource API handles reconnection, last-event-id replay, and parsing automatically.

SSE is already part of the NGSI-LD API specification for subscription notifications. The FiWorks Broker should support it natively — a client subscribes and receives entity updates as SSE events. Simpler than WebSockets for unidirectional server-push.

FeatureDetail
TransportHTTP/1.1 (long-lived response), works through proxies and CDNs
FeaturesAuto-reconnect, last-event-id replay, named event types
ApproachBuilt into fwHttp — SSE is just a long-lived HTTP response with Transfer-Encoding: chunked and Content-Type: text/event-stream. No separate library needed.
Effort<1 week — extend fwHttp connection handling to support long-lived responses with push. Trivial format: data: {json}\n\n.

SSE is the simplest path to real-time browser notifications. Unlike WebSockets, it works through HTTP proxies, CDNs, and load balancers without special configuration. For the common case (server pushes updates to client), SSE is the right tool.

NATS

TCP / TLS / WebSocket Priority: Medium fw-lib: knats

NATS is an ultra-lightweight, cloud-native messaging system. Text-based protocol (like Redis), sub-millisecond latency, publish/subscribe + request/reply + queue groups. JetStream adds persistence, exactly-once delivery, and key-value/object stores. Used by Kubernetes (k3s), Synadia, and cloud-native microservices.

FeatureDetail
TransportTCP (4222), TLS, WebSocket
ProtocolText-based (PUB, SUB, MSG, CONNECT), trivially simple
Key featuresAt-most-once by default, request/reply, queue groups, JetStream for persistence
C librariesnats.c (official C client, maintained by Synadia)
ApproachCustom fw-lib (knats) — NATS core protocol is trivially simple (text-based, ~6 commands). A fw-lib can implement it in ~300 lines. JetStream adds complexity but can be added incrementally.
Effort1 week — core pub/sub/request. +1 week for JetStream persistence.

NATS is an attractive alternative to Kafka for lightweight deployments — a single NATS server binary (10 MB) vs a Kafka+Zookeeper cluster. For edge deployments where a full Kafka is overkill, NATS provides durable messaging with minimal footprint.

Overview

Protocol Summary & Roadmap

Protocol Domain Approach Priority Effort
MQTT / MQTTS IoT fw-lib (kmqtt) Critical 3–4 weeks
WebSockets / WSS IoT / Web fw-lib (kws) Critical 1–2 weeks
CoAP / CoAPS IoT fw-lib (kcoap) High 2–3 weeks
SSE Web / Streaming Built into fwHttp High <1 week
OPC-UA Industrial Wraps open62541 Critical 2–3 weeks
Modbus TCP Industrial fw-lib (kmodbus) High 1 week
DDS Streaming Wraps Fast-DDS High 1–2 weeks
Kafka Streaming Wraps librdkafka High 1 week
AMQP 1.0 IoT / Enterprise Wraps qpid-proton Medium 1–2 weeks
BACnet Building Wraps bacnet-stack Medium 2 weeks
KNX Building fw-lib (kknx) Medium 2 weeks
NATS Streaming fw-lib (knats) Medium 1–2 weeks
gRPC API Wraps grpc-c Medium 3–4 weeks
PROFINET Industrial Wraps p-net Low 2–3 weeks
EtherNet/IP Industrial Wraps OpENer Low 2–3 weeks

Build or Wrap?

The decision for each protocol is simple: if the protocol is simple enough to implement in <1000 lines of C, build a fw-lib from scratch (MQTT, CoAP, Modbus, WebSockets, KNX, NATS, SSE). Full control over memory, threading, and integration with the fw-libs ecosystem.

If the protocol has a 500+ page spec, wrap an existing battle-tested C library (OPC-UA, DDS, Kafka, AMQP, BACnet, gRPC, PROFINET, EtherNet/IP). The complexity is in the spec, not the implementation — and compliance matters more than performance for these protocols.

Total Protocol Effort: ~24–36 weeks

This is not sequential. The critical protocols (MQTT, WebSockets, OPC-UA, SSE) are built first as part of the FiWorks Multi-Agent and FiWorks Broker work. Industrial protocols (Modbus, BACnet, KNX) and streaming protocols (DDS, Kafka, NATS) are added incrementally as FiWorks Multi-Agent plugins.

Most protocol libraries are 1–2 weeks each because they're either simple fw-libs or thin wrappers. The foundation work (fwHttp client, TLS, epoll event loop) is shared across all of them. Once kmqtt is built, kcoap reuses 80% of the I/O infrastructure. Once the FiWorks Multi-Agent plugin interface exists, adding a new protocol is a matter of implementing the codec.

Like all fw-libs, every protocol library is available to application programmers — not just the platform. A developer can use kmqtt or kcoap directly in their C application, with full access to fwAlloc memory management and fwHttp event loops.