FiWorks NGSIv2 Broker

Why a Separate NGSIv2 Broker?

NGSIv2 is the predecessor to NGSI-LD and remains widely deployed. Many existing FIWARE deployments, IoT devices, and third-party integrations still speak NGSIv2. Rather than forcing a migration, the FiWorks NGSIv2 Broker provides native NGSIv2 support as a dedicated component.

The key architectural decision: both the NGSI-LD and NGSIv2 brokers share the same MongoDB storage backend. An entity created via the NGSIv2 API is immediately visible through the NGSI-LD API, and vice versa. This means deployments can run both protocols simultaneously without data duplication or synchronization overhead.

Shared Storage, Separate APIs

The NGSIv2 and NGSI-LD brokers are separate processes listening on different ports, but both read from and write to the same MongoDB collections. The fwNgsiv2 library handles the translation between NGSIv2 JSON format and the internal FtNode data model that MongoDB stores.

Aspect	NGSI-LD Broker	NGSIv2 Broker
API	NGSI-LD (ETSI GS CIM 009)	NGSIv2 (Telefonica/FIWARE)
@context	Full JSON-LD semantics via fwJsonld	None — plain attribute names
Validation	fwNgsild	fwNgsiv2
Representation formats	Normalized, concise, simplified	Normalized, keyValues, values, unique
Storage	Shared MongoDB
HTTP server	fwHttp (epoll, SO_REUSEPORT)

NGSIv2 API Coverage

Entity Operations

• POST /v2/entities — create entity (with options=upsert support)
• GET /v2/entities — query entities with filters (q, mq, georel, geometry, coords, orderBy)
• GET/PATCH/PUT/DELETE /v2/entities/{id}/attrs — full CRUD on entity attributes
• GET/PUT/DELETE /v2/entities/{id}/attrs/{name} — individual attribute operations
• GET/PUT /v2/entities/{id}/attrs/{name}/value — direct value access

Batch Operations

• POST /v2/op/update — batch create/update/delete with actionType
• POST /v2/op/query — batch query with POST body

Subscriptions & Registrations

• Full CRUD on /v2/subscriptions and /v2/registrations
• Notification formats: normalized, keyValues
• Custom notifications via httpCustom with template support

Strict URL Parameter Validation

Unlike other NGSIv2 implementations that silently ignore unsupported URL parameters, the FiWorks NGSIv2 Broker rejects unknown parameters with a 400 BadRequest. Each endpoint has a hardcoded list of allowed parameters (implemented in fwNgsiv2). A typo like ?limt=10 instead of ?limit=10 returns an immediate error rather than silently returning unfiltered results.

What Gets Checked

All validation is handled by the fwNgsiv2 library, shared between the broker and any other component that processes NGSIv2 payloads:

• Forbidden characters — < > " ' = ; ( ) rejected in entity IDs, types, attribute names, and metadata names
• Entity structure — id and type mandatory on create, forbidden on update (they come from the URL)
• Attribute structure — value mandatory, only value/type/metadata fields allowed
• Subscription structure — subject/notification required, http/httpCustom mutual exclusion, attrs/exceptAttrs mutual exclusion
• Registration structure — dataProvided/provider required, provider.http.url mandatory
• Geo queries — georel, geometry, coords must all be present together, valid geometry types enforced

Building Blocks

Library	Role in the NGSIv2 Broker
fwHttp	HTTP server — epoll, zero-copy, SO_REUSEPORT multi-process scaling
fwNgsiv2	NGSIv2 payload validation, URL parameter checking, format conversion
fwJson	JSON parsing — zero-copy, in-place, produces FtNode tree
fwHash	Subscription matching, query caches, attribute type maps
fwAlloc	Bump allocator — O(1) alloc, bulk free per request
fwTrace	Structured logging with 6400 trace levels
fwProm	Built-in Prometheus metrics — no sidecar needed

Estimated Effort

Estimated effort: 1–2 months with Claude Max. The NGSIv2 API is simpler than NGSI-LD (no @context, no linked-data semantics), and the validation library (fwNgsiv2) is already implemented. The main work is the service routines for each endpoint and the MongoDB query translation layer.

Expected Performance vs Orion

Orion (Telefonica's NGSIv2 broker) is implemented in C++ but is single-threaded, uses libmicrohttpd for HTTP, and carries significant legacy overhead from its NGSIv1 origins. The FiWorks NGSIv2 Broker eliminates all of that:

Metric	Orion (measured)	FiWorks NGSIv2 Broker (projected)
Single-core throughput	~3,000–4,000 req/s	~10,000–15,000 req/s
System throughput (same HW)	~3,000–4,000 req/s (single thread)	~15,000–30,000 req/s (multi-core via SO_REUSEPORT)
p99 latency	~10–30ms	~1–5ms
RAM (broker process)	~200–400 MB	~30–100 MB

The single-core 3–4x comes from fwHttp (epoll, zero-copy) replacing libmicrohttpd, and from a clean FtNode-native data model with no legacy NGSIv1 conversions. The system-wide 5–8x comes from multi-core scaling — Orion is fundamentally single-threaded, while the FiWorks broker uses all available cores.