fwTrace — Structured Logging & Tracing

Overview

What It Does

fwTrace provides structured logging and tracing for all FiWorks components. It is designed for production systems where performance matters: trace levels that are disabled cost nothing beyond a single bitmap check, while enabled levels produce fully timestamped, atomic log lines routed to file, screen, or both.

Near-zero overhead when trace levels are disabled — each log call checks a bitmap bit before doing any work; disabled levels compile down to a branch-predict-friendly no-op
8 log types: Fatal (X), Error (E), Warning (W), Info (I), Verbose (V), Trace (T), Debug (D), Fixme (F)
Fine-grained trace levels — up to 6400 individual levels (200 × 32-bit bitmap), each independently switchable at runtime
writev()-based atomic log lines — a single writev() call assembles header and message; no interleaving in multi-threaded code
Timestamps relative to process start — compact, easy to diff across runs
Log to file, screen, or both — configured at ktInit() time
Thread-safe — mutex-protected writes ensure clean output under concurrency

Macros

KT_E(...) KT_W(...) KT_I(...) KT_V(...) KT_T(level, ...) KT_X(exitCode, ...) KT_RE(retVal, ...) KT_RVE(...)

KT_T takes a numeric trace level so the bitmap check can gate it. KT_X logs a fatal message and exits with the supplied exit code. KT_RE and KT_RVE log an error and return a value in one step — a common pattern in error paths.

API Reference

Initialization

ktInit

int ktInit(
    const char* progName,    // Program name (used for log filename)
    const char* logDir,      // Log directory (NULL for stdout only)
    bool       logToScreen, // Echo to stdout in addition to file
    const char* logLevel,    // "ERR", "WARN", "INFO", "VERBOSE", "TRACE", "DEBUG"
    const char* traceLevels, // Trace levels: "5,10,15-20,100"
    bool       verbose,     // Enable verbose output
    bool       debug,       // Enable debug output
    bool       fixme        // Enable FIXME markers
);

Initializes the logging system. Creates <logDir>/<progName>.log, backing up any existing log to .old. Pass NULL for logDir to log to stdout only.

Trace Level Control

void  ktTraceLevelSet(const char* levelString);  // "5,10,15-20"
void  ktTraceLevelSetOne(int level);             // Single level (0-3199)
bool ktTraceLevelCheck(unsigned int level);     // Check if level is enabled

Trace levels can be set as individual values, ranges, or combinations:

ktTraceLevelSet("5,10,100");         // Individual levels
ktTraceLevelSet("15-20");            // Range
ktTraceLevelSet("5,10,15-20,100-150"); // Combined

Global Variables

extern bool ktInfo;    // Info messages enabled
extern bool ktVerbose; // Verbose messages enabled
extern bool ktDebug;   // Debug messages enabled
extern bool ktFixme;   // FIXME messages enabled

Log Line Format

<TYPE>: <TIMESTAMP>: <FILE>[<LINE>]: <FUNCTION>: <MESSAGE>

E: 000012.345: main.c[42]: processRequest: Connection refused
T: 000012.346: parser.c[100]: parse: entering parse loop (5)

Timestamps are microsecond-precision relative to process start, making it easy to diff across runs. The trace level number appears in parentheses at the end of T: lines.

Design

Library Logging via Callbacks (FLOG)

Libraries such as fwJson, fwAlloc, and fwHttp must not depend on fwTrace directly. Requiring fwTrace as a dependency of every leaf library would create a coupling problem: a standalone JSON parser should not drag in a logging framework, and the logging framework should not be initialized before the library is usable.

Instead, libraries log through a callback function pointer defined in fwBase. The callback is NULL by default, which means library log calls are silently dropped unless an application has registered one.

The Callback Interface

// fwBase/fwLibLog.h
typedef void (*KLogFunction)(int severity, int level,
    const char* fileName, int lineNo, const char* functionName,
    const char* format, ...);

extern KLogFunction kLogFunction;

Libraries use FLOG_* macros: FLOG_E, FLOG_W, FLOG_I, FLOG_V, FLOG_T, FLOG_X, FLOG_RE, FLOG_RVE. These check if (kLogFunction != NULL) before calling, so they cost a single pointer comparison when no callback is registered.

Severity Mapping

Severity value	Log type	Notes
`1`	Error (E)
`2`	Warning (W)
`3`	Info (I)
`4`	Verbose (V)
`5`	Trace (T)	`level` parameter carries the trace level number
`7`	Fatal (X)	`level` parameter carries the exit code

Registering the Callback

The application calls kInit(myCallback) at startup. The callback typically routes to ktOut():

static void libLogFunction(int severity, int level,
    const char* fileName, int lineNo, const char* functionName,
    const char* format, ...)
{
    va_list args;
    va_start(args, format);
    vsnprintf(buf, sizeof(buf), format, args);
    va_end(args);

    if      (severity == 1) ktOut(fileName, lineNo, functionName, 'E', -1,    "%s", buf);
    else if (severity == 2) ktOut(fileName, lineNo, functionName, 'W', -1,    "%s", buf);
    else if (severity == 3) ktOut(fileName, lineNo, functionName, 'I', -1,    "%s", buf);
    else if (severity == 4) ktOut(fileName, lineNo, functionName, 'V', -1,    "%s", buf);
    else if (severity == 5) ktOut(fileName, lineNo, functionName, 'T', level, "%s", buf);
    else if (severity == 7) ktOut(fileName, lineNo, functionName, 'X', level, "%s", buf);
}

Standalone Fallback

A default stderr fallback is provided in fwBase. If no callback is registered, FLOG_* outputs go to stderr with a simple format. This means libraries work fully standalone without fwTrace — useful for unit tests, tools, and any context where the full logging stack is not initialized.

Important: fwBase and fwTrace must never use FLOG_* macros. These two components sit below the callback layer; using FLOG_* inside them would risk infinite recursion.

Reference

Trace Level Allocation Table

Each library gets a dedicated range of trace level numbers to avoid collisions across the stack. Enabling level 42 in an application turns on exactly one library's trace output, with no crosstalk.

Library	Range	Size	Description
fwBase	0–19	20	Reserved
fwAlloc	20–39	20	Memory allocation
fwJson	40–79	40	JSON parsing/rendering
fwHash	80–89	10	Hash tables (reserved)
fwProm	90–99	10	Prometheus metrics (reserved)
fwHttp	100–149	50	HTTP server & client
fwJsonld	150–199	50	JSON-LD @context
fwNgsild	200–399	200	NGSI-LD validation
kdb	400–899	500	Database engine
kipc	900–999	100	Inter-process communication
fwNgsiv2	1000–1049	50	NGSIv2 validation

App-level trace levels (e.g., Orion-LD uses 1000+) live above the library ranges. fwTrace supports up to 6400 levels.

Architecture

Layering

The logging stack is strictly layered. Higher layers depend on lower ones; lower layers never depend on higher ones. The callback mechanism is the only crossing point between the library layer and the fwTrace layer.

Layer	Components	Logging mechanism
Application	Orion-LD, FiWorks Broker, etc.	Calls `ktInit()` and `kInit(callback)` at startup; uses `KT_*` macros directly for its own log lines
Libraries	fwJson, fwAlloc, fwHttp, fwNgsild, …	Use `FLOG_*` macros only — no direct fwTrace dependency
fwBase	fwLibLog.h, kInit.c	Defines the callback infrastructure; provides the stderr fallback; must not use `FLOG_*`
fwTrace	ktOut, file I/O, bitmap, timestamps	Actual log output; must not use `FLOG_*`

At startup the application registers its callback once via kInit(). From that point on, any FLOG_* call anywhere in the library stack flows through the callback into ktOut(), which applies the bitmap check, formats the header, and emits a single atomic writev() to the configured output destination(s).