← Home

The JSONL contract

Stream.

One JSON object per line, terminated by \n. Same records on every sink — file (-o FILE), UNIX-domain socket, or TCP (--data-socket SPEC). Read-only by design: sloth never reads from a connected client. Read-only is a statement about commands, not data — the stream carries no authentication and no encryption, so reachability is the access control. It stays local unless you explicitly say otherwise.

Three transports, same payload.

Each line is identical regardless of sink. -o FILE and --data-socket SPEC can both run at once and broadcast every record to every active sink.

TransportConfigured byUse
File (append) -o /var/log/sloth.jsonl log forwarder pulls / tail -f pipelines
UNIX-domain (recommended) --data-socket unix:/run/sloth/sloth.sock local consumer on the same host. Created 0600: the kernel checks the peer's credentials on connect(), so filesystem permissions are the authentication — no keys, nothing to rotate or expire
TCP (loopback) --data-socket tcp:127.0.0.1:8765 the default. Any local user on the host can read it; nothing off-host can
TCP (routable) --data-socket tcp:HOST:PORT --data-socket-allow-remote opt-in only. Refused without the flag, and warns when used — anyone who can reach the port reads every observation. Prefer a tunnel

Exposure guard. Binding any address outside 127.0.0.0/8 — including the 0.0.0.0 wildcard — is refused unless --data-socket-allow-remote is passed, and the refusal names what to add. The whole loopback /8 binds freely, and unix: paths are unaffected, so no working configuration changed. To reach the socket from another host, keep sloth on loopback and move the bytes with something built for it: ssh -N -L 8765:127.0.0.1:8765 user@sensor, the mutual-TLS stunnel templates in examples/stunnel/, or push outbound to a SIEM with examples/forwarder/. Full trust boundary: docs/wiki/data-socket-exposure.md.

Backpressure. The socket writer is non-blocking and delivers whole records or none. A slow client gets a bounded queue (512 KiB) of complete lines; a record that has started is always finished before the next. On overflow sloth drops whole incoming records and sends a socket_gap record (seq, dropped, dropped_total) before the next line it delivers. A client that accepts no bytes for 30 s, or whose pipe breaks, is disconnected; reconnect to resume. Discard any unterminated bytes before EOF.

Seven record types.

Every line is a JSON object with at least type and ts (Unix seconds). Fields are append-only; consumers ignore unknown type values and unknown keys gracefully.

json dns 
{"type":"dns","ts":1700000000,
 "src":"192.168.1.5",
 "qname":"example.com",
 "qtype":"A",
 "answer":"93.184.216.34",
 "is_resp":1}
json tls 
{"type":"tls","ts":1700000001,
 "src":"10.0.0.5","dst":"93.184.216.34",
 "host":"example.com",
 "ver":"TLS 1.3",
 "ja3":"deadbeefcafef00d…"}
json quic 
{"type":"quic","ts":1700000002,
 "src":"10.0.0.5","dst":"1.1.1.1",
 "host":"cloudflare.com",
 "ver":"v1"}
json http 
{"type":"http","ts":1700000003,
 "src":"10.0.0.5",
 "host":"example.com",
 "method":"GET",
 "path":"/index.html"}
json ntp 
{"type":"ntp","ts":1700000004,
 "src":"10.0.0.1","dst":"192.168.1.5",
 "mode":"server",
 "version":4,"stratum":1,"ref":"GPS"}
json icmp 
{"type":"icmp","ts":1700000005,
 "src":"192.168.1.5","dst":"8.8.8.8",
 "desc":"Echo Req",
 "ty":8,"code":0,"seq":42,"v6":0}
json alert 
{"type":"alert","ts":1700000006,
 "title":"THREAT_DOMAIN",
 "detail":"192.168.1.5 queried …",
 "key":"threat-d:malware.testing.com",
 "sev":2,"ty":3,"count":1,
 "incident_id":"9f2c41ab77e30d58"}
json alert.escalate 
{"type":"alert.escalate","ts":1700000041,
 "event_id":"9f2c41ab77e30d58-0002",
 "incident_id":"9f2c41ab77e30d58",
 "key":"mgmtfuzz:ba:ad:f0:0d:00:01",
 "title":"MGMT_FUZZ","detail":"…",
 "sev":2,"ty":31,"prev_sev":1,
 "observations":2,"evaluations":36,"count":36}

sev is the severity tier: 0 = LOW (yellow), 1 = WARN (orange), 2 = CRIT (red). The enum is stable. ts on alerts is the last_evaluated time of the dedup key, not the first observation.

The alert record is written only when a dedup key is new. Everything that happens to an alert afterwards — a WARN → CRIT escalation, changed evidence, expiry — rides the alert.create / alert.update / alert.escalate / alert.resolve lifecycle records added in #98, so a pipeline that pages on CRIT must read alert.escalate. They join on incident_id, which is stable from create to resolve. An incident resolves after 300 s in which no rule re-asserted its key (monotonic clock), and nothing is emitted for an evaluation that re-renders identical evidence — a retained condition held across 100 polls is one create and silence. count and evaluations are rule ticks, not packets or incidents; observations only moves when the evidence does.

Reference consumer.

examples/consumer/sloth-stream.py is the textbook consumer loop — connect, stream, parse, filter, reconnect. Stdlib only. Read it as a template for porting to Go, Node, or Swift's Network.framework; the structure ports directly.

sh Tail every record, ANSI colour 
python3 examples/consumer/sloth-stream.py \
    unix:/run/sloth/sloth.sock
sh Alert lifecycle, over Tailscale 
python3 examples/consumer/sloth-stream.py \
    tcp:100.64.0.5:8765 \
    --type alert.create,alert.escalate,\
alert.update,alert.resolve
sh Raw JSON into jq 
python3 examples/consumer/sloth-stream.py \
    unix:/run/sloth/sloth.sock --raw | jq .
sh Type tally every 5s 
python3 examples/consumer/sloth-stream.py \
    unix:/run/sloth/sloth.sock --count

Reference SIEM forwarder.

examples/forwarder/sloth-forward.py reads from the data socket, batches, and pushes to a downstream SIEM. Three sinks ship; the sink interface is two members (.name, .send(batch)) so adding Loki, Datadog, or your in-house collector is ~30 lines.

sh Splunk HEC 
python3 examples/forwarder/sloth-forward.py \
    unix:/run/sloth/sloth.sock \
    --sink hec \
    --hec-url       https://splunk.example.com:8088/services/collector \
    --hec-token-env SLOTH_HEC_TOKEN
sh Elasticsearch (daily rolled) 
python3 examples/forwarder/sloth-forward.py \
    unix:/run/sloth/sloth.sock \
    --sink elastic \
    --es-url       https://elastic.example.com:9200 \
    --es-index     'sloth-events-%Y.%m.%d' \
    --es-api-key-env SLOTH_ES_API_KEY
sh RFC 5424 syslog (UDP) 
python3 examples/forwarder/sloth-forward.py \
    unix:/run/sloth/sloth.sock \
    --sink syslog \
    --syslog-host siem.example.com \
    --syslog-port 514
sh Alerts-only feed, big batches 
python3 examples/forwarder/sloth-forward.py \
    unix:/run/sloth/sloth.sock \
    --type alert \
    --batch-size 500 --batch-ms 500 \
    --sink hec --hec-url … --hec-token-env …
DELIVERY

Non-durable by design.

The data socket is non-durable, and the forwarder doesn't pretend otherwise. When the downstream sink fails after --max-retries, the batch is dropped and a count is logged to stderr.

If your deployment needs durability, also pass -o /var/log/sloth.jsonl to sloth and ship the file with a log-shipping agent (filebeat, vector, fluent-bit) on a separate path. That pipeline gives you durability; this one gives you low-latency triage.

The stats line (every --stats-interval seconds, default 30s) is the operator's ground truth: received ≥ forwarded + dropped. A non-zero dropped over a long window means the sink can't keep up.