CDG / ENGINEERINGContactSign in ↗

Cybersecurity

Athernex

I built a Rust security information and event management system: from event ingestion and correlation to investigations, evidence retention, and bounded agent authority.

Research applicationoriginal systempublic

ORIGINAL CONTRIBUTIONS

What I built.

  • Built a Rust SIEM with validated event ingestion, event normalization, five correlation rules, and evidence-backed investigations.
  • Implemented separate operator authentication and task-scoped agent capabilities with expiry and use limits.
  • Designed compact event encoding, bounded retention, replay protection, and inspectable storage accounting.

Athernex / SYSTEM MAP

How this system connects.

Collector &ingest1Rust eventengine2SQLite evidenceledger3Capabilitygateway4
  1. 01 / Collector & ingest: Normalize permitted event inputs.
  2. 02 / Rust event engine: Validate events and correlate findings.
  3. 03 / SQLite evidence ledger: Keep inspectable evidence under retention budgets.
  4. 04 / Capability gateway: Allow agents only task-scoped evidence access.
Directed connections
  • 1 → 2: validated event batch (implemented)
  • 2 → 3: atomic persistence / retained blocks (implemented)
  • 4 → 3: scoped evidence reads via engine (implemented)

SYSTEMS DESIGN & OWNERSHIP

Engineering below the surface.

A security engine from admission to investigation

Implemented the Rust pipeline from normalized event admission through detection, correlation, findings and investigation. The design gives event identity, provenance and evidence retention explicit representations so a finding can be traced back to the records that produced it.

Persistence and replay are part of the trust model

SQLite WAL and transactional batches protect admitted events. Compact AEF1 encoding and compressed retained blocks trade encoding complexity for explicit storage accounting. The replay-identity repair prevents different source/key pairs from aliasing and preserves deduplication after archive restoration.

Agent capability is narrower than operator authority

Task-bound grants constrain the resource, action, lifetime and use count available to an agent. Authenticated operator and CSRF boundaries remain separate. HTTP tests exercise concurrent consumption of the final permitted use rather than relying only on a UI check.

EXPLORE THE ARCHITECTURE

Components. Contracts. Consequences.

ATHERNEX · SINGLE LOCAL TRUST DOMAIN

01 / Collector & ingest

Normalize permitted event inputs.

Implementation

journald parsing or validated InputEvent batches enter the engine.

Security & failure behavior

Trusted collector provenance differs from token-scoped remote ingest. Invalid fields, oversized batches and unauthenticated API inputs are rejected.

Tradeoff

Bounded schema limits payload flexibility and makes field budgets explicit.

02 / Rust event engine

Validate events and correlate findings.

Implementation

Transactions persist labels, payloads and idempotency hashes; detection rules evaluate retained events.

Security & failure behavior

Conflicting replay facts fail atomically. A failed batch rolls back; storage limits surface errors.

Tradeoff

Single-process serialization simplifies correctness; horizontal scale is unverified.

03 / SQLite evidence ledger

Keep inspectable evidence under retention budgets.

Implementation

SQLite WAL, compact AEF1 payloads and compressed retained blocks.

Security & failure behavior

Evidence hashes bind response proposals to their evidence. Unflushed telemetry is memory-only; full storage fails closed.

Tradeoff

Payload compression is not equivalent to total on-disk storage savings.

04 / Capability gateway

Allow agents only task-scoped evidence access.

Implementation

Grant digest lookup, expiry and use limits checked in a transaction.

Security & failure behavior

Agent grants never authorize operator console access or host tools. Expired, revoked, exhausted or mis-scoped grants deny access.

Tradeoff

Local task grants are not a general distributed identity provider.

Open full 2D architecture →

THE WORKING APPLICATION

From signal to investigation.

Actual Athernex Rust SIEM console displaying a controlled synthetic investigation environment
Actual Rust console · controlled synthetic fixtures · source baseline b5623df
Athernex incident investigation, analyst decisions and preserved evidence
Preserved evidence and analyst workflow · synthetic public fixture

TECHNICAL CASE STUDY

Decisions behind the system.

Engineering problem

Turn bounded host and synthetic security telemetry into inspectable findings while keeping agent permissions separate from operator authority.

Requirements & constraints

Atomic event batches, explicit replay handling, bounded memory and storage, and an investigation path that never grants an agent host authority.

Architecture & technologies

Rust / Axum HTTP; SQLite WAL ledger; AEF1 varint event encoding; flate2 retained blocks; SHA-256 evidence hashes; journald collector. The replay uses an in-memory engine.

Security & trust boundaries

Operator authentication and CSRF protect console mutations. Task-bound grants have resource, action, lifetime and use-count limits. Ingest identities cannot impersonate trusted journal collectors.

Original engineering

Repository history records charlesderek implementing the SIEM engine, normalized event codec, durable admission and console. Scope is this independently implemented subsystem; dependencies retain their own authorship.

Working implementation

Batch validation, idempotency fencing, five detection rules, evidence-backed findings, proposal review, bounded retention, scoped capabilities, and an operational console are present in source.

Experiments & missing capabilities

Datacenter orchestration and research screens do not establish production orchestration. No production scale or load test is claimed. Cross-platform host collection is not verified by this replay.

Failure modes & concurrency

Invalid batches and conflicting replays roll back; exhausted capability grants deny access. SQLite transactions serialize ledger changes. HTTP tests exercise simultaneous last-use consumption.

Testing & evidence

Baseline executed locally: 39 library tests and 9 HTTP integration tests passed on Linux. Fixed-clock Rust replay exports 12 synthetic events, correlated findings, and allow/deny capability decisions.

Observable improvement

This cycle replaces ambiguous source:key replay identities with an unambiguous separator and normalizes legacy retained identities on restore. Colon-containing source/key pairs previously collided. Regression and migration tests verify independent identities and retained replay protection.

Deployment model

Single local Rust process with SQLite and a local operator console. The public portfolio hosts captured fixture output; no production agent or operator endpoint is exposed.

Decisions & alternatives

A bounded single-ledger model makes transactional behavior inspectable. A distributed event bus would add availability and ordering tradeoffs that this demonstration does not validate.

Demonstration & limitations

Use the Evidence Lab below to inspect actual replay output. Data is synthetic; no host journald collection, privileged response, or production performance claim is part of this demonstration.

ENGINEERING SCOPE

Clear boundaries.

Verified original Rust SIEM implementation scope; third-party dependencies excluded.

  • Single-host research application; production scale and load capacity have not been established.
  • Agent grants do not authorize arbitrary host actions. Public replay uses synthetic data.