PRISM Edge Collector (prism-plc)

Why it exists

The turbines publish everything an operator needs, but only to the OEM's HMI. This service sits on the OT network, polls the PLCs directly over Ethernet/IP or OPC UA, and turns that stream into something an open platform can use. It is also where PRISM's data contract lives, and CI in both this repo and the dashboard repo checks the contract against each other.

Components

• Collector in Python 3.11 on asyncio, one task per PLC, up to five PLCs. Protocol abstraction through a base connection, a factory, and Ethernet/IP and OPC UA implementations.
• API server in Node and Express with JWT validation against Authentik, L5X parsing, uploads and ZIP downloads.
• Web UI in React 18 and Vite with TanStack Query and Playwright end-to-end tests.
• Storage in ClickHouse 24.8, with InfluxDB 2.7 still supported. The ClickHouse writer is a duck-typed drop-in for the Influx writer, selected by one setting.
• ch-replicator, a separate Python service with Prometheus metrics that ships the OT buffer to the central cluster.

Five Docker images plus cloudflared. Deployed to the Windows OT host by Ansible over WinRM, with a PowerShell script as the imperative fallback.

The trigger-log subsystem

Operators rely on high-resolution logs around events: a trip, a start, a threshold breach. The subsystem keeps a pre- and post-event ring buffer, evaluates five rule modes (rising edge, falling edge, state change, threshold high, threshold low), and hands captures to four pluggable writers: CSV, JSON, a SQLite index, and VTHMI binary.

The VTHMI writer produces files bit-compatible with the OEM HMI's Trigger.log: a 192-byte header, 300 tag definitions of 1,496 bytes each, and 1,920-byte records holding a Windows FILETIME and 300 little-endian floats. It was written once in JavaScript and then ported to Python, and it is covered by thirteen dedicated test files. The orchestrator owns no asyncio task of its own; the collector wires its fast tick loop and slow flush loop, and the clock and event-id factory are injectable, so the whole thing is deterministically testable with a fake clock.

Alarms from descriptions

No alarm database existed. The alarm service classifies tags as alarm candidates by matching name and description against patterns like alarm, fault, trip, shutdown, lockout and emergency, infers severity, then maintains active-alarm state and a bounded event log.

The replicator's batch-boundary bug

The collector writes every point in a poll cycle with the same timestamp, so many rows share one time. A limit-bounded read can cut through the middle of a timestamp, and advancing the watermark past it would silently drop the unread rows. The replicator trims trailing rows that share the batch's maximum timestamp and lets the next round pick them up, and a safety-lag filter guarantees a timestamp is fully flushed before it is ever read. The watermark is persisted atomically on a Docker volume. During a WAN outage the replicator simply stops advancing, and the OT buffer's 30-day retention is the queue. Nothing is lost and there is no message broker to run.

ClickHouse schema, with reasons

Timestamps use DateTime64(9) with DoubleDelta and ZSTD codecs. Float values use Gorilla, which is purpose-built for slowly changing telemetry. All four tag columns are LowCardinality(String). Ordering is by PLC, tag and time, with monthly partitions and a 30-day TTL. Column order is pinned in three places with lock-step comments.

Security details

The API middleware uses the socket's remote address rather than the framework's req.ip for CIDR source checks, because the latter honours X-Forwarded-For when trust-proxy is on and could be spoofed. It has a hand-rolled IPv4 CIDR matcher, IPv4-mapped-IPv6 normalisation, and constant-time service-token comparison. Service tokens accept a comma-separated list so rotation windows can overlap. A route policy matrix gives a read-only engineer role GET access plus an explicit allowlist of non-GET paths.