| Admin |
A login with the realm role admin: runs the platform. |
| Analyst |
A login with the role analyst: reads assigned data, runs SQL, makes views and APIs. |
| API key |
A secret that opens published APIs (chdc_ + 48 hex). Shown once, only its hash is stored. |
| Bare metal |
Software installed directly on servers (systemd services), without Docker or Kubernetes. |
| Bronze |
Layer 1: values as text, as received, with lineage columns. |
| Bucket |
A top-level container in S3 storage. We use raw and lake. |
| Catalog |
The list of tables and their current version. Ours is Nessie, named lake in Spark and Trino. |
cluster.env |
The single settings file for all servers. Never committed. |
| Dataset |
The tables of one folder across the three layers. |
| Folder |
The place a vendor uploads into; one folder is one dataset. |
| Gold |
Layer 3: distinct rows across all files of a folder. |
| Iceberg |
The table format used for all lake tables (Parquet files plus metadata). |
| Keycloak |
The login and roles server. |
| Lineage columns |
_file_id, _source_file, _ingested_at, _row_id in bronze. They say which file and row a value came from. |
| Nessie |
The Iceberg catalog service; keeps a commit history of table changes. |
| Parquet |
The columnar file format that holds table data. |
| Project |
A vendor’s unit of work; contains folders. |
| Published API |
A view served live at /api/v1/data/<name>. |
| Push API |
The vendor-facing API at /api/v1/push/* for creating projects/folders and sending files by machine. |
| Push key |
A secret for the push API (chdcp_ + 48 hex), scoped to a vendor, project or folder. |
| Quarantined |
A file that was stored but is a type the worker cannot process yet. |
| Raw bucket |
Where original uploads live, unmodified. |
| Registry |
The ingest database: vendors, projects, folders, files, events, views, keys, audit. |
| Role |
One of vendor, analyst, admin. |
| S3 |
The object-storage protocol SeaweedFS speaks. |
| Silver |
Layer 2: cleaned and typed rows. |
| Stage |
Where a processing file is: starting, reading, bronze, silver, gold, done. |
| systemd unit |
The service definition that starts, stops and restarts a program. |
| Tunnel |
An SSH port forward from your machine to a service that listens only on the server’s localhost. |
| Vendor |
A company that sends data. Has logins, projects and folders. |
| View |
A saved analyst query whose result is materialized through bronze, silver and gold. |
| Worker |
The Python service that runs Spark for each uploaded file. |