SeaweedFS

What it is for

SeaweedFS (Apache 2.0, one binary weed) gives the lake an S3-compatible store. It replaces MinIO Community Edition, which stopped shipping binaries in 2025. Everything else only needs an endpoint, a key pair and path-style S3, so another S3 store could replace it.

Two buckets:

Bucket	Holds	Written by
raw	Original uploads, never modified. Key: vendor/project/folder/YYYY/MM/DD/<uuid>/<name>	The API
lake	Iceberg tables as Parquet plus metadata, under warehouse/	Spark and Trino

Where it lives

Binary	/usr/local/bin/weed (install step 01)
Service	seaweed (step 04), runs weed server in single-node mode
Data	/data/seaweed (step 03.5), owned by lake
Identity file	/etc/lake/seaweed/s3.json (one identity, keys from cluster.env)
Ports	S3 8333, master 9333, filer 8888, volume 8082, all on 127.0.0.1

Check it

systemctl is-active seaweed
export AWS_ACCESS_KEY_ID=<S3_ACCESS_KEY> AWS_SECRET_ACCESS_KEY=<S3_SECRET_KEY> AWS_DEFAULT_REGION=us-east-1
aws --endpoint-url http://127.0.0.1:8333 s3 ls                  # buckets: raw, lake

Web pages (through the tunnel): master http://localhost:19333, filer http://localhost:18888.

How to use it

E="--endpoint-url http://127.0.0.1:8333"
aws $E s3 ls s3://raw/ --recursive --human-readable | tail      # newest originals
aws $E s3 cp s3://raw/<key> ./copy.csv                           # fetch one original
aws $E s3 ls s3://lake/warehouse/ --recursive | head            # table files
aws $E s3 cp ./file.csv s3://raw/test/file.csv                   # test upload

For a one-off look at the cluster itself:

weed shell -master=127.0.0.1:9333       # then: volume.list, cluster.check

Do not delete or move objects under lake/warehouse/ by hand: Iceberg’s metadata would then point at files that are gone. Remove data through the application (Archive or Delete of a file, which expires snapshots properly).

Configure it

Keys, endpoint and bucket names come from cluster.env (S3_ACCESS_KEY, S3_SECRET_KEY, S3_ENDPOINT, RAW_BUCKET, LAKE_BUCKET). Change them there and re-run step 04. Version: SEAWEED_VERSION and SEAWEED_SHA256.

Services that use the keys: the API, the worker/Spark, Trino, Nessie. Each has its own copy in its config file, so after rotating keys re-run the install step of each one.

Things that go wrong

Symptom	Likely cause
aws fails with an empty endpoint	cluster.env not sourced: set -a; source /opt/lake/cluster.env; set +a
Upload returns 502	The service is down or the disk is full: df -h /data, journalctl -u seaweed
Wrong key accepted	It should not be; step 04’s built-in negative test checks this
Spark job cannot read s3a://raw/...	S3A block missing: re-run step 07.5

Production notes

Single node today: no replication. Replication, backup of /data/seaweed and a multi-server layout are planned work. Bind it to the private interface and allow its port only from the other servers.