Data & storage

VOLT splits data by weight. The server keeps metadata, small enough for a relational database. Everything derived from simulation data (dumps, GLB models, preview rasters, Parquet tables, logs, whiteboard state) lives in the object store of a cluster, on the machine whose daemon owns it. Each trajectory records its storage cluster (storageClusterId), and analyses write their artifacts to that same cluster, whichever cluster computed them.

Overview

Server database

Postgres on a server, SQLite in the browser and desktop runtimes (see Database). The tables, from src/core/bootstrap/entities.ts:

team_clusters holds each cluster's credentials (encrypted), status, role, queue settings and last reported host capabilities. storage_placements records which cluster holds what, and cluster_transfer_jobs the moves between clusters. cluster_metric_samples keeps 24 hours of heartbeat metrics.

Server data directory

SERVER_DATA_DIR (/var/lib/volt-server in the containers) holds the files the server serves itself, which today are avatars:

• user-files/<bucket>/<key>, plus <key>.meta.json with the content type. USER_FILES_ROOT moves user-files elsewhere.
• They are served at /api/files/<bucket>/<key>, with absolute URLs built from SERVER_ENDPOINT.

Temporary files go to storage/temp under the server's working directory and are cleaned up by the temp storage lifecycle.

Daemon database

The daemon's database is separate from the server's. On a self-hosted stack it is the Postgres database volt-cluster (the daemon creates it when missing); with no DATABASE_URL it is SQLite at $DAEMON_DATA_DIR/daemon.sqlite, in WAL mode. Its schema is synchronized on every start.

Daemon data directory

DAEMON_DATA_DIR (/var/lib/volt-daemon in the containers, ~/.volt-daemon/data otherwise):

The object store

FilesystemObjectStore (src/shared/storage/FilesystemObjectStore.ts) is a plain directory tree with S3-like semantics: buckets, keys, metadata, listing by prefix, ranged reads and composing parts into one object.

• An object's bytes are at object-store/objects/<BUCKET_PREFIX><bucket>/<key>.
• Its metadata (ETag, content type, content encoding, custom fields) is at object-store/metadata/<BUCKET_PREFIX><bucket>/<key>.json.
• Keys cannot escape their bucket directory.
• The buckets are created on startup. BUCKET_PREFIX is cluster- in the compose stack and the web runtime, so the directories are named cluster-volt-dumps and so on.

Other machines reach the store only through the daemon's object gateway, directly or tunneled.

Deleting a trajectory removes its trajectory-<id>/ prefix from volt-dumps, volt-models, volt-rasterizer and volt-plugins, and its plugins/trajectory-<id>/ prefix. Deleting an analysis removes its analysis-<id>/ prefixes the same way, including its logs.

Compressed dumps and models

Frame dumps and GLB models are stored zstd-compressed (.zst), compressed by the zstd program at level 5 with TRAJECTORY_ZSTD_THREADS threads (2 by default). Models are stored with Content-Encoding: zstd, and the server always streams them compressed: with Content-Encoding: zstd when the request's Accept-Encoding allows it, so the browser decodes them, and otherwise with X-Volt-Resource-Encoding: zstd for the client to decode. Analysis and rasterization jobs decompress a dump or model to a local file before reading it.

Parquet and DuckDB

Tabular data is Apache Parquet, read in place with DuckDB rather than loaded whole:

• trajectory.parquet holds every frame's atoms (timestep, atom_index, id, type, x, y, z, plus every per-atom property the dumps carry), written once all of a trajectory's frames are processed. ParquetTrajectoryFrameStore queries it for frame reads, atom lookups and statistics.
• Plugin exposures are one Parquet file per exposure and timestep, which ParquetPluginPropertyStore and the exposure readers query for per-atom values, listings, color coding and filters.

The daemon reads them with in-memory DuckDB connections (@duckdb/node-api) against local copies in its Parquet caches. Plugins write theirs with the DuckDB built into CoreToolkit, whose memory the daemon caps with VOLT_DUCKDB_MEMORY_LIMIT_MB (25% of RAM, between 1 GB and 16 GB, unless set).

Whiteboards and notebooks

• Whiteboards. The whiteboards row holds the metadata and payloadKey; the board's state is state.json in volt-whiteboards on a cluster, and uploaded assets sit next to it.
• Notebooks. A notebook's cells are in the server database (scripting_notebooks.content). On a server, opening one copies the content into its Jupyter container on a cluster. In the browser runtime, kernels keep their environments and working files in the OPFS directory volt-notebooks.

Docker volumes

The single-host stack (packages/apps/desktop/stack/compose.yml, project volt):

The install files the server generates for another machine define postgres-data (that cluster's volt-cluster database) and daemon-data (its DAEMON_DATA_DIR). Back up the database volumes and the daemon data volumes together: the server's records point at objects in the daemons' stores.

In the browser and the desktop app

The web runtime keeps everything in the origin private file system (OPFS):

A browser may evict an origin's storage when space runs low, and clearing site data deletes the workspace. The desktop app's origin is its own persistent session (persist:volt) under the app's user data. See Web runtime.