residual/rechat
Watch
3
0
Fork
You've already forked rechat
 0
Code Issues 12 Pull requests Projects Releases Packages Wiki Activity Actions

Design: storage architecture #5

New issue
Open
opened 2026-10-16 17:06:36 +00:00 by muse-lantern · 0 comments
muse-lantern commented 2026-10-16 17:06:36 +00:00
Member
Copy link

proj-rechat-storage

Backend storage design (decided 2026-10-16).

Principles

  • No single SQLite file for everything (blast radius, backup granularity)
  • Pure Go, no cgo (modernc.org/sqlite) — static binary requirement
  • Messages rotate to compressed archives; files have lifecycle policies

Layout

data/
  users.db                  # accounts, credentials, profiles, client_keys
  servers/
    <uuid>/
      hot.db                # recent messages, WAL mode (~30d window)
      archive/
        2026-09.jsonl.zst   # monthly, immutable, compressed
  files/
    index.db                # hash → metadata + lifecycle state
    hot/<hash>.ext
    archive/<hash>.ext.zst

Message lifecycle

  • hot.db keeps rolling ~30d window (configurable per-server?)
  • Janitor moves older messages to monthly compressed JSONL archives
  • UUIDv7 IDs are time-ordered → ORDER BY id is chronological, WHERE id > ? is pagination

File lifecycle

  • HOT → 30d no download + not needed as seeder → ARCHIVED (compressed)
  • ARCHIVED → 90d → PRUNED (deleted, magnet 404s)
  • Swarm healthy + zero downloads → EXTERNAL_ONLY (server drops copy early)

users.db tables

  • users(uid, nick, credential, profile, created_at)
  • client_keys(uid, pubkey, device_name, created_at, last_seen, status)

Open

  • Message hot window: 30d default? Per-server configurable?
proj-rechat-storage Backend storage design (decided 2026-10-16). ## Principles - No single SQLite file for everything (blast radius, backup granularity) - Pure Go, no cgo (modernc.org/sqlite) — static binary requirement - Messages rotate to compressed archives; files have lifecycle policies ## Layout ``` data/ users.db # accounts, credentials, profiles, client_keys servers/ <uuid>/ hot.db # recent messages, WAL mode (~30d window) archive/ 2026-09.jsonl.zst # monthly, immutable, compressed files/ index.db # hash → metadata + lifecycle state hot/<hash>.ext archive/<hash>.ext.zst ``` ## Message lifecycle - hot.db keeps rolling ~30d window (configurable per-server?) - Janitor moves older messages to monthly compressed JSONL archives - UUIDv7 IDs are time-ordered → ORDER BY id is chronological, WHERE id > ? is pagination ## File lifecycle - HOT → 30d no download + not needed as seeder → ARCHIVED (compressed) - ARCHIVED → 90d → PRUNED (deleted, magnet 404s) - Swarm healthy + zero downloads → EXTERNAL_ONLY (server drops copy early) ## users.db tables - users(uid, nick, credential, profile, created_at) - client_keys(uid, pubkey, device_name, created_at, last_seen, status) ## Open - Message hot window: 30d default? Per-server configurable?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
residual/rechat#5
No description provided.