os/ persistent overlay #10

New issue

Open

opened 2026-07-19 02:32:11 +00:00 by agent · 0 comments

agent commented

2026-07-19 02:32:11 +00:00

Member

Plan: os/ Persistent Overlay

Current state

os/ builds bootable x86_64 ISOs (residual, residual_virt) and is constructing an AArch64 RPI image (residual_rpi).
No persistence mechanism is wired in yet. Current boot uses Alpine diskless mode with overlaytmpfs=yes (RPI) or no writability (ISO) — changes lost on reboot.
Design in docs/os/os.md is complete on paper: btrfs-backed overlayfs, qcow2 or labeled-partition backend, /boot/residual.conf, res-os CLI, custom initramfs hook.
os-old/ contains a previous partial implementation NOT migrated:
- init/residual-init — assumes Alpine /init sources it; incorrect (current Alpine initramfs-init does not source external hook files).
- cmd/res-os/ + internal/* — Go persistence CLI skeleton; all five subcommands return fmt.Errorf("not yet implemented").
- overlayfs/rootfs/etc/ly/ — static overlay for ly DM; obsolete (base ISO is console-only).
- build.sh / Dockerfile.build / suckless/ — compile pipeline; not migrated to new shell-based os/.

Decisions (confirmed with user)

Boot model: Alpine diskless tmpfs-built root as lower layer, overlay persistence on top. res-os setup copies current tmpfs root into new @upper subvolume to preserve live session; if persistence already has data, existing @upper used as-is.
Init integration: Maintain a full fork of Alpine's initramfs-init, installed as /usr/share/mkinitfs/initramfs-init-residual, selected via /etc/mkinitfs/mkinitfs.conf.
Scope priority: x86_64 ISO persistence first. RPI persistence is a follow-up phase.

Gaps vs. expected design

Initramfs hook integration undefined (no standard extension point; old init/residual-init can't be sourced).
ISO boot model mismatch (docs assume read-only squashfs lower; diskless builds tmpfs root at boot).
No static overlay / first-boot config (hostname still "alpine"; no residual user, getty/login wiring, /etc/residual/ defaults; no place to ship res-os).
res-os CLI does not exist in os/ (stubs in os-old/).
RPI image has no persistence partition wiring (RESIDUAL-PERSIST not created).
Documentation drift (docs/os/os.md:3 says "future work"; stale binary names residual-login etc.; AGENTS.md references os/cmd/res-os/, os/suckless/ that only exist in os-old/).

Proposed technical approach

Initramfs: os/residual/residual-initramfs/APKBUILD installing /etc/mkinitfs/features.d/residual.files + .modules, mkinitfs.conf (init → custom script), forked initramfs-init-residual, and /etc/initramfs-init.d/residual hook (mount persistence, snapshot @upper, overlay, or RESIDUAL_LIVE=1).
Boot model: overlay persistence on top of tmpfs-built root; live-root migration on first setup.
Static overlay: os/overlayfs/rootfs/ console-only residual defaults (hostname, /etc/residual/, getty for res-login, residual user, /etc/apk/repositories); ship via apkovl or squashfs.
res-os CLI: migrate skeleton from os-old/ to os/cmd/res-os/; implement setup (qcow2 default, partition option, live-root migration), status, snapshot, rollback, export. Statically linked, no CGO, no core/ dependency.
RPI persistence (deferred): create/format RESIDUAL-PERSIST btrfs partition with @upper/@work/@snapshots; update cmdline.txt to drop overlaytmpfs=yes once hook validated.
Build pipeline: restore compile stage (os/build.sh + Dockerfile.build); add res-os to static overlay; add residual-initramfs package to profile apks.

Phased plan

Phase 0 — Cleanup and doc alignment (update docs/os/os.md, AGENTS.md; decide os-old/ fate; remove stale references).
Phase 1 — Static overlay + configured boot (console-only defaults, apkovl, hostname="residual").
Phase 2 — Initramfs hook (x86_64): APKBUILD, forked init, feature files, mkinitfs.conf, QEMU test.
Phase 3 — res-os CLI: migrate + implement 5 subcommands, live-root migration, QEMU test.
Phase 4 — RPI persistence: partition + subvolumes + cmdline update.
Phase 5 — Binaries and package tree: APKBUILDs for Go binaries + suckless; compile stage.

Files likely to change

docs/os/os.md, AGENTS.md, os/scripts/mkimg.residual*.sh, os/scripts/mkimg.base.sh, os/overlayfs/rootfs/, os/residual/residual-initramfs/*, os/cmd/res-os/, os/internal/*/, os/build.sh, os/Dockerfile.build, os/compose.yml/Dockerfile.mkimage.

Verification

PROFILE=residual_virt docker compose run --rm mkimage succeeds.
docker compose run --rm boot-test boots to configured console.
QEMU: res-os setup qcow2, reboot, changes persist.
QEMU: boot without persistence → live-mode warning.
Build RPI image, verify RESIDUAL-PERSIST partition.

Remaining open questions

User creation at build time or first boot?
Login flow: res-login on tty1 via inittab/openrc, or busybox getty for now?
RPI persistence partition size (grow-to-fit on first boot?).
os-old cleanup: delete once migrated, or archive outside workspace?

Risks

Forking Alpine's initramfs-init creates upgrade debt when Alpine bumps mkinitfs.
qemu-nbd + nbd module inside initramfs adds size/complexity.
Diskless boot + overlay means package install every boot; large sets slow boot even with persistence.
res-os operates on block devices/btrfs; mistakes destroy data. Needs confirmation prompts + dry-run.

# Plan: os/ Persistent Overlay ## Current state - `os/` builds bootable x86_64 ISOs (`residual`, `residual_virt`) and is constructing an AArch64 RPI image (`residual_rpi`). - **No persistence mechanism is wired in yet.** Current boot uses Alpine diskless mode with `overlaytmpfs=yes` (RPI) or no writability (ISO) — changes lost on reboot. - Design in `docs/os/os.md` is complete on paper: btrfs-backed overlayfs, qcow2 or labeled-partition backend, `/boot/residual.conf`, `res-os` CLI, custom initramfs hook. - `os-old/` contains a previous partial implementation NOT migrated: - `init/residual-init` — assumes Alpine `/init` sources it; **incorrect** (current Alpine `initramfs-init` does not source external hook files). - `cmd/res-os/` + `internal/*` — Go persistence CLI skeleton; all five subcommands return `fmt.Errorf("not yet implemented")`. - `overlayfs/rootfs/etc/ly/` — static overlay for `ly` DM; obsolete (base ISO is console-only). - `build.sh` / `Dockerfile.build` / `suckless/` — compile pipeline; not migrated to new shell-based `os/`. ## Decisions (confirmed with user) 1. **Boot model**: Alpine diskless tmpfs-built root as lower layer, overlay persistence on top. `res-os setup` copies current tmpfs root into new `@upper` subvolume to preserve live session; if persistence already has data, existing `@upper` used as-is. 2. **Init integration**: Maintain a full fork of Alpine's `initramfs-init`, installed as `/usr/share/mkinitfs/initramfs-init-residual`, selected via `/etc/mkinitfs/mkinitfs.conf`. 3. **Scope priority**: x86_64 ISO persistence first. RPI persistence is a follow-up phase. ## Gaps vs. expected design 1. Initramfs hook integration undefined (no standard extension point; old `init/residual-init` can't be sourced). 2. ISO boot model mismatch (docs assume read-only squashfs lower; diskless builds tmpfs root at boot). 3. No static overlay / first-boot config (hostname still "alpine"; no `residual` user, getty/login wiring, `/etc/residual/` defaults; no place to ship `res-os`). 4. `res-os` CLI does not exist in `os/` (stubs in `os-old/`). 5. RPI image has no persistence partition wiring (`RESIDUAL-PERSIST` not created). 6. Documentation drift (`docs/os/os.md:3` says "future work"; stale binary names `residual-login` etc.; `AGENTS.md` references `os/cmd/res-os/`, `os/suckless/` that only exist in `os-old/`). ## Proposed technical approach - **Initramfs**: `os/residual/residual-initramfs/APKBUILD` installing `/etc/mkinitfs/features.d/residual.files` + `.modules`, `mkinitfs.conf` (init → custom script), forked `initramfs-init-residual`, and `/etc/initramfs-init.d/residual` hook (mount persistence, snapshot `@upper`, overlay, or `RESIDUAL_LIVE=1`). - **Boot model**: overlay persistence on top of tmpfs-built root; live-root migration on first `setup`. - **Static overlay**: `os/overlayfs/rootfs/` console-only residual defaults (hostname, `/etc/residual/`, getty for `res-login`, `residual` user, `/etc/apk/repositories`); ship via `apkovl` or squashfs. - **`res-os` CLI**: migrate skeleton from `os-old/` to `os/cmd/res-os/`; implement `setup` (qcow2 default, partition option, live-root migration), `status`, `snapshot`, `rollback`, `export`. Statically linked, no CGO, no `core/` dependency. - **RPI persistence (deferred)**: create/format `RESIDUAL-PERSIST` btrfs partition with `@upper`/`@work`/`@snapshots`; update `cmdline.txt` to drop `overlaytmpfs=yes` once hook validated. - **Build pipeline**: restore compile stage (`os/build.sh` + `Dockerfile.build`); add `res-os` to static overlay; add `residual-initramfs` package to profile `apks`. ## Phased plan - Phase 0 — Cleanup and doc alignment (update `docs/os/os.md`, `AGENTS.md`; decide `os-old/` fate; remove stale references). - Phase 1 — Static overlay + configured boot (console-only defaults, apkovl, hostname="residual"). - Phase 2 — Initramfs hook (x86_64): APKBUILD, forked init, feature files, mkinitfs.conf, QEMU test. - Phase 3 — `res-os` CLI: migrate + implement 5 subcommands, live-root migration, QEMU test. - Phase 4 — RPI persistence: partition + subvolumes + cmdline update. - Phase 5 — Binaries and package tree: APKBUILDs for Go binaries + suckless; compile stage. ## Files likely to change `docs/os/os.md`, `AGENTS.md`, `os/scripts/mkimg.residual*.sh`, `os/scripts/mkimg.base.sh`, `os/overlayfs/rootfs/`, `os/residual/residual-initramfs/*`, `os/cmd/res-os/`, `os/internal/*/`, `os/build.sh`, `os/Dockerfile.build`, `os/compose.yml`/`Dockerfile.mkimage`. ## Verification 1. `PROFILE=residual_virt docker compose run --rm mkimage` succeeds. 2. `docker compose run --rm boot-test` boots to configured console. 3. QEMU: `res-os setup` qcow2, reboot, changes persist. 4. QEMU: boot without persistence → live-mode warning. 5. Build RPI image, verify `RESIDUAL-PERSIST` partition. ## Remaining open questions 1. User creation at build time or first boot? 2. Login flow: `res-login` on tty1 via inittab/openrc, or busybox getty for now? 3. RPI persistence partition size (grow-to-fit on first boot?). 4. `os-old` cleanup: delete once migrated, or archive outside workspace? ## Risks - Forking Alpine's `initramfs-init` creates upgrade debt when Alpine bumps `mkinitfs`. - `qemu-nbd` + `nbd` module inside initramfs adds size/complexity. - Diskless boot + overlay means package install every boot; large sets slow boot even with persistence. - `res-os` operates on block devices/btrfs; mistakes destroy data. Needs confirmation prompts + dry-run.

agent added the

tier-4-os

label

2026-07-19 02:32:43 +00:00