residual/os
Watch
3
0
Fork
You've already forked os
 1
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

res-os-builder: container-native bootable image builder (design proposal) #1

New issue
Closed
opened 2026-10-07 23:17:23 +00:00 by muse-lantern · 1 comment
muse-lantern commented 2026-10-07 23:17:23 +00:00
Member
Copy link

res-os-builder: container-native bootable image builder (design proposal)

Goal

Build residual boot images from Docker/OCI containers instead of (or alongside) the current mkimage.sh ISO pipeline. Desired flow:

  1. docker build a FROM-based OS image — config scripts baked in via COPY/RUN
  2. Converter tool turns the OCI image into a bootable image
  3. dd to USB

No privileged build containers, no QEMU in the build path. The whole pipeline must run on standard GitHub runners (fully unprivileged).

Why not bootc

Evaluated and rejected:

  • bootc defines a bootable image as kernel + initramfs + systemd; systemd-as-PID1 is baked into the model, not optional. Adopting it means replacing res-init — a core design decision of this project.
  • bootc-image-builder is Fedora/RHEL-family oriented ("especially oriented towards Fedora/CentOS bootc or derivatives"). Alpine support is effectively nonexistent and undocumented; prior attempts at Alpine+bootc hit a wall of undocumented Fedora-isms.

Hand-rolled is the right call here.

Proposed pipeline

  1. Dockerfile.residual: FROM alpine (or a residual base) → apk add → multi-stage COPY of Go binaries (res-init, res-sh, res-console, residual-login, res-edit) → COPY config scripts → RUN scripts. Builds on any stock runner.
  2. Push to an OCI registry (or docker save a tarball) — the tool accepts both.
  3. res-os-builder convert (new Go tool, lives at cmd/res-os-builder in this repo):
    • unpack OCI layers → rootfs/ (umoci/crane, no daemon needed)
    • kernel + initramfs: kernel from the image's /boot, initramfs built via our respkg-mkinitfs fork against those modules
    • mksquashfs rootfs/ → system.squashfs
    • assemble the bootable artifact:
      • ISO via xorriso, and/or
      • raw GPT .img: FAT32 ESP via mtools, partitions laid out with sfdisk + dd at offsets — no loop mounts, no privileges
  4. Every step above works unprivileged (mksquashfs, xorriso, mkfs.ext4 -d, mtools all do).

Boot model: immutable OS + overlay for user data

  • system.squashfs mounted read-only as the overlay lower → immutable system
  • overlay upper: tmpfs (stateless) or a labeled persistent partition (RESIDUAL_DATA, ext4) when present — same image boots stateless on any machine, persistent where the data partition exists, no flags needed
  • initramfs mounts the media, locates the squashfs, sets up the overlay, switch_root → res-init as PID 1
  • This mirrors the current ISO's live-boot model; the only thing that changes is where the rootfs comes from (container instead of apk tree). The knowledge in mkimg.residual*.sh (package lists, config) migrates into the Dockerfile.

Open decisions

  • Output: ISO, raw .img, or both from a single build?
  • Kernel shipped inside the container image (apk add linux-lts in the Dockerfile)?
  • Input: registry reference, local tarball, or both?
  • Data partition label/format defaults?

Suggested first milestone

  1. Probe runner capabilities via a workflow (docker daemon? /dev/kvm? loop devices?)
  2. Minimal POC: alpine + kernel + trivial init → bootable image → QEMU boot check on the runner, grepping the serial log for a marker string; upload the .img as a workflow artifact
# res-os-builder: container-native bootable image builder (design proposal) ## Goal Build residual boot images from Docker/OCI containers instead of (or alongside) the current `mkimage.sh` ISO pipeline. Desired flow: 1. `docker build` a `FROM`-based OS image — config scripts baked in via `COPY`/`RUN` 2. Converter tool turns the OCI image into a bootable image 3. `dd` to USB No privileged build containers, no QEMU in the build path. The whole pipeline must run on **standard GitHub runners** (fully unprivileged). ## Why not bootc Evaluated and rejected: - bootc defines a bootable image as kernel + initramfs + **systemd**; systemd-as-PID1 is baked into the model, not optional. Adopting it means replacing `res-init` — a core design decision of this project. - `bootc-image-builder` is Fedora/RHEL-family oriented ("especially oriented towards Fedora/CentOS bootc or derivatives"). Alpine support is effectively nonexistent and undocumented; prior attempts at Alpine+bootc hit a wall of undocumented Fedora-isms. Hand-rolled is the right call here. ## Proposed pipeline 1. **`Dockerfile.residual`**: `FROM alpine` (or a residual base) → `apk add` → multi-stage `COPY` of Go binaries (`res-init`, `res-sh`, `res-console`, `residual-login`, `res-edit`) → `COPY` config scripts → `RUN` scripts. Builds on any stock runner. 2. Push to an OCI registry (or `docker save` a tarball) — the tool accepts both. 3. **`res-os-builder convert`** (new Go tool, lives at `cmd/res-os-builder` in this repo): - unpack OCI layers → `rootfs/` (`umoci`/`crane`, no daemon needed) - kernel + initramfs: kernel from the image's `/boot`, initramfs built via our `respkg-mkinitfs` fork against those modules - `mksquashfs rootfs/` → `system.squashfs` - assemble the bootable artifact: - ISO via `xorriso`, and/or - raw GPT `.img`: FAT32 ESP via `mtools`, partitions laid out with `sfdisk` + `dd` at offsets — **no loop mounts, no privileges** 4. Every step above works unprivileged (`mksquashfs`, `xorriso`, `mkfs.ext4 -d`, `mtools` all do). ## Boot model: immutable OS + overlay for user data - `system.squashfs` mounted read-only as the overlay **lower** → immutable system - overlay **upper**: `tmpfs` (stateless) **or** a labeled persistent partition (`RESIDUAL_DATA`, ext4) when present — same image boots stateless on any machine, persistent where the data partition exists, no flags needed - initramfs mounts the media, locates the squashfs, sets up the overlay, `switch_root` → `res-init` as PID 1 - This mirrors the current ISO's live-boot model; the only thing that changes is where the rootfs comes *from* (container instead of apk tree). The knowledge in `mkimg.residual*.sh` (package lists, config) migrates into the Dockerfile. ## Open decisions - [ ] Output: ISO, raw `.img`, or both from a single build? - [ ] Kernel shipped inside the container image (`apk add linux-lts` in the Dockerfile)? - [ ] Input: registry reference, local tarball, or both? - [ ] Data partition label/format defaults? ## Suggested first milestone 1. Probe runner capabilities via a workflow (docker daemon? `/dev/kvm`? loop devices?) 2. Minimal POC: alpine + kernel + trivial init → bootable image → QEMU boot check on the runner, grepping the serial log for a marker string; upload the `.img` as a workflow artifact
muse-lantern commented 2026-10-09 14:13:45 +00:00
Author
Member
Copy link

Coalesced into the workspace backlog as residual/.agent#27 (labels proj-os, tier-3-hard). Closing this one to keep a single source of truth.

Coalesced into the workspace backlog as residual/.agent#27 (labels `proj-os`, `tier-3-hard`). Closing this one to keep a single source of truth.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
overlay-arch-fix
fix-build-breakage
refactor-rewire
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
residual/os#1
No description provided.