Install

Release Runbook

View as markdown

This is the operational checklist for Shelbi maintainers publishing a release. Run every command from a clean checkout of https://github.com/jlong/shelbi unless a step says otherwise.

Current Release Scope

  • Primary repository: jlong/shelbi
  • Release binary: shelbi
  • Release branch: main
  • Release tags: vMAJOR.MINOR.PATCH, for example v0.1.0
  • Homebrew tap: unresolved; plan recommends shelbi/homebrew-shelbi if a Shelbi org exists, otherwise jlong/homebrew-shelbi
  • APT repository: unresolved; plan recommends a dedicated Pages or object storage repository
  • APT repository domain: unresolved; plan recommends https://apt.shelbi.dev
  • Current release owner: unresolved
  • Signing-key owner: unresolved

Do not ship a release until every unresolved owner, repository, and domain is confirmed in the maintainer channel.

Required Access

The release operator needs these local tools:

cargo --version
git --version
gh --version
goreleaser --version
jq --version
dpkg-deb --version
ar --version
tar --version
gpg --version
apt-ftparchive --version
brew --version
docker --version

The release automation needs these secrets:

  • GITHUB_TOKEN or a GitHub App token with permission to create releases and upload release artifacts in jlong/shelbi.
  • TAP_GITHUB_TOKEN, a fine-scoped token or GitHub App credential that can write only to the confirmed Homebrew tap where possible.
  • APT_REPO_TOKEN, a fine-scoped token or GitHub App credential that can write to the confirmed APT hosting repository.
  • APT_GPG_PRIVATE_KEY, an ASCII-armored private key for APT metadata signing.
  • APT_GPG_PASSPHRASE, the passphrase for the APT private key.
  • APT_SIGNING_KEY_ID, the public fingerprint for the APT metadata signing key.
  • OIDC permissions for keyless Sigstore signing, or COSIGN_* only if the release workflow chooses long-lived cosign secrets.

The signing-key owner rotates APT and artifact signing keys. The release owner rotates publishing tokens. Rotate immediately after suspected exposure, staff offboarding, or a failed release where a secret may have been printed in logs.

Pre-Tag Checks

Start from an up-to-date main:

git fetch origin main --tags
git checkout main
git pull --ff-only origin main
git status --short

git status --short must print nothing. Confirm the version before tagging:

VERSION=0.1.0
test "$(cargo metadata --no-deps --format-version 1 | jq -r '.packages[] | select(.name == "shelbi") | .version')" = "$VERSION"

Run the workspace tests and build the exact release profile:

cargo test --workspace
cargo build --release --bin shelbi
./target/release/shelbi --version

The version output must match the tag you intend to publish:

./target/release/shelbi --version | grep "shelbi $VERSION"

Curate the changelog before tagging. site/content/docs/changelog.mdx is hand-maintained and updates nowhere else, so add or finish this version's entry now: a feature-level, newest-first note dated to when the work landed on main, folding internal refactors into the capability they enabled. Commit it to main before the tag so the published site matches the release.

GoReleaser Dry Run

Validate the GoReleaser configuration:

goreleaser check

Build local snapshot artifacts without publishing:

goreleaser release --snapshot --clean

Expected snapshot output under dist/:

  • shelbi_Darwin_x86_64.tar.gz
  • shelbi_Darwin_arm64.tar.gz
  • shelbi_Linux_x86_64.tar.gz
  • shelbi_0.1.0_amd64.deb
  • checksums.txt
  • signed checksums or Sigstore bundle, if signing is configured
  • Homebrew formula output, if the confirmed tap is configured
  • APT repository metadata output, if the confirmed repository is configured

Linux arm64 and Debian arm64 artifacts are out of scope until they are built and smoke-tested in CI or on a reliable runner.

Inspect the generated files:

find dist -maxdepth 2 -type f | sort
grep -E 'shelbi_(Darwin|Linux)|\\.deb$' dist/checksums.txt

Local Artifact Verification

Verify archive contents and checksums:

shasum -a 256 -c dist/checksums.txt
tar -tzf dist/shelbi_Linux_x86_64.tar.gz | grep '^shelbi$'
tar -xzf dist/shelbi_Linux_x86_64.tar.gz -C /tmp
/tmp/shelbi --version
rm /tmp/shelbi

Verify the Debian package locally:

DEB=$(find dist -name 'shelbi_*_amd64.deb' | head -n 1)
dpkg-deb --info "$DEB"
dpkg-deb --contents "$DEB"
dpkg-deb --field "$DEB" Package Version Architecture Maintainer
dpkg-deb --field "$DEB" Package | grep '^shelbi$'
dpkg-deb --field "$DEB" Version | grep "^$VERSION"
dpkg-deb --field "$DEB" Architecture | grep '^amd64$'

Install and execute the package in a clean Debian or Ubuntu container:

docker run --rm -v "$PWD/dist:/dist:ro" debian:bookworm bash -euxo pipefail -c '
  apt-get update
  apt-get install -y /dist/shelbi_*_amd64.deb
  shelbi --version
  command -v shelbi
'

Homebrew Formula Verification

After the dry run generates or updates the formula, check the formula in the confirmed tap checkout:

TAP_DIR=/tmp/homebrew-shelbi-tap
git clone git@github.com:OWNER/HOMEBREW_TAP_REPO.git "$TAP_DIR"
cd "$TAP_DIR"
brew audit --strict --online Formula/shelbi.rb
brew style Formula/shelbi.rb
brew install --build-from-source Formula/shelbi.rb
shelbi --version
brew test shelbi
brew uninstall shelbi

Replace OWNER/HOMEBREW_TAP_REPO with the confirmed Homebrew tap. If the tap uses a different formula path, update the path before release and commit the runbook correction in the same change.

APT Repository Verification

Verify repository metadata before users can consume it:

APT_ROOT=dist/apt
find "$APT_ROOT" -type f | sort
test -f "$APT_ROOT/dists/stable/Release"
test -f "$APT_ROOT/dists/stable/InRelease"
test -f "$APT_ROOT/dists/stable/Release.gpg"
gpg --verify "$APT_ROOT/dists/stable/Release.gpg" "$APT_ROOT/dists/stable/Release"
gpg --verify "$APT_ROOT/dists/stable/InRelease"
grep '^Suite: stable$' "$APT_ROOT/dists/stable/Release"
grep '^Codename: stable$' "$APT_ROOT/dists/stable/Release"
grep '^Architectures: amd64$' "$APT_ROOT/dists/stable/Release"
grep 'pool/.*/shelbi_.*_amd64.deb' "$APT_ROOT/dists/stable/main/binary-amd64/Packages"

Then install from the staged repository in a container:

docker run --rm -v "$PWD/dist/apt:/repo:ro" debian:bookworm bash -euxo pipefail -c '
  apt-get update
  apt-get install -y ca-certificates gnupg
  install -d -m 0755 /etc/apt/keyrings
  cp /repo/shelbi-archive-keyring.gpg /etc/apt/keyrings/shelbi.gpg
  echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/shelbi.gpg] file:/repo stable main" > /etc/apt/sources.list.d/shelbi.list
  apt-get update
  apt-cache policy shelbi
  apt-get install -y shelbi
  shelbi --version
'

For the live repository, replace the file:/repo source with the confirmed HTTPS APT domain and repeat the same container test.

Tag And Publish

Create an annotated tag only after the dry run and local verification pass:

VERSION=0.1.0
git checkout main
git pull --ff-only origin main
git tag -a "v$VERSION" -m "Shelbi v$VERSION"
git push origin "v$VERSION"

Publish with GoReleaser:

goreleaser release --clean

Verify the GitHub release:

gh release view "v$VERSION" --repo jlong/shelbi
gh release download "v$VERSION" --repo jlong/shelbi --dir "/tmp/shelbi-v$VERSION"
cd "/tmp/shelbi-v$VERSION"
shasum -a 256 -c checksums.txt

Expected published artifacts:

  • Darwin amd64 archive
  • Darwin arm64 archive
  • Linux amd64 archive
  • Debian amd64 package
  • checksums.txt
  • Artifact signatures, if signing is enabled
  • SBOM/provenance files, if configured
  • Homebrew formula update in the confirmed tap
  • APT pool package, Packages, Release, InRelease, and Release.gpg

Post-Release Verification

Verify install paths after publication:

brew update
brew install shelbi
shelbi --version
brew test shelbi
brew uninstall shelbi

Verify APT from the live repository:

docker run --rm debian:bookworm bash -euxo pipefail -c '
  apt-get update
  apt-get install -y ca-certificates curl gnupg
  install -d -m 0755 /etc/apt/keyrings
  curl -fsSL https://APT_DOMAIN/shelbi-archive-keyring.gpg -o /etc/apt/keyrings/shelbi.gpg
  echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/shelbi.gpg] https://APT_DOMAIN stable main" > /etc/apt/sources.list.d/shelbi.list
  apt-get update
  apt-cache policy shelbi
  apt-get install -y shelbi
  shelbi --version
'

Replace APT_DOMAIN with the confirmed release domain. The installed version must match VERSION.

Rollback

Bad GitHub Artifacts

If an artifact is corrupt before the release is announced, stop package publication, delete the draft or unpublished release, and rerun the release from the same tag:

VERSION=0.1.0
gh release delete "v$VERSION" --repo jlong/shelbi --yes
goreleaser release --clean

After public announcement or package-manager publication, do not replace or delete GitHub release assets. Publish a patch version from a fix commit:

NEXT_VERSION=0.1.1
git checkout main
git pull --ff-only origin main
git tag -a "v$NEXT_VERSION" -m "Shelbi v$NEXT_VERSION"
git push origin "v$NEXT_VERSION"
goreleaser release --clean

If the tag points at the wrong commit and the release is not public yet, delete the release and tag, then recreate the tag on the correct commit:

VERSION=0.1.0
gh release delete "v$VERSION" --repo jlong/shelbi --yes
git push origin ":refs/tags/v$VERSION"
git tag -d "v$VERSION"
git checkout CORRECT_COMMIT_SHA
git tag -a "v$VERSION" -m "Shelbi v$VERSION"
git push origin "v$VERSION"
goreleaser release --clean

Announce the tag move in the maintainer channel. Never move a tag silently after users may have fetched it. After public announcement, leave the bad tag in place and publish a patch version.

Bad Homebrew Formula

Revert the formula commit in the tap and push the revert:

cd "$TAP_DIR"
git pull --ff-only
git log --oneline -- Formula/shelbi.rb
git revert BAD_FORMULA_COMMIT_SHA
brew audit --strict --online Formula/shelbi.rb
brew test shelbi
git push origin HEAD

If the formula points at a bad GitHub artifact, complete the GitHub artifact rollback first, then update the formula checksums and run the formula verification again.

Bad APT Package

Republish APT metadata without the bad package as the candidate version. Leave the bad .deb in pool/ for auditability unless it is actively harmful. The resulting repository must no longer advertise the bad version:

apt-cache policy shelbi

Regenerate and sign repository metadata:

apt-ftparchive packages pool > dists/stable/main/binary-amd64/Packages
gzip -kf dists/stable/main/binary-amd64/Packages
apt-ftparchive release dists/stable > dists/stable/Release
gpg --batch --yes --default-key "$APT_SIGNING_KEY_ID" --clearsign -o dists/stable/InRelease dists/stable/Release
gpg --batch --yes --default-key "$APT_SIGNING_KEY_ID" -abs -o dists/stable/Release.gpg dists/stable/Release

Run the live APT container verification again. Publish a maintainer note telling users to run sudo apt update before retrying. APT does not automatically downgrade installed packages; once at least two versions exist, document the manual downgrade:

apt-cache madison shelbi
sudo apt install shelbi=PREVIOUS_GOOD_VERSION
sudo apt-mark hold shelbi

APT Key Compromise

Treat a suspected signing-key exposure as a release incident:

  1. Freeze APT publication and remove repository write credentials from CI.
  2. Revoke the compromised key if a revocation certificate exists.
  3. Generate a new offline signing key and store its revocation certificate in the maintainer secret store.
  4. Export the new public key as shelbi-archive-keyring.gpg.
  5. Replace APT_GPG_PRIVATE_KEY, APT_GPG_PASSPHRASE, APT_SIGNING_KEY_ID, and any deployment secret that could have accessed the old private key.
  6. Re-sign the repository metadata with the new key.
  7. Publish the new keyring at the confirmed APT domain.
  8. Publish user-facing migration instructions that replace the old keyring file before running sudo apt update.

Use these local commands for the new key material:

gpg --batch --full-generate-key
gpg --list-secret-keys --keyid-format LONG
gpg --output shelbi-archive-keyring.gpg --export "$NEW_APT_SIGNING_KEY_ID"
gpg --output shelbi-archive-revocation.asc --gen-revoke "$NEW_APT_SIGNING_KEY_ID"

Do not delete the old public key from public history. Users need a clear migration path from the old key to the new one.