Runbooks
Copy-paste procedures for the common operational tasks -deploys, releases, log checks, and support lookups.
Last updated Thu Jul 16 2026 00:00:00 GMT+0000 (Coordinated Universal Time)
TL;DR -A collection of ready-to-run procedures for the everyday jobs: deploying code, cutting iOS and Android builds, watching logs, checking health, running read-only database queries, and diagnosing a missing purchase.
Who this is for -DevOps & Operations: your muscle-memory reference. QA: the build and test-email procedures. Support & Engineering: the log, health-check, database, and "purchase not credited" playbooks.
A runbook is a short, exact recipe for a routine operational task -the kind of thing you do often enough to want copy-paste commands, but not so often that you've memorized every flag. Each section below is one task. Read the plain-English line under the heading to know what it's for, then run the commands.
All commands assume the repo checkout on the VPS (/root/social-live).
Deploy latest code
Ship the newest backend and admin code to production.
git pull
docker compose -f docker-compose.prod.yml --env-file .env.production up -d --build backend
docker compose -f docker-compose.prod.yml --env-file .env.production up -d --build admin
Release iOS build (TestFlight / App Store)
Build the iPhone app on the dev Mac and upload it to Apple for testing or release.
On the dev Mac, from the repo root:
# 1. bump the +N build number in pubspec.yaml (must be unique per upload)
# 2. build and upload
flutter build ipa
xcrun altool --upload-app --type ios \
-f build/ios/ipa/social_live.ipa \
-u "ambegepy@gmail.com" -p "@keychain:SOCIAL_LIVE_ALTOOL"
The app-specific password lives in the dev Mac's login keychain
(SOCIAL_LIVE_ALTOOL). Full procedure incl. IPA verification and
troubleshooting: .claude/skills/release-ipa/SKILL.md. After upload the
build processes in App Store Connect for ~5–30 min, then attach it to the
version under App Store → Add Build.
Release Android build (Google Play)
Build a signed Android app and upload it to Google Play. The first block is a one-time signing setup you only do once per machine.
First-time signing setup (once per machine):
# 1. generate an upload keystore (keep it safe, never commit it)
keytool -genkey -v -keystore ~/social-live-upload.jks \
-keyalg RSA -keysize 2048 -validity 10000 -alias upload
# 2. copy the template, then fill in real passwords + the .jks path
cp android/key.properties.example android/key.properties
If keytool isn't on PATH (no system JDK installed), use the copy
bundled with Android Studio:
"/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/keytool".
The same JBR is used by flutter build appbundle.
android/key.properties and the .jks are git-ignored. When
key.properties is present the release build signs with the upload key
(android/app/build.gradle.kts); without it, release falls back to debug
signing so local flutter run --release still works.
Build the release bundle for Play:
# bump the +N build number in pubspec.yaml first (unique per upload)
flutter build appbundle
# output: build/app/outputs/bundle/release/app-release.aab
Upload the .aab to Play Console → Testing → Internal testing → Create
release. The first upload unblocks creating the coins_* products.
Watch logs
Tail the backend's live output, or scan the last hour for errors.
docker logs social-live-backend --since 10m -f
docker logs social-live-backend --since 1h 2>&1 | grep -iE "error|exception"
Check platform health
Confirm every container is up and the API answers.
docker ps --format "table {{.Names}}\t{{.Status}}"
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:3000/api/wallet/coin-packages # expect 200
Database access (read-only queries)
Open a database shell to look things up. These are for reading only -see the rule at the end.
docker exec social-live-postgres psql \
-U $(grep '^POSTGRES_USER=' .env.production | cut -d= -f2) \
-d $(grep '^POSTGRES_DB=' .env.production | cut -d= -f2)
Useful queries:
-- A user's identity state
SELECT id, email, "emailVerifiedAt", "phoneNumber" FROM "User" WHERE email = '...';
-- A user's money history (newest first)
SELECT type, coins, "bonusCoins", "diamondMicros", channel, "externalRef", "createdAt"
FROM "LedgerTransaction" WHERE "userId" = '...' ORDER BY "createdAt" DESC LIMIT 50;
-- Negative wallets (refund debtors)
SELECT "userId", coins FROM "Wallet" WHERE coins < 0;
Rule: never hand-edit Wallet or LedgerTransaction rows. Balance
corrections go through admin endpoints so a ledger row records them.
Add/remove a QA test email
Change which emails can log in with a fixed QA code, then reload the backend. This is an env-only change, so no rebuild is needed.
vi .env.production # edit OTP_TEST_EMAILS (comma-separated)
docker compose -f docker-compose.prod.yml --env-file .env.production up -d backend
Env-only: no rebuild. Expect a brief postgres recreate (env_file hash change) -data persists on the volume.
Run test suites
Run the automated backend and mobile tests.
npm --workspace backend run test # backend (Jest)
flutter test # mobile
npm --workspace backend run prisma:generate # if specs fail on missing enum members
Known pre-existing failures live outside the money/auth paths and are tracked in the repo; a suite that was green before your change must be green after.
Investigate "purchase not credited"
A user paid but got no coins -here's how to tell whether it self-heals or needs you.
docker logs social-live-backend | grep -i "purchase\|iap"-look for verification failures.- Ledger check: does a
COIN_PURCHASErow exist for the store order id (externalRef)? - If the user paid but no row exists: the client will redeliver the unfinished purchase on next wallet open and the credit will replay — that is the designed self-heal. Only if verification itself errors (store API outage, misconfigured keys) does it need operator action.
Certificate / nginx
Reload nginx after config edits; TLS renewals are automatic.
TLS is host-level nginx + certbot (deploy/nginx/, see
docs/vps-deployment.md in the repo). Renewals are certbot-automatic;
after nginx config edits: nginx -t && systemctl reload nginx.
Related pages
- Deployment & Operations -the container topology and standard deploy commands.
- Backup & Restore -nightly backups and the restore drill.
- Android release & Drive upload -the scripted Android release loop.
- Environment variables -settings referenced above, like
OTP_TEST_EMAILS.