Deployment Runbook

Production runs on a Sakura VPS (8 GB) at saikokustudio.com with a self-hosted MongoDB (see project notes). This page lists everything you must do after deploying a code release so that the server does not crash and existing data stays consistent.

TL;DR

1. Pull the new code and restart the service.
2. From the matching repo root, run the required scripts for any PR whose number you have not run yet.
3. Verify the health endpoints respond 200.

If a Python ValidationError or KeyError shows up at runtime right after deploy, the most likely cause is that a required one-shot script was skipped. Check the table below.

---

Required one-shot scripts

All scripts are idempotent. Re-running them is safe. Every script supports --dry-run, which prints what would happen without modifying the DB.

PR	Repo	Command (from repo root)	Required?	What it does
#85 PR1	butler-platform	cd auth/backend && uv run python -m scripts.seed_tax_rate_master	Yes	Seeds 3 default rates (10% main, 8% reduced, 0% exempt) into tax_rate_master. Without this the Studio Admin dropdown is empty and no corporate can pick a main rate.
#85 PR3	butler-tax	cd backend && uv run python -m scripts.cleanup_system_settings_tax_rates	Optional	Removes the orphaned system_settings["tax_rates"] MongoDB document (no longer read). Skipping it leaves stale data but does not break anything.
#85 PR4	butler-tax	cd backend && uv run python -m scripts.migrate_tax_category_to_structured	Yes	Converts the legacy "課税仕入 10%" strings in journal_rules / tax_firm_journal_rules / receipts / invoices / system_settings.journal_map into the structured {type, rate} form expected by the new code. Without this, existing rules will fail Pydantic validation and the API will return 500. The script makes a <collection>__backup__<ts> copy before modifying anything.
#88	butler-tax	cd backend && PYTHONPATH=. uv run python scripts/migrate_email_cc_split.py	Optional	Splits the legacy corporate_email_settings.cc_mode + always_cc_emails into independent always_cc_emails / always_bcc_emails lists. The resolver has a legacy fallback so this is not required for correctness, but running it normalises the stored data to the new schema.
#175 (S1)	butler-platform	cd auth/backend && uv run python -m scripts.seed_account_master	Yes	Seeds the standard chart of accounts (109 accounts covering the 4 TestData companies) into account_master. Idempotent — existing codes are never overwritten, so operator edits survive re-runs. Required before the journal/reporting engine (#175 S2+) can classify entries.
減価償却 D1	butler-platform	cd auth/backend && uv run python -m scripts.seed_asset_useful_lives	Yes (for the fixed-asset register, D2+)	Seeds the statutory useful-life categories and the NTA official depreciation-rate tables (別表第八 straight-line / 別表第十 200% DB / 別表第九 250% DB, lives 2–50; the 200%/250% switch is by acquisition date) plus the small-asset / SME special-measure amount thresholds into asset_useful_lives / depreciation_rates / depreciation_thresholds. Idempotent and non-destructive — Studio Admin corrections survive re-runs; *_250 fields are backfilled only when physically absent. Rate values are flagged for tax-accountant confirmation.
#175 (S5)	butler-tax	cd backend && uv run python -m scripts.backfill_auto_journal	Optional	Generates draft journal entries from already-approved receipts/invoices and confirmed matches (--corporate-id <id> to limit to one corporate). Idempotent — sources with an existing entry (source_type+source_id unique index) are skipped. New activity is journalized automatically by hooks, so this is only needed to backfill history; drafts do not affect reports until confirmed in 仕訳帳.
#90 (Slice B) + system emails	butler-platform	cd auth/backend && uv run python -m scripts.seed_email_template_master	Optional (recommended)	Seeds platform-default email templates into email_templates: document emails (invoice/estimate/receipt × ja/en) and system/invitation emails (tax-firm & sales-agent invitation, sales-agent login, email-change confirm/notice, law-watch — ja & en). Row-level, safe to re-run: inserts missing (type, locale) rows, refreshes rows still authored by seed_script to the latest coded default, and never overwrites Studio-Admin edits (rows whose updated_by is a real uid). Re-run after deploy to pick up new/updated default rows. email_service falls back to coded defaults (email_default_templates.py) when a row is absent, so seeding is not required for correctness, but it lets Studio Admin edit the shared defaults. Requires INTERNAL_SERVICE_TOKEN for the /internal/email-templates read path used by butler-tax.
Permission matrix	butler-tax	cd backend && uv run python -m scripts.backfill_employee_permissions	Yes	Backfills new-shape employees.permissions (sections/capabilities/features) by snapshotting each employee's role-group default from the Platform Tier0 template (GET /internal/permission-defaults). Preserves any existing taxAdvisorChat into features.tax_advisor_chat; drops legacy dataProcessing/reportExtraction. Idempotent (skips employees already in the new shape). Supports --dry-run. Requires INTERNAL_SERVICE_TOKEN shared with butler-platform. Without it, existing users have empty permissions once the consumption side gates pages.

Running order on a fresh production server

# 1. butler-platform: seed the new tax-rate master
cd butler-platform/auth/backend
uv run python -m scripts.seed_tax_rate_master --dry-run   # preview
uv run python -m scripts.seed_tax_rate_master              # apply

# 2. butler-tax: migrate the tax_category strings to structured form
cd butler-tax/backend
uv run python -m scripts.migrate_tax_category_to_structured --dry-run
uv run python -m scripts.migrate_tax_category_to_structured

# 3. butler-tax: drop the orphaned tax_rates document
cd butler-tax/backend
uv run python -m scripts.cleanup_system_settings_tax_rates

---

Environment variables checklist

Verify these before bringing up the new release. Missing values cause the server to fail at startup with an EnvironmentError or to crash on first request with a KeyError.

butler-platform/auth/backend (port 8003)

Variable	Required	Notes
MONGODB_URI	yes	mongodb://...
MONGODB_DB_NAME	yes	butler_platform in production
INTERNAL_SERVICE_TOKEN	yes	Shared secret with butler-tax / butler-law. Must match the value those services use.
FIREBASE_CREDENTIALS_PATH	yes	Path to the service-account JSON file. The file must exist on disk and be readable by the service user.
STRIPE_SECRET_KEY	yes	Live key in production
STRIPE_WEBHOOK_SECRET	yes	From the production Webhook endpoint (see #57)
CORS_ORIGINS	yes	Comma-separated list including the Studio Admin / auth-frontend / butler-tax origins
LAW_API_URL	yes (#161)	butler-law backend URL (http://localhost:8002 if co-located). The daily revision-watch job polls GET /api/internal/revisions with INTERNAL_SERVICE_TOKEN.
STUDIO_ADMIN_URL	yes (#161)	https://admin.saikoku-studio.com in production — link target in revision-watch notification emails.
SENDGRID_API_KEY	yes	For invitation / notification emails. Without it sends are skipped with a warning (dev default).
SENDGRID_FROM_EMAIL	yes	Fallback FROM only — the operational FROM lives in the email_settings master, editable on the Studio Admin メールテンプレ page, which validates against SendGrid verified senders on save (unverified addresses fail every send with HTTP 403; only billing@saikokustudio.com is verified today). Complete SendGrid domain authentication (DNS) before using a noreply@ address.

butler-tax/backend (port 8001)

Variable	Required	Notes
MONGODB_URI	yes	self-hosted MongoDB URI
MONGODB_DB_NAME	yes	tax_agent in production (test uses tax_agent_test)
PAYROLL_MONGODB_URI	no	Payroll datastore (#178). Empty → uses MONGODB_URI. Set to isolate payroll on a separate cluster (Atlas / Butler Labor)
PAYROLL_DB_NAME	no	Empty → <MONGODB_DB_NAME>_payroll. Payroll is physically separated from business data; classification etc. live here, only the journal crosses into the main DB
PLATFORM_API_URL	yes	http://localhost:8003 if co-located, otherwise the platform URL
INTERNAL_SERVICE_TOKEN	yes	Must equal the platform's value
FIREBASE_CREDENTIALS_PATH	yes	Same service-account JSON; needed for Firestore PII collections (company_profiles, bank_accounts, clients)
ANTHROPIC_API_KEY	yes	For Butler Chat
GOOGLE_API_KEY	yes	For Gemini OCR
ENVIRONMENT	yes	production enables stricter startup checks (e.g. INTERNAL_SERVICE_TOKEN required)

butler-law/backend (port 8002)

Variable	Required	Notes
MONGODB_URI	yes
MONGODB_DB_NAME	yes	butler_law in production
PLATFORM_API_URL	yes
INTERNAL_SERVICE_TOKEN	yes	Same shared secret. Guards GET /api/internal/revisions (revision feed polled by the platform's law-watch, #161).
ANTHROPIC_API_KEY	yes

butler-platform/studio_admin/frontend (port 3004)

Variable	Required	Notes
NEXT_PUBLIC_PLATFORM_API_URL	yes	Platform API origin; rewrites proxy /api/* to it
LAW_API_URL	yes	butler-law backend origin; rewrites proxy /api/law/* to it (#156: Law admin UI lives here)

If ENVIRONMENT=production and any required variable is missing for butler-tax, startup raises:

RuntimeError: INTERNAL_SERVICE_TOKEN is required in production

That is intentional. Fix the env var rather than removing the check.

---

Native dependencies

WeasyPrint (PDF rendering, butler-tax)

The PDF generation path (invoice_pdf_service) links against libpango, libcairo, and libgdk-pixbuf at runtime. On Debian/Ubuntu install before starting the service:

apt-get update
apt-get install -y \
  libpango-1.0-0 \
  libpangoft2-1.0-0 \
  libharfbuzz0b \
  libfontconfig1 \
  libcairo2 \
  libgdk-pixbuf-2.0-0 \
  libffi8 \
  shared-mime-info

Missing any of these surfaces as a Python OSError: cannot load library at the first PDF request, not at startup. So a server can look healthy until someone clicks "PDF を生成".

When containerising, mirror this list into the Dockerfile.

---

Health checks after deploy

Service	URL	Expected
butler-platform	https://saikokustudio.com/health	200 {"status":"ok"}
butler-tax	https://saikokustudio.com/api/v1/health	200
butler-law	https://saikokustudio.com/law/health	200
Studio Admin (frontend)	https://admin.saikoku-studio.com	200 (or 307 to login)

If any of these returns 502 / 500 and you have not run the scripts above for a freshly deployed PR, run them first before debugging further.

---

When this page should be updated

Every PR that ships:

• a new one-shot script (under scripts/ in either repo), or
• a new required environment variable, or
• a new system-level dependency (apt package, native library, container layer),

must add a row to the appropriate table above in the same PR. If you skip this step, the next person who deploys will hit the exact issue you would have wanted documented.

Treat this page as the single source of truth for what "deploy a release" means in this project.