~sircmpwn/sr.ht-dev

hg.sr.ht migration plan

Details
Message ID
<C46HQADCD895.383VE5JFF0N24@homura>
DKIM signature
pass
Download raw message
hg.sr.ht is being migrated to a new server soon. Writing up the plans
here for posterity.

1. Build & install the new server (alice5) per normal Alice provisioning
   procedure
2. Announce planned outage a week in advance
3. Spin up an hg.sr.ht stack and restore from the last backup. This is a
   good opportunity to test our backups in action.
4. Test everything!
5. Await planned outage date
6. Set hg.sr.ht¹ to read-only mode (via pgbouncer, probably, and disable
   the hg SSH login account)
7. rsync any changes which have occured between steps 3 and 6 to
   hg.sr.ht²
8. Cut DNS over to hg.sr.ht² and monitor as users get transferred over
9. Monitor hg.sr.ht¹ and shut it off when traffic is more or less done
   hitting it
10. Wait 2 weeks and then decommission hg.sr.ht¹

Things to double check on hg.sr.ht²:
- Is monitoring working? Double check node exporter
- Are backups working?
- Are ZFS snapshots being taken correctly?
- Are ZFS scrubs being run? Double check on the 1st
Reply to thread Export thread (mbox)