Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by mail-b.sr.ht (Postfix) with ESMTPS id 04CFFFF0DE for <~sircmpwn/sr.ht-dev@lists.sr.ht>; Fri, 18 Sep 2020 16:03:33 +0000 (UTC) Authentication-Results: mail-b.sr.ht; dkim=fail reason="key not found in DNS" (0-bit key) header.d=cmpwn.com header.i=@cmpwn.com header.b=BuPtqpdG MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpwn.com; s=key1; t=1600445012; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nIyegK+joHZmP9vKBdqPCqcxGfuqSlY/CKQocVMMeWc=; b=BuPtqpdGpylwDUdOJnBtCwCLCngVkvp66ouZjKu77hriOEmHq04uRgbVH4WjAkOS5jH7cF M7Xry4wUM0VRTJVre00intC1TT2qnC12wwtauIwW9pSl1W+fpQo4ItGzRTw+/duKNUfYkP Vdc5bsVn9rI1O9I1BeCjtN/QCKesxw0O5nuA09SVSn3gK4cal0AIWcuijukibTaTMxTi6x cPOluaNKBXT5WHooMGc8sfeDsAtW6Ll6gdEKwj3rDH2LNlzHGm1Vt9Jp8Bx7N441K3/OBl 35sIodTUjORUSNrn4R91+NJBaHdkG4tmk7yhrpcGevQPNgH+GMHaBlk4ZOGk0A== Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Subject: git.sr.ht storage growth rate X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: "Drew DeVault" To: <~sircmpwn/sr.ht-dev@lists.sr.ht> Date: Fri, 18 Sep 2020 11:55:32 -0400 Message-Id: X-Spam-Score: 0.40 Currently git.sr.ht is growing at about 50G per week, or 5% of its current capacity, and will run out of storage within 9 weeks. Growth is fairly linear; the rate seems to change slowly. Might be worth expanding the metrics.sr.ht retention rate to make long-term trends more obvious. Some initial observations: - tenshi is currently only giving git.sr.ht half of the disk space on the host, we could reduce this growth to 2.5% by just adding the other half (extending deadline to 21 weeks) - sakyua1 has enough disk space to reduce the growth rate to 1% per week (giving us 200 weeks). Problems: hg.sr.ht is also there, we wanted to use sakyua1 for other stuff. - If enabled, ZFS deduplication ratio would be about 2.3 and should (double check this) fit within our RAM requirements if we expanded git.sr.ht's share of tenshi's RAM. - I didn't check but I don't think ZFS compression would save us much; git already compresses its objects. Dedupe would be a lot more effective. - If tenshi's disks were reprovisioned, we could probably expand its capacity 4x. Combined with reclaiming the half that is unused, that'd be an 8x improvement, reducing the growth rate to ~0.6%/week, and likely giving us 2-3 years of storage runway at least. Future work with availability and storage distribution could affect these plans but I'd rather defer that work, so let's just buy at least enough time to extend the deadline past when we'd expect to look at that work. Budget shouldn't be an issue for any of these solutions. Will probably cost <$500 in any case. Will revisit this and come up with a detailed action plan in 3 weeks.