Received: from singpolyma.net (singpolyma.net [192.99.233.116])
	by mail.sr.ht (Postfix) with ESMTPS id 79856401F9
	for <~sircmpwn/sr.ht-dev@lists.sr.ht>; Sat,  9 Feb 2019 01:56:11 +0000 (UTC)
Authentication-Results: mail.sr.ht;
	dkim=pass (1024-bit key) header.d=singpolyma.net header.i=@singpolyma.net header.b=NWz3EJZr
Received: by singpolyma.net (Postfix, from userid 1000)
	id 44E3E4860A06; Sat,  9 Feb 2019 01:55:47 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=singpolyma.net;
	s=iweb; t=1549677347;
	bh=LfXFRtC8TQfm+aTJ5pRiyKQKqBRJU80NEkhbvV6/6Ps=;
	h=From:To:Cc:Subject:Date:From;
	b=NWz3EJZrzvpAehykTuIO2JuQyGuLhVzCTXGxLmoargl8loARH4adXY2YVAQkgAB5G
	 PZ7kqqxXI5xjyYxuzWZsHeEI0NITYJH/lIB/aLRFt9JPg3+E4aD6kper7SGSqVOxa3
	 FtrdTMa3CLSm8KfmxDnffI93gvedEiXmryVc30Pk=
From: Stephen Paul Weber <singpolyma@singpolyma.net>
To: ~sircmpwn/sr.ht-dev@lists.sr.ht
Cc: Stephen Paul Weber <singpolyma@singpolyma.net>
Subject: [PATCH scm.sr.ht] Add API to list all public repos
Date: Fri,  8 Feb 2019 20:55:20 -0500
Message-Id: <20190209015520.17458-1-singpolyma@singpolyma.net>
X-Mailer: git-send-email 2.11.0

This is especially useful for crawlers and archivers, and is essential to
adding support for sr.ht to https://www.softwareheritage.org/

The API pages at 100 per page instead of 11 like other APIs to lower the
number of requests required to get the whole list.
---
 scmsrht/blueprints/api.py | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/scmsrht/blueprints/api.py b/scmsrht/blueprints/api.py
index d24c02e..4228205 100644
--- a/scmsrht/blueprints/api.py
+++ b/scmsrht/blueprints/api.py
@@ -14,7 +14,8 @@ repo_json = lambda r: {
     "description": r.description,
     "created": r.created,
     "updated": r.updated,
-    "visibility": r.visibility.value
+    "visibility": r.visibility.value,
+    "clone": r.owner.canonical_name + "/" + r.name
 }
 
 wh_json = lambda wh: {
@@ -57,6 +58,26 @@ def repos_POST(oauth_token):
         return valid.response
     return repo_json(repo)
 
+@api.route("/api/repos/all")
+def repos_all():
+    start = request.args.get('start') or -1
+    Repository = current_app.Repository
+    repos = (Repository.query
+        .filter(Repository.visibility == RepoVisibility.public)
+    )
+    if start != -1:
+        repos = repos.filter(Repository.id <= start)
+    repos = repos.order_by(Repository.id.desc()).limit(100).all()
+    if len(repos) != 100:
+        next_id = -1
+    else:
+        next_id = repos[-1].id
+        repos = repos[:99]
+    return {
+        "next": next_id,
+        "results": [repo_json(r) for r in repos]
+    }
+
 @api.route("/api/repos/~<owner>")
 def repos_username_GET(owner):
     User = current_app.User
-- 
2.11.0