~sircmpwn/sr.ht-dev

This thread contains a patchset. You're looking at the original emails, but you may wish to use the patch review UI. Review patch
1 2

[PATCH scm.sr.ht] Add API to list all public repos

Details
Message ID
<20190209015520.17458-1-singpolyma@singpolyma.net>
Sender timestamp
1549677320
DKIM signature
missing
Download raw message
Patch: +22 -1
This is especially useful for crawlers and archivers, and is essential to
adding support for sr.ht to https://www.softwareheritage.org/

The API pages at 100 per page instead of 11 like other APIs to lower the
number of requests required to get the whole list.
---
 scmsrht/blueprints/api.py | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/scmsrht/blueprints/api.py b/scmsrht/blueprints/api.py
index d24c02e..4228205 100644
--- a/scmsrht/blueprints/api.py
+++ b/scmsrht/blueprints/api.py
@@ -14,7 +14,8 @@ repo_json = lambda r: {
    "description": r.description,
    "created": r.created,
    "updated": r.updated,
    "visibility": r.visibility.value
    "visibility": r.visibility.value,
    "clone": r.owner.canonical_name + "/" + r.name
}

wh_json = lambda wh: {
@@ -57,6 +58,26 @@ def repos_POST(oauth_token):
        return valid.response
    return repo_json(repo)

@api.route("/api/repos/all")
def repos_all():
    start = request.args.get('start') or -1
    Repository = current_app.Repository
    repos = (Repository.query
        .filter(Repository.visibility == RepoVisibility.public)
    )
    if start != -1:
        repos = repos.filter(Repository.id <= start)
    repos = repos.order_by(Repository.id.desc()).limit(100).all()
    if len(repos) != 100:
        next_id = -1
    else:
        next_id = repos[-1].id
        repos = repos[:99]
    return {
        "next": next_id,
        "results": [repo_json(r) for r in repos]
    }

@api.route("/api/repos/~<owner>")
def repos_username_GET(owner):
    User = current_app.User
-- 
2.11.0
Details
Message ID
<20190209151202.GB1628@homura.localdomain>
In-Reply-To
<20190209015520.17458-1-singpolyma@singpolyma.net> (view parent)
Sender timestamp
1549725123
DKIM signature
missing
Download raw message
Gonna have to reject this patch for the time being. The lack of
discoverability on git.sr.ht is a feature, not a bug. In the future
there will be a central project management hub (at the top-level sr.ht
domain), which will have indexing and browsing. This feature will be
more appropriate there.

For the time being I suggest using external methods (spidering the
broader web) to find repos to archive, or archiving repos on a
by-request basis.
Reply to thread Export thread (mbox)