Hiya! I've been thinking about dispatch.sr.ht lately. Now that most of sr.ht's APIs are fleshed out, it's time to start thinking about how to get them talking to each other more. The goal for some time now has been to use dispatch.sr.ht, but I'm not satisfied with the current design. So, here's a different design. Opening the floor to discussion. Let me start by enumerating some use-cases: *.sr.ht N git repos N trackers 1 mailing list (sr.ht-dev) aerc 1 git repo, 1 mailing list, 1 tracker scdoc 1 git repo, shares mailing list with others, 1 tracker git-send-email.io 1 git repo, shares mailing list with others sway 1 git repo on GitHub, uses builds.sr.ht for builds With these, I want to be able to, for example: - Send patches sent to a mailing list to CI, ideally associating them with a specific git repo in the case of shared lists - Ditto, but for mailing lists on external services - Link ticket mentions in emails, commits, etc, to git repos - Sync tickets between todo.sr.ht and GitHub/GitLab/etc - Sync GitLab MRs/GitHub PRs with mailing list threads and vice versa - Import git repos, tickets, etc from other services The current approach of implementing each specific use-case in its own job, one at a time, is not scalable to handling all of these use-cases and all of the options I want for customization. To address this, I would like to move dispatch.sr.ht to a YAML-based configuration file. Consider the following primitives: - Sources (e.g. a GitHub repo, lists.sr.ht mailing list, etc) - Events (e.g. a git commit, an email to the list) - Sinks (e.g. builds.sr.ht/submit) Sources emit events, which you can route to a sink. Your dispatch graph can be configured with a YAML file which lists routes: - name: forward changelog source: git.sr.ht/~sircmpwn/*.sr.ht/push/tag sink: lists.sr.ht/~sircmpwn/sr.ht-announce - name: push to build source: git.sr.ht/~sircmpwn/*.sr.ht/push/* sink: builds.sr.ht/submit secrets: true - name: patch to build source: lists.sr.ht/~sircmpwn/sr.ht-dev/email/patch sink: builds.sr.ht/submit secrets: false email-results: true - name: link commits to tickets source: git.sr.ht/~sircmpwn/*.sr.ht/push/* sink: todo.sr.ht/~sircmpwn/$repo_name - name: link tickets to commits source: todo.sr.ht/~sircmpwn/*.sr.ht sink: git.sr.ht/~sircmpwn/$tracker_name You can store this file in a git repo anywhere and point dispatch.sr.ht to it. It could be a branch of some project, or a standalone repo defining a graph for a number of projects, or a .dispatch.yml file sitting on your master branch. I would also consider letting you upload YAML files to dispatch.sr.ht on the fly (which will also be useful for migrating existing GitHub rules painlessly). In terms of implementation, sources would implement a number of interfaces. Things like: - Commit - Ticket - Email - Patch Sources emit events which implement one or more of these interfaces, and sinks have a list of the kinds of events they understand. For example, the builds.sr.ht sink might understand patches and commits. Patches could come from lists.sr.ht, commits from git.sr.ht or GitHub, etc. It could also support more specific events, e.g. "GitHubPullRequest", explicitly. Some other considerations: - Eventually the top-level project hub will be configuring these for you automatically when you create a new project. So instead of (1) create repo, (2) create tracker, (3) create list, (4) rig up builds on dispatch, (5) rig up ticket references, (etc), it'll be: (1) fill out project details and accept the sane defaults, then change them later if you need to. Therefore, these files need to be machine editable. - This format would also be fairly easy to update on the web, to avoid confronting users with a scary YAML file and a bunch of stuff to learn upfront. We can both render the graph as HTML and edit it with some simple forms. Whenever you try to update the graph, we'll do some quick validation. If, for example, we need you to authorize us to access your GitHub account, we'll push back with an error and a URL to correct it. Feedback welcome.