Matthias Beyer <mail@beyermatthias.de> writes:

> On 06-02-2021 10:02:55, William Casarin wrote:
>> https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation
>
> This seems to be really cool and exactly what I thought of.
> Not sure, though, how much this is bound to kernel lists or whether this is just
> the default configuration of the tool?

I had the same thought. In the description b4 says it works with
public-inbox archives so I figured it might just work:

  "This is a helper utility to work with patches and pull requests made
  available via a **public-inbox** archive like lore.kernel.org"

Was about to try it out on the notmuch public-inbox.

Hey guys,

William Casarin <jb55@jb55.com> writes:
> Matthias Beyer <mail@beyermatthias.de> writes:
>
>> On 06-02-2021 10:02:55, William Casarin wrote:
>>> https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation
>>
>> This seems to be really cool and exactly what I thought of.
>> Not sure, though, how much this is bound to kernel lists or whether this is just
>> the default configuration of the tool?
>
> I had the same thought. In the description b4 says it works with
> public-inbox archives so I figured it might just work:
>
>   "This is a helper utility to work with patches and pull requests made
>   available via a **public-inbox** archive like lore.kernel.org"
>
> Was about to try it out on the notmuch public-inbox.

I packaged up b4 and submitted it to the list. It's pretty easy to
configure. I was testing the (unofficial) notmuch public-inbox so I just
added this config to my notmuch .git/config:

  [b4]
    # Where to look up threads by message id
    midmask = https://yhetil.org/notmuch/%s
    #
    # When recording Link: trailers, use this mask
    linkmask = https://yhetil.org/notmuch/%s

Then I was able to pull patch sets via:

  $ b4 am -v1 20201108190311.1397-1-jonas@bernoul.li

This pulls all the Reviewed-by: trailers from the thread as well and
adds them to the patches. Handy:

  Looking up https://yhetil.org/notmuch/20201108190311.1397-1-jonas%40bernoul.li
  Grabbing thread from yhetil.org/notmuch
  Analyzing 68 messages in the thread
  ---
  Writing ./20201116_jonas_another_set_up_emacs_cleanup.mbx
    [PATCH 01/27] emacs: silence byte-compiler
      + Reviewed-by: William Casarin <jb55@jb55.com>
    [PATCH 02/27] emacs: define notmuch-message-mode-map explicitly
      + Tested-by: William Casarin <jb55@jb55.com>
    [PATCH 03/27] emacs: add doc-string to notmuch-tree-mode-map
    [PATCH 04/27] emacs: don't fset keymaps
      + Reviewed-by: William Casarin <jb55@jb55.com>
      + Tested-by: William Casarin <jb55@jb55.com>

  [...]

Cheers,
Will

Would be nice if b4 supported notmuch databases instead of public
inboxes. I've all my mail (lots and lots of public mailing lists) in
notmuch an my primary workflow is really driven by queries to the local
database.

> On Feb 6, 2021, at 17:28, andi@srht.l.notmuch.email wrote:
> 
> Would be nice if b4 supported notmuch databases instead of public
> inboxes. I've all my mail (lots and lots of public mailing lists) in
> notmuch an my primary workflow is really driven by queries to the local
> database.

It’s really odd that b4 doesn’t support applying raw mbox over stdin. I might look into adding that, then you could use notmuch show —format=mbox thread:id and pipe it to b4 am.

Then I wouldn’t have needed to patch b4 for sourcehut support... could just curl the mbox and pipe it.

Please excuse this non plaintext email as I’m sending it from my phone.

Cheers,

On Sat, Feb 06, 2021 at 10:42:57AM -0800, William Casarin wrote:
> > This seems to be really cool and exactly what I thought of.
> > Not sure, though, how much this is bound to kernel lists or whether this is just
> > the default configuration of the tool?
> 
> I had the same thought. In the description b4 says it works with
> public-inbox archives so I figured it might just work:
> 
>   "This is a helper utility to work with patches and pull requests made
>   available via a **public-inbox** archive like lore.kernel.org"
> 
> Was about to try it out on the notmuch public-inbox.

While that's the case, I believe that what sr.ht calls "public-inbox" is
not the same as "public-inbox" as defined by public-inbox.org.

-K

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> On Sat, Feb 06, 2021 at 10:42:57AM -0800, William Casarin wrote:
>> > This seems to be really cool and exactly what I thought of.
>> > Not sure, though, how much this is bound to kernel lists or whether this is just
>> > the default configuration of the tool?
>> 
>> I had the same thought. In the description b4 says it works with
>> public-inbox archives so I figured it might just work:
>> 
>>   "This is a helper utility to work with patches and pull requests made
>>   available via a **public-inbox** archive like lore.kernel.org"
>> 
>> Was about to try it out on the notmuch public-inbox.
>
> While that's the case, I believe that what sr.ht calls "public-inbox" is
> not the same as "public-inbox" as defined by public-inbox.org.

ah yes I wasn't referring to anything wrt. sourcehut here. I tested in
on a public-inbox instance here[1] and it worked great!

One thing that we've been discussing is the ability for `b4 am` to take
in an mbox over stdin, is there any reason why it doesn't have this
capability? That would allow us to curl sourcehut mbox's, or get mbox's
from a notmuch query. I couldn't find a way to do this.

Awesome tool!

Cheers,
Will

[1] https://yhetil.org/notmuch/

William Casarin <jb55@jb55.com> writes:

> Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:
>> While that's the case, I believe that what sr.ht calls "public-inbox" is
>> not the same as "public-inbox" as defined by public-inbox.org.
>
> ah yes I wasn't referring to anything wrt. sourcehut here. I tested in
> on a public-inbox instance here[1] and it worked great!
>
> One thing that we've been discussing is the ability for `b4 am` to take
> in an mbox over stdin, is there any reason why it doesn't have this
> capability? That would allow us to curl sourcehut mbox's, or get mbox's
> from a notmuch query. I couldn't find a way to do this.

ah nm, looks like I missed the -m option:

b4 am 20210206202618.32583-1-jb55@jb55.com -m mbox

Cheers,
Will

William Casarin <jb55@jb55.com> writes:

>> On Feb 6, 2021, at 17:28, andi@srht.l.notmuch.email wrote:
>> 
>> Would be nice if b4 supported notmuch databases instead of public
>> inboxes. I've all my mail (lots and lots of public mailing lists) in
>> notmuch an my primary workflow is really driven by queries to the local
>> database.
>
> It’s really odd that b4 doesn’t support applying raw mbox over stdin.
> I might look into adding that, then you could use notmuch show
> —format=mbox thread:id and pipe it to b4 am.

looks like it does via the -m option. Here's a notmuch-am script that
does a b4 am on a notmuch thread:

  #!/usr/bin/env bash
  set -eou pipefail
  mbox=$(mktemp)
  notmuch show --format=mbox "$1" > "$mbox"
  msgid=$(grep -i ^message-id "$mbox" | cut -d" " -f2 | head -n1)
  b4 am "$msgid" -m "$mbox"
  rm -f "$mbox"


Example:

  $ notmuch am thread:000000000001b70e
  Analyzing 6 messages in the thread
  ---
  Writing ./20210206_jb55_b4_init_at_0_6_2.mbx
    [PATCH] b4: init at 0.6.2
      + Reviewed-by: Xinglu Chen <public@yoctocell.xyz> (✓ DKIM/yoctocell.xyz)
      + Reviewed-by: Matthias Beyer <mail@beyermatthias.de>

William Casarin <jb55@jb55.com> writes:

> William Casarin <jb55@jb55.com> writes:
>
>>> On Feb 6, 2021, at 17:28, andi@srht.l.notmuch.email wrote:
>>> 
>>> Would be nice if b4 supported notmuch databases instead of public
>>> inboxes. I've all my mail (lots and lots of public mailing lists) in
>>> notmuch an my primary workflow is really driven by queries to the local
>>> database.
>>
>> It’s really odd that b4 doesn’t support applying raw mbox over stdin.
>> I might look into adding that, then you could use notmuch show
>> —format=mbox thread:id and pipe it to b4 am.
>
> looks like it does via the -m option. Here's a notmuch-am script that
> does a b4 am on a notmuch thread:
>
>   #!/usr/bin/env bash
>   set -eou pipefail
>   mbox=$(mktemp)
>   notmuch show --format=mbox "$1" > "$mbox"
>   msgid=$(grep -i ^message-id "$mbox" | cut -d" " -f2 | head -n1)
>   b4 am "$msgid" -m "$mbox"
>   rm -f "$mbox"

Here's a little fzf script that uses this:

https://asciinema.org/a/KB25MsQfyq3WPPAwdewQ2LvYS

That's enough fun for today :)

On 06-02-2021 19:28:29, William Casarin wrote:
> William Casarin <jb55@jb55.com> writes:
> > It’s really odd that b4 doesn’t support applying raw mbox over stdin.
> > I might look into adding that, then you could use notmuch show
> > —format=mbox thread:id and pipe it to b4 am.
> 
> looks like it does via the -m option. Here's a notmuch-am script that
> does a b4 am on a notmuch thread:
> 

So from what I see, we don't even need a public-inbox anymore, am I right?
Seems like all the tools are there already and all one needs to do is to
integrate b4 into their mail workflow (neomutt+notmuch for me, for example).

It might still be a valuable option if this list gets some tracktion, though.

Matthias

Hi,

so, to sum it up real quick:

* We discussed whether we want to run a public-inbox[0] for this ML, giving the
  users the option to mirror it as well as subscribing via NNTP for example.

* The initial discussion about public-inbox took place because we wanted to use
  b4 [1], for its features of aggregating git-trailers into the appropriate
  commit messages.
  Though, we found that b4 can also do this for mbox files, which ultimatively
  makes it possible to pipe a thread directly from ones favourite MUA (e.g.
  mutt) to b4 and then to git-am.  notmuch integration is also possible [2] as
  William has mentioned earlier in this thread.  I bet emacs/gnus [3], aerc and
  other plaintext MUAs [4] are also possible!

* We also discussed about self-hosting this ML.
  We (kind of?) agreed that this might be an idea later, but right now it would
  just be effort that does _not_ result in value put into nixpkgs itself, so it
  really is not necessary. Maybe at some later point. Mailinglists can be moved
  easily anyways.


Did I miss anything?

Matthias

[0] https://public-inbox.org
[1] https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation
[2] https://lists.sr.ht/~andir/nixpkgs-dev/%3C87eehtrqa6.fsf%40jb55.com%3E#%3C874kiosgiq.fsf@jb55.com%3E
[3] https://git.kyleam.com/piem/about/
[4] https://useplaintext.email/

On 06-02-2021 20:56:43, William Casarin wrote:
> >   #!/usr/bin/env bash
> >   set -eou pipefail
> >   mbox=$(mktemp)
> >   notmuch show --format=mbox "$1" > "$mbox"
> >   msgid=$(grep -i ^message-id "$mbox" | cut -d" " -f2 | head -n1)
> >   b4 am "$msgid" -m "$mbox"
> >   rm -f "$mbox"
> 
> Here's a little fzf script that uses this:
> 
> https://asciinema.org/a/KB25MsQfyq3WPPAwdewQ2LvYS
> 

Would you share this fzf script please?

Maybe we need a repository "nixpkgs-ml-tooling" or something like this, where we
can collect such tools?

Matthias

Hi again.

Sorry for the noise, but I have some more ideas for tooling to drive this ML.


First of all, I think we really need a repo to put all the tools and scripts we
use.
The b4-notmuch integration for example, but also the script that is used to
submit patches as github PRs.


On submitting patches to github PRs:
From what I see, the PR message right now includes the following data (using the
b4 patchset [0] for reference):

* git trailers
* Message-Id
* Link to the patchset

I think this can be improved with the following changes (I would submit a patch,
but I don't know whether the tool/script is even open source at this point):

* In the PR: A "mailto" link to reply on the thread the patch was submitted
  with, possibly with CC to the patch Author.
  This way, people on github are one click from replying to a patchset.
* On this ML: An automatically generated message with
    * A link to the PR
    * A summary of the "b4 am | git-am" output (which trailers were added, for
      example)
    * A git remote URL to `git fetch` the patchset from github
      (This would be nice so I can see my patchset in my git-log output as it
      appears on github)
    * A link to reply to the issue on github via mail. I bet this is possible as
      github generates message IDs rather transparently (see below).


Another thought that I currently have is that comments on the PR are not
forwarded to me and comments I make on the patchset are not forwarded to github.
I think that is possible, but am not sure how.
I also think that a proper mirroring of comments is _hard_ because one easily
can build an endless loop.

From what I see, github sends a mail for each comment, and the message ID of
the mail refers to the comment URL!!!

For example: I have a mail that has the message ID

    Message-ID: <NixOS/nixpkgs/pull/97666/c702657111@github.com>

that refers to:

    https://github.com/NixOS/nixpkgs/pull/97666/#issuecomment-702657111

One can clearly see the scheme:

    Message-ID:     <user>/<repo>/pull/<PR ID>/c<number>@github.com
    URL: github.com/<user>/<repo>/pull/<PR ID>/#issuecomment-<number>

That is really nice. Maybe we can make something out of that?

[0] https://github.com/NixOS/nixpkgs/pull/112244

On Sun, Feb 07 2021, Matthias Beyer wrote:

> First of all, I think we really need a repo to put all the tools and scripts we
> use.
> The b4-notmuch integration for example, but also the script that is used to
> submit patches as github PRs.
>
>
> On submitting patches to github PRs:
> From what I see, the PR message right now includes the following data (using the
> b4 patchset [0] for reference):
>
> * git trailers
> * Message-Id
> * Link to the patchset
>
> I think this can be improved with the following changes (I would submit a patch,
> but I don't know whether the tool/script is even open source at this point):

Andreas' mail2pr script can be found at <https://github.com/andir/mail2pr>,
if this is what you are referring to.

On 07-02-2021 14:55:58, Xinglu Chen wrote:
> Andreas' mail2pr script can be found at <https://github.com/andir/mail2pr>,
> if this is what you are referring to.

Nice, thanks!

Matthias Beyer <mail@beyermatthias.de> writes:

> On 06-02-2021 20:56:43, William Casarin wrote:
>> Here's a little fzf script that uses this:
>> 
>> https://asciinema.org/a/KB25MsQfyq3WPPAwdewQ2LvYS
>> 
>
> Would you share this fzf script please?

It's something like:

get_threadid() {
  notmuch search --output=summary --format=json "$@" |
  jq -rcC '.[] | [.date_relative,.subject,.authors,.thread] | @tsv' |
  fzf -m |
  cut -d $'\t' -f4 |
  sed 's,^,thread:,g'
}

notmuch am $(get_threadid to:nixpkgs-dev)

> Maybe we need a repository "nixpkgs-ml-tooling" or something like this, where we
> can collect such tools?

sure