Hi all,

Since about a week, one of my webhooks has stopped receiving requests.

I double checked if it was still enabled and it had been deleted (maybe
a side effect of a database migration?). I created it again and it seems
to have stayed there:

~$ hut lists webhook list
#30 https://bot.diabeteman.com/sourcehut/

Here is the command I used to create it:

https://git.sr.ht/~rjarry/aerc/tree/master/item/contrib/ircbot/install-webhook.sh

Several patchsets have been received since then but they didn't trigger
a request on my http server. The last request I got is from august 12:

~# { grep -h POST.*sourcehut /var/log/nginx/access.log*; zgrep -h POST.*sourcehut /var/log/nginx/access.log*.gz; } | sort
[11/Aug/2023:21:06:28 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"
[11/Aug/2023:21:16:19 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"
[11/Aug/2023:21:27:20 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"
[11/Aug/2023:23:31:11 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"
[12/Aug/2023:00:54:41 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"
[12/Aug/2023:09:22:51 +0200] 2604:bf00:710:0:5054:ff:fec4:6bfb -> bot.diabeteman.com: POST /sourcehut/ HTTP/1.1 -> 200 "Go-http-client/1.1"

Here is my nginx config if that matters:

https://git.sr.ht/~rjarry/aerc/tree/master/item/contrib/ircbot/nginx.conf

NB: I know there are some ACLs here but I never got any reject for POST
requests on that virtual host.

Does that ring a bell to anyone?

Thanks!

-- 
Robin

Robin Jarry, Aug 22, 2023 at 19:01:
> Does that ring a bell to anyone?

Hi all, this is a gentle ping.

I assume either this is an issue on my end (PEBCAK?) or nobody uses
these webhooks.

Could a kind administrator have a look at the logs to clear my
confusion?

Thanks!

Do you see anything interesting when you dig through the webhook logs
via GraphQL?

Drew DeVault, Aug 30, 2023 at 08:59:
> Do you see anything interesting when you dig through the webhook logs
> via GraphQL?

I tried this query on https://lists.sr.ht/graphql

  query {
    me {
      canonicalName
      lists {
        cursor
        results {
          id, name, updated
          webhooks {
            cursor
            results {
              url
              deliveries {
                cursor
                results {
                  uuid, date, event, requestBody, responseBody,
                  responseHeaders, responseStatus
                }
              }
            }
          }
        }
      }
    }
  }

But I get errors:

  Native webhooks are not supported with web authentication

I don't know what that means.

On Wed, Aug 30, 2023 at 11:48:24AM +0200, Robin Jarry wrote:
> But I get errors:
>
>   Native webhooks are not supported with web authentication
>
> I don't know what that means.

You have to run this query through curl/hut(1), e.g. like this:

hut graphql lists

Thorben Günther, Aug 30, 2023 at 11:56:
> On Wed, Aug 30, 2023 at 11:48:24AM +0200, Robin Jarry wrote:
> > But I get errors:
> >
> >   Native webhooks are not supported with web authentication
> >
> > I don't know what that means.
>
> You have to run this query through curl/hut(1), e.g. like this:
>
> hut graphql lists

Ok thanks here's the output:

  ~$ hut graphql lists --stdin <<EOF
  query {
    user(username: "rjarry") {
      canonicalName
      list(name: "aerc-devel") {
        name
        webhook(id: 30) {
          id
          url
          query
          events
          deliveries {
            cursor
            results {
              uuid, date, event, requestBody, responseBody,
              responseHeaders, responseStatus
            }
          }
        }
      }
    }
  }
  EOF
  {
    "user": {
      "canonicalName": "~rjarry",
      "list": {
        "name": "aerc-devel",
        "webhook": {
          "id": 30,
          "url": "https://bot.diabeteman.com/sourcehut/",
          "query": "query {\n\twebhook {\n\t\tuuid\n\t\tevent\n\t\tdate\n\t\t... on PatchsetEvent {\n\t\t\tpatchset {\n\t\t\t\tid\n\t\t\t\tsubject\n\t\t\t\tversion\n\t\t\t\tprefix\n\t\t\t\tlist {\n\t\t\t\t\tname\n\t\t\t\t\towner {\n\t\t\t\t\t\t... on User {\n\t\t\t\t\t\t\tcanonicalName\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tsubmitter {\n\t\t\t\t\t... on User {\n\t\t\t\t\t\tcanonicalName\n\t\t\t\t\t}\n\t\t\t\t\t... on Mailbox {\n\t\t\t\t\t\tname\n\t\t\t\t\t\taddress\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n",
          "events": [
            "PATCHSET_RECEIVED"
          ],
          "deliveries": {
            "cursor": null,
            "results": []
          }
        }
      }
    }
  }

It looks like there were no deliveries. Which is in line with what
I observe. However, several patchsets were received since this webhook
was installed.

Gentle bump again. Could an admin have a look at the logs?

Thanks!

On 9/4/23 17:40, Robin Jarry wrote:
> Gentle bump again. Could an admin have a look at the logs?

I did, but couldn't really find anything relevant. I suspect there might 
be a bug involved, either something systemic with webhooks, or some 
botched error handling that your specific webhook is hitting. I'll look 
into it, but it might take a moment.

Cheers,
Conrad