= Dog Food Experience Report
As mentioned in the latest release kickoff[0], we attempted to use our
own radicle-link suite of tools. We managed to get things working with
a few (too many) papercuts. This email is intended to capture those
sharp edges so that we have a list of issues to solve and/or discuss.
== git version check
The seed we were running was using a git version of 2.25.1. The
`must_namespace` function is supposed to check the git version and
allow us to check if we have access to a particular git functionality
for namespacing. This seemed to fail on the seed and ended up in `ls
refs` failing.
== link-replication validation bug
Which leads us to a bug in link-replication. The `pre_validate` for
`ForClone` did not seem to guard against the missing `rad` refs. This
allowed the code to reach the call to `UpdatedTips`, which rightfully
told us that there's a `BUG` since we should have a `rad/id`, which
was supposed to be checked by `pre_validate`.
== lnk-identities track
This code is still not updated to respect the latest link-tracking
changes. It requires `--peer` to be present when calling the track
subcommand.
There was also a bug where it fails if the URN didn't exist in the
monorepo, which shouldn't be required.
== Diff does not show delegation changes
In `lnk identities {person, project} {diff, accept}` the diff between
the two identities does not contain the difference between delegation
changes. It should do this, otherwise it's hard to determine what was
changed in the terms of delegations.
== No flow for linking working copy to monorepo
It's possible to land in a scenario where you have an existing working
copy that does not have a remote that points to radicle-link
storage, but we do have a URN for that existing project. It would be
nice to look at what is required for this flow to work, e.g. finding
the include file for that URN, adding the remote. We should consider
whether we need tooling for this or whether it should just be well
documented on how to set it up.
[0]: https://lists.sr.ht/~radicle-link/dev/%3CCAH_DpYQX4HmvU0%2BeXzwM1wgOSrL8y21CZnwWY2EtOhiSrC0boQ%40mail.gmail.com%3E
A few additional things I noticed:
== Gitoxide signal handling woes
The kaput gitoxide signal handling is causing our signal handlers to be
lost in linkd, which means that linkd keeps failing to delete it's
socket files.
== Confirmation required prompts in lnk sync
lnk sync doesn't prompt you if replication reveals that a URN requires
confirmation on some changes. It seems like it should be straightforward
to implement and possibly we could do something nice like suggest the
command to accept the changes
== Include files
Two things. Firstly there's no easy way to set up the include files for
an existing project. Secondly the include files are apparently not
updated during `lnk sync`. I think we just forgot to wire it up.
== Peer listing
Frequently I want to see what peers I know about for a URN. Listing refs
is fiddly for this. It seems to me like we should probably have a simple
command for listing the peers for a particular URN, possibly with some
options to scope by delegate.
And I forgot one thing :)
== Flow for merging a new delegate
When Alex added my URN as a delegate, it required me to perform an
update (whether the same update or a no-op), to have my signature on
my local view of the identity. I could then merge his
changes. Followed by him merging my changes. This seems like a lot of
steps to do a single change. We should research options on how to make
this easier.
On Tue Jun 21, 2022 at 10:01 PM IST, Alex Good wrote:
> == Peer listing>> Frequently I want to see what peers I know about for a URN. Listing refs> is fiddly for this. It seems to me like we should probably have a simple> command for listing the peers for a particular URN, possibly with some> options to scope by delegate.
I believe `lnk identities {project, person} tracked` will do this for
you. Scoping by delegate would be a new functionality :) But perhaps they could be listed as the display output when you get/list an identity.
More shenanigans
== Non-fast forward during push
Having not synced from the seed, I tried to push a change which
resulted in:
---
request-pull replication error: non-fast-forward update of refs/namespaces/hnrkxafojjsz4m55qxbwigh1z8sdt7mai81gy/refs/remotes/hydjhd8q9nkoxzkpddhcuue9xzpfr4bn6d44fo1f4q1japwm4brhh6/rad/signed_refs (current: 9c3cbbbdf0e8b7242b7b5490b8611870423c570e, new: a91e8f917a6a0541a0159aac1decfce19e233cf8)
---
Alex had pushed changes, but we're not sure why `current` was not a
fast-forward of `new`.
== All references are returned from gitoxide
Every time we push, we get a slew of messages saying all references
were updated. We should remove this print while we're still depending
on gitoxide as it renders the output confusing and useless.
Even more shenanigans
== Rejected updates of rad/id
Alex and I peformed a series of updates and merges to add myself as a
delegate to the radicle-link project we're hosting. The series of
steps were as follows:
1. Alex created radicle-link and pushed to the seed
2. I replicate radicle-link to my storage
3. Alex added my rad/id to the set of delegates and pushed to the seed
4. I replicate the changes
5. I performed the same delegates change
6. I merged Alex's update with my update, and pushed
7. Alex pulled and merged my update with his update
Now, when Alex replicates my changes to the project, he will always
have a rejected update of the project's rad/id. While both documents
have the same representation, their commit histories will differ.
I need to understand whether this should be expected, and rejcted
updates is more of a warning, or if this flow can be improved to not
run into this.
== Can't specify branch when fetching from remote
When setting up a remote for a peer, we cannot do:
git fetch other-peer kickflip
Instead, the reference needs to be of the form
`<peer_id>/heads/kickflip`. It might be that we need to rethink the
fetch refspecs.
== git push did not sync branch
When I performed a `git push` of a branch, the seed didn't end up
pulling it. Instead, I needed to `lnk sync` to ensure it was there. We
might have missed something in the `gitd` implementation for making a
`request-pull` call.
== include files not updating on track
The call to `include::update` seems to be having no effect on the
include file. Easiest way to see this is by calling `lnk identities
track` and not seeing any peers in the include file.
On Fri Jun 24, 2022 at 9:29 AM IST, Fintan Halpenny wrote:
> == include files not updating on track>> The call to `include::update` seems to be having no effect on the> include file. Easiest way to see this is by calling `lnk identities> track` and not seeing any peers in the include file.
Alex, can you confirm this on your side? I just tried tracking your
peer and it added the entry in the include file. I was rebased on my
tracking improvement changes though.
Otherwise, if this works fine, then we just need to add include
updates to other sections of the code. Possibly add some tooling for
linking and updating too.
On 24/06/22 10:19am, Fintan Halpenny wrote:
> On Fri Jun 24, 2022 at 9:29 AM IST, Fintan Halpenny wrote:> > == include files not updating on track> >> > The call to `include::update` seems to be having no effect on the> > include file. Easiest way to see this is by calling `lnk identities> > track` and not seeing any peers in the include file.>> Alex, can you confirm this on your side? I just tried tracking your> peer and it added the entry in the include file. I was rebased on my> tracking improvement changes though.
This is still not working for me. However, the reason is probably not to
do with the include file logic per se. The problem appears to be that in
my monorepo your remote has no `rad/self` reference, which means that it
doesn't appear as a tracked peer in the
librad::git::identities::relations::tracked` iterator.
Some notes and issues encountered in migrating our cli to linkd:
* There's a `--tmp-root` flag and a `--lnk-home` flag, it seems like the naming should be consistent, eg. `--root` and `--tmp-root` (we use `--root` for our services)
* It seems like there's no way to pass the key passphrase via the command-line, which forces the use of ssh-agent -- though this is fine for user machines, it's a bit annoying for servers
* `--help` doesn't describe what values can be passed to `--track`
* After restarting the daemon a couple times, it will no longer startup, giving me an "address already in use" no matter what port I use in `--protocol-listen`
Good news is I’m going to be able to delete a lot of code if the migration works. Biggest unknown at the moment is link-hooks integration.
On Tue Jul 5, 2022 at 12:01 PM IST, Alexis Sellier wrote:
> Good news is I’m going to be able to delete a lot of code if the migration works. Biggest unknown at the moment is link-hooks integration.
Whew!
Regarding link-hooks, I'm currently looking into it. I had a chat with
Alex around the technical scoping of this. He rightly pointed out that
there's nothing that communicates a replication's result on the end of
a request-pull. We're thinking that the ProtocolEvent is the right
structure for notifying a replication's results.
> Regarding link-hooks, I'm currently looking into it. I had a chat with> Alex around the technical scoping of this. He rightly pointed out that> there's nothing that communicates a replication's result on the end of> a request-pull. We're thinking that the ProtocolEvent is the right> structure for notifying a replication's results.
Hmm are you saying that if a client request-pulls a seed (ie. pushes to a seed),
there's no hook that will be run on success on the seed? That's our only use-case for
hooks at the moment, since we'd want to set HEAD on success.
On Tue Jul 5, 2022 at 12:30 PM IST, Alexis Sellier wrote:
>> > Regarding link-hooks, I'm currently looking into it. I had a chat with> > Alex around the technical scoping of this. He rightly pointed out that> > there's nothing that communicates a replication's result on the end of> > a request-pull. We're thinking that the ProtocolEvent is the right> > structure for notifying a replication's results.>> Hmm are you saying that if a client request-pulls a seed (ie. pushes to a seed),> there's no hook that will be run on success on the seed? That's our only use-case for> hooks at the moment, since we'd want to set HEAD on success.
Ya, that's how the discussion came up. We're looking into satisfying
that use case, and any request-pull success in general.
* I've successfully integrated replv3 into the radicle-cli flow (cloning, tracking,
patch creation, patch review and merge...). The next step is server-side integration
via hooks.
* I've noticed that the output of replicate and request-pull is the same, whether or not
refs were actually updated. Basically, the `applied` and `updated` ref list will always
contain all refs, whether or not these refs were actually updated. This is a problem
for generated user-facing output, as the user has no way of knowing if something changed
or not.
------- Original Message -------
On Tuesday, July 5th, 2022 at 14:52, Fintan Halpenny <fintan.halpenny@gmail.com> wrote:
> > > On Tue Jul 5, 2022 at 12:30 PM IST, Alexis Sellier wrote:> > > > Regarding link-hooks, I'm currently looking into it. I had a chat with> > > Alex around the technical scoping of this. He rightly pointed out that> > > there's nothing that communicates a replication's result on the end of> > > a request-pull. We're thinking that the ProtocolEvent is the right> > > structure for notifying a replication's results.> > > > Hmm are you saying that if a client request-pulls a seed (ie. pushes to a seed),> > there's no hook that will be run on success on the seed? That's our only use-case for> > hooks at the moment, since we'd want to set HEAD on success.> > > Ya, that's how the discussion came up. We're looking into satisfying> that use case, and any request-pull success in general.
Few more notes on sync output:
* It seems like if the peer-id is wrong for the seed, we get a `NoConnection` error.
It would be nice to get a more specific error so that we can relay that back to the user.
* When 'sync' is not able to make a connection to the seed at all, eg. wrong address or
server is down, it just hangs for about 1 minute. I wonder how we should treat that?
Other tools will return immediately if there is no host on the other end, which would
be the desired behavior. Should we handle timeouts ourselves and just use a short (eg. 3s)
timeout?
------- Original Message -------
On Thursday, July 7th, 2022 at 13:10, Alexis Sellier <alexis@radicle.foundation> wrote:
> > > * I've successfully integrated replv3 into the radicle-cli flow (cloning, tracking,> patch creation, patch review and merge...). The next step is server-side integration> via hooks.> > * I've noticed that the output of replicate and request-pull is the same, whether or not> refs were actually updated. Basically, the `applied` and `updated` ref list will always> contain all refs, whether or not these refs were actually updated. This is a problem> for generated user-facing output, as the user has no way of knowing if something changed> or not.> > > ------- Original Message -------> On Tuesday, July 5th, 2022 at 14:52, Fintan Halpenny fintan.halpenny@gmail.com wrote:> > > > > On Tue Jul 5, 2022 at 12:30 PM IST, Alexis Sellier wrote:> > > > > > Regarding link-hooks, I'm currently looking into it. I had a chat with> > > > Alex around the technical scoping of this. He rightly pointed out that> > > > there's nothing that communicates a replication's result on the end of> > > > a request-pull. We're thinking that the ProtocolEvent is the right> > > > structure for notifying a replication's results.> > > > > > Hmm are you saying that if a client request-pulls a seed (ie. pushes to a seed),> > > there's no hook that will be run on success on the seed? That's our only use-case for> > > hooks at the moment, since we'd want to set HEAD on success.> > > > Ya, that's how the discussion came up. We're looking into satisfying> > that use case, and any request-pull success in general.
On Thu Jul 7, 2022 at 12:10 PM IST, Alexis Sellier wrote:
> * I've successfully integrated replv3 into the radicle-cli flow (cloning, tracking,> patch creation, patch review and merge...). The next step is server-side integration> via hooks.
That's great! It would be good to get your eyes on this further
discussion on hooks I just kicked off
https://lists.sr.ht/~radicle-link/dev/%3CCL9K0UAFNS9Z.39B2MK9CA4NIQ%40haptop%3E
sorry, just realised I posted it to dev when really it should have been discuss!
> * I've noticed that the output of replicate and request-pull is the same, whether or not> refs were actually updated. Basically, the `applied` and `updated` ref list will always> contain all refs, whether or not these refs were actually updated. This is a problem> for generated user-facing output, as the user has no way of knowing if something changed> or not.
Ya, I documented this previously in the thread:
https://lists.sr.ht/~radicle-link/discuss/%3CCKVZUEU2M3EV.3ZLUSXZ7O9GV%40haptop%3E#%3CCKXKOEA2XG7N.392E5967BW6PJ@haptop%3E
It's an issue with gitoxide (of course...)
>>> ------- Original Message -------> On Tuesday, July 5th, 2022 at 14:52, Fintan Halpenny <fintan.halpenny@gmail.com> wrote:>>> > >> > >> > On Tue Jul 5, 2022 at 12:30 PM IST, Alexis Sellier wrote:> > >> > > > Regarding link-hooks, I'm currently looking into it. I had a chat with> > > > Alex around the technical scoping of this. He rightly pointed out that> > > > there's nothing that communicates a replication's result on the end of> > > > a request-pull. We're thinking that the ProtocolEvent is the right> > > > structure for notifying a replication's results.> > > >> > > Hmm are you saying that if a client request-pulls a seed (ie. pushes to a seed),> > > there's no hook that will be run on success on the seed? That's our only use-case for> > > hooks at the moment, since we'd want to set HEAD on success.> > >> > >> > Ya, that's how the discussion came up. We're looking into satisfying> > that use case, and any request-pull success in general.