~radicle-link/dev

lnk clone v1 PROPOSED

This patch implements a lnk clone command, as well as a few other bits
and pieces to make the integration with gitd easier.

The most interesting thing here is the addition of
`librad::git::identities::project::heads` which contains functions for
working with the default branch of a project identity. This is used by
`lnk clone` to allow checking out a project without having to know which
peer you want to check out.

I also updated lnk-identities to correctly set up the include file for
checked out and created identities (this logic looked to have gotten a
little lost in translation from the daemon). While we were here Fintan
and I also modified `gitd` so that it accepts URLs in the same form as
those in the include file so users can use `url.<base>.insteadOf` to
point rad remotes at a local gitd.

Published-At: https://github.com/alexjg/radicle-link/tree/patches/lnk-clone/v1

Alex Good (4):
  Add default_branch_head and set_default_branch
  lnk-identities: update path logic and set up include
  Make gitd accept the URL format of include files
  Add lnk clone

 bins/Cargo.lock                               |   3 +
 cli/gitd-lib/src/git_subprocess.rs            |  12 +-
 cli/gitd-lib/src/git_subprocess/command.rs    |  12 +-
 cli/gitd-lib/src/lib.rs                       |   1 +
 cli/gitd-lib/src/processes.rs                 |  20 +-
 cli/gitd-lib/src/server.rs                    |   5 +-
 cli/gitd-lib/src/ssh_service.rs               |  50 +++
 cli/lnk-exe/src/cli/args.rs                   |   1 +
 cli/lnk-identities/Cargo.toml                 |   3 +
 cli/lnk-identities/src/cli/args.rs            |  10 +-
 cli/lnk-identities/src/cli/eval/person.rs     |   7 +-
 cli/lnk-identities/src/cli/eval/project.rs    |   7 +-
 cli/lnk-identities/src/git/checkout.rs        |  70 ++--
 cli/lnk-identities/src/git/existing.rs        |  16 +-
 cli/lnk-identities/src/git/new.rs             |  20 +-
 cli/lnk-identities/src/identity_dir.rs        |  38 ++
 cli/lnk-identities/src/lib.rs                 |   1 +
 cli/lnk-identities/src/person.rs              |   5 +-
 cli/lnk-identities/src/project.rs             |  32 +-
 .../t/src/tests/git/checkout.rs               |   5 +-
 .../t/src/tests/git/existing.rs               |   6 +-
 cli/lnk-identities/t/src/tests/git/new.rs     |   5 +-
 cli/lnk-sync/Cargo.toml                       |  10 +-
 cli/lnk-sync/src/cli/args.rs                  |  23 +-
 cli/lnk-sync/src/cli/main.rs                  |  46 ++-
 cli/lnk-sync/src/forked.rs                    |  87 ++++
 cli/lnk-sync/src/lib.rs                       |   3 +
 librad/src/git/identities/project.rs          |   2 +
 librad/src/git/identities/project/heads.rs    | 305 ++++++++++++++
 librad/t/src/integration/scenario.rs          |   1 +
 .../scenario/default_branch_head.rs           | 387 ++++++++++++++++++
 test/it-helpers/Cargo.toml                    |   3 +
 test/it-helpers/src/lib.rs                    |   1 +
 test/it-helpers/src/working_copy.rs           | 291 +++++++++++++
 34 files changed, 1358 insertions(+), 130 deletions(-)
 create mode 100644 cli/gitd-lib/src/ssh_service.rs
 create mode 100644 cli/lnk-identities/src/identity_dir.rs
 create mode 100644 cli/lnk-sync/src/forked.rs
 create mode 100644 librad/src/git/identities/project/heads.rs
 create mode 100644 librad/t/src/integration/scenario/default_branch_head.rs
 create mode 100644 test/it-helpers/src/working_copy.rs

-- 
2.36.1
Does it indeed work to set HEAD to an oid? I was under the impression that
`git-clone` would expect it to be a symref; if it is not, it doesn't know which
branch to check out. I believe it will leave the cloned repo in a detached head
state, which is gradually less scary than not checking out anything at all.



          
          
          
        
      

      
      
      
      

      

      
      
      
      

      
      
        
          






Awesome, one thought around setting the `HEAD` from my perspective:

Currently, from what I understand, the latest commit amongst delegates
is the one chosen as the HEAD commit for the project. I'd propose that we eventually
also allow the caller to specify a threshold of delegates that should "agree"
on this commit (ie. it's in their published history). This could constitute
a sort of quorum, and would be more representative of the intent of the delegate
group than a single delegate. It would also prevent attacks where a single
delegate is able to patch `HEAD` without consent from other delegates.

The `HEAD` would then be set according to this threshold, which could either be
the same threshold that is used to find quorum when updating a project identity,
or a separate configurable threshold that is only used for the code. For projects
with lots of delegates, it might make sense to only require a small subset to
push a commit, to reduce coordination.

Just wanted to mention this as this is the strategy we've planned to use on our
own git bridge, though it hasn't been implemented yet, and we'd benefit from
aligning on this.

------- Original Message -------
This function summarises Rust.
I think I've addressed most of the other feedback in the v2 re-roll but
I would like to leave this particular point for a separate patch if
possible. This is because I introduced a few other testing helpers
(`TestProject::maintainers`) and would like to spend a bunch of time
updating the tests to use them but I want to get `lnk-clone` in and done
sooner than that so we can start using it.
You mean as the argument to `checkout`? The difficulty with that would
be that the fetch logic in `Peer::checkout` and `Local::checkout` needs
a `CanOpenStorage`.
Next
Although I think it would generally be preferable to promote explicit signoff and nudge people away from trusting random names from the internet, I believe what you want is to pick the merge base (i.e. most recent common commit).



          
          
          
        
      

      
      
      
      
      
      
      
      
      
      
      
      
      
      

      

      
      
      
      

      

      
      
      
      
      
      
      
      

      

      
      
      
      
      
      
      
      

      

      
      
      
      
      
      
      
      
      
      
      
      

      
      
        
          






Yes. This is tricky to solve though. In order for `git clone` to do
sensible things we need to have `refs/HEAD` be a symbolic ref pointing
at`refs/namespaces/<urn>/refs/heads/<default branch>`. We _do_ know that
there is something under `refs/namespaces/<urn>/refs/heads/<default branch>`
but it might be ahead of the merge base.

I think what we want to do then is to actually update the local view to
point at the merge base. This does mean that you might lose the local
view's ref if it is ahead of the merge base, the implication here is
that the only time you would do this is if the local view is not really
interesting - i.e. the local view is not a delegate.

This makes me think that maybe the correct thing to do is to check if
the local peer is a delegate - if they are then just set the default
head to whatever the local peer thinks it is - otherwise set it to the
merge base. Make sense?
Debatable and should probably be a config option. Two viewpoints:

1. When I do `git clone` I want the latest version of the URN, the fact
   that there's a monorepo between me and the network is sort of
   irrelevant, I just want to grab from my seeds the latest state and get
   it into a working copy.
2. I want to have explicit control over when I request things from the
   network, I never want to do `sync` on `git clone`

I'm in the former camp but the latter doesn't seem that weird to me. For
now I think we could just build `1` as it's what we need to be able to
use this thing.
Next
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git:

curl -s https://lists.sr.ht/~radicle-link/dev/patches/32542/mbox | git am -3
Learn more about email & git

[PATCH v1 1/4] Add default_branch_head and set_default_branch Export this patch

When checking out projects from the monorepo it is useful to set the
`refs/namespaces/<urn>/HEAD` reference to the default branch of the
project so that the resulting working copy is in a useful state (namely
pointing at the latest commit for the default branch).

In general this is not possible because delegates may have diverging
views of the project, but often they do not disagree. Add
`librad::git::identities::project::heads::default_branch_head` to
determine if there is an agreed on default branch commit and
`librad::git::identities::project::heads::set_default_branch` to set the
local `HEAD` ref where possible.

Signed-off-by: Alex Good <alex@memoryandthought.me>
---
 librad/src/git/identities/project.rs          |   2 +
 librad/src/git/identities/project/heads.rs    | 305 ++++++++++++++
 librad/t/src/integration/scenario.rs          |   1 +
 .../scenario/default_branch_head.rs           | 387 ++++++++++++++++++
 test/it-helpers/Cargo.toml                    |   3 +
 test/it-helpers/src/lib.rs                    |   1 +
 test/it-helpers/src/working_copy.rs           | 291 +++++++++++++
 7 files changed, 990 insertions(+)
 create mode 100644 librad/src/git/identities/project/heads.rs
 create mode 100644 librad/t/src/integration/scenario/default_branch_head.rs
 create mode 100644 test/it-helpers/src/working_copy.rs

diff --git a/librad/src/git/identities/project.rs b/librad/src/git/identities/project.rs
index 753358bf..ed9f003d 100644
--- a/librad/src/git/identities/project.rs
+++ b/librad/src/git/identities/project.rs
@@ -8,6 +8,8 @@ use std::{convert::TryFrom, fmt::Debug};
use either::Either;
use git_ext::{is_not_found_err, OneLevel};

pub mod heads;

use super::{
    super::{
        refs::Refs as Sigrefs,
diff --git a/librad/src/git/identities/project/heads.rs b/librad/src/git/identities/project/heads.rs
new file mode 100644
index 00000000..447e352e
--- /dev/null
+++ b/librad/src/git/identities/project/heads.rs
@@ -0,0 +1,305 @@
use std::{collections::BTreeSet, convert::TryFrom, fmt::Debug};

use crate::{
    git::{
        storage::{self, ReadOnlyStorage},
        Urn,
    },
    identities::git::VerifiedProject,
    PeerId,
};
use git_ext::RefLike;
use git_ref_format::{lit, name, Namespaced, Qualified, RefStr, RefString};

#[derive(Clone, Debug, PartialEq)]
pub enum DefaultBranchHead {
    /// Not all delegates agreed on an ancestry tree. Each set of diverging
    /// delegates is included as a `Fork`
    Forked(BTreeSet<Fork>),
    /// All the delegates agreed on an ancestry tree
    Head {
        /// The most recent commit for the tree
        target: git2::Oid,
        /// The branch name which is the default branch
        branch: RefString,
    },
}

#[derive(Clone, Debug, std::hash::Hash, PartialEq, Eq, PartialOrd, Ord)]
pub struct Fork {
    /// Peers which are in the ancestry set of this fork but not the tips. This
    /// means that these peers can appear in multiple forks
    pub ancestor_peers: BTreeSet<PeerId>,
    /// The peers pointing at the tip of this fork
    pub tip_peers: BTreeSet<PeerId>,
    /// The most recent tip
    pub tip: git2::Oid,
}

pub mod error {
    use git_ref_format as ref_format;
    use std::collections::BTreeSet;

    use crate::git::storage::read;

    #[derive(thiserror::Error, Debug)]
    pub enum FindDefaultBranch {
        #[error("the project payload does not define a default branch")]
        NoDefaultBranch,
        #[error("no peers had published anything for the default branch")]
        NoTips,
        #[error(transparent)]
        RefFormat(#[from] ref_format::Error),
        #[error(transparent)]
        Read(#[from] read::Error),
    }

    #[derive(thiserror::Error, Debug)]
    pub enum SetDefaultBranch {
        #[error(transparent)]
        Find(#[from] FindDefaultBranch),
        #[error(transparent)]
        Git(#[from] git2::Error),
        #[error("the delegates have forked")]
        Forked(BTreeSet<super::Fork>),
    }
}

/// Find the head of the default branch of `project`
///
/// In general there can be a different view of the default branch of a project
/// for each peer ID of each delegate and there is no reason that these would
/// all be compatible. It's quite possible that two peers publish entirely
/// unrelated ancestry trees for a given branch. In this case this function will
/// return [`DefaultBranchHead::Forked`].
///
/// However, often it's the case that delegates do agree on an ancestry tree for
/// a particular branch and the difference between peers is just that some are
/// ahead of others. In this case this function will return
/// [`DefaultBranchHead::Head`].
///
/// # Errors
///
/// * If the project contains no default branch definition
/// * No peers had published anything for the default branch
pub fn default_branch_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
    if let Some(default_branch) = &project.payload().subject.default_branch {
        let local = storage.peer_id();
        let branch_refstring = RefString::try_from(default_branch.to_string())?;
        let mut multiverse = Multiverse::new(branch_refstring.clone());
        let peers =
            project
                .delegations()
                .into_iter()
                .flat_map(|d| -> Box<dyn Iterator<Item = PeerId>> {
                    use either::Either::*;
                    match d {
                        Left(key) => Box::new(std::iter::once(PeerId::from(*key))),
                        Right(person) => Box::new(
                            person
                                .delegations()
                                .into_iter()
                                .map(|key| PeerId::from(*key)),
                        ),
                    }
                });
        for peer_id in peers {
            let tip = peer_commit(storage, project.urn(), peer_id, local, &branch_refstring)?;
            if let Some(tip) = tip {
                multiverse.add_peer(storage, peer_id, tip)?;
            }
        }
        multiverse.finish()
    } else {
        Err(error::FindDefaultBranch::NoDefaultBranch)
    }
}

/// Determine the default branch for a project and set the local HEAD to this
/// branch
///
/// In more detail, this function determines the local head using
/// [`default_branch_head`] and then sets the following references to the
/// `DefaultBranchHead::target` returned:
///
/// * `refs/namespaces/<URN>/refs/HEAD`
/// * `refs/namespaces/<URN>/refs/<default branch name>`
///
/// # Why do this?
///
/// When cloning from a namespace representing a project to a working copy we
/// would like, if possible, to omit the specification of which particular peer
/// we want to clone. Specifically we would like to clone
/// `refs/namespaces/<URN>/`. This does work, but the working copy we end up
/// with does not have any contents because git uses `refs/HEAD` of the source
/// repository to figure out what branch to set the new working copy to.
/// Therefore, by setting `refs/HEAD` and `refs/<default branch name>` of the
/// namespace `git clone` (and any other clone based workflows) does something
/// sensible and we end up with a working copy which is looking at the default
/// branch of the project.
///
/// # Errors
///
/// * If no default branch could be determined
pub fn set_default_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<git2::Oid, error::SetDefaultBranch> {
    let urn = project.urn();
    let default_head = default_branch_head(storage, project)?;
    match default_head {
        DefaultBranchHead::Head { target, branch } => {
            // Note that we can't use `Namespaced` because `refs/HEAD` is not a `Qualified`
            let head =
                RefString::try_from(format!("refs/namespaces/{}/refs/HEAD", urn.encode_id()))
                    .expect("urn is valid namespace");
            let branch_head = Namespaced::from(lit::refs_namespaces(
                &urn,
                Qualified::from(lit::refs_heads(branch)),
            ));

            let repo = storage.as_raw();
            repo.reference(&head, target, true, "set head")?;
            repo.reference(&branch_head.into_qualified(), target, true, "set head")?;
            Ok(target)
        },
        DefaultBranchHead::Forked(forks) => Err(error::SetDefaultBranch::Forked(forks)),
    }
}

fn peer_commit(
    storage: &storage::Storage,
    urn: Urn,
    peer_id: PeerId,
    local: &PeerId,
    branch: &RefStr,
) -> Result<Option<git2::Oid>, error::FindDefaultBranch> {
    let remote_name = RefString::try_from(peer_id.default_encoding())?;
    let reference = if local == &peer_id {
        RefString::from(Qualified::from(lit::refs_heads(branch)))
    } else {
        RefString::from(Qualified::from(lit::refs_remotes(remote_name)))
            .join(name::HEADS)
            .join(branch)
    };
    let urn = urn.with_path(Some(RefLike::from(reference)));
    let tip = storage.tip(&urn, git2::ObjectType::Commit)?;
    Ok(tip.map(|c| c.id()))
}

#[derive(Debug)]
struct Multiverse {
    branch: RefString,
    histories: Vec<History>,
}

impl Multiverse {
    fn new(branch: RefString) -> Multiverse {
        Multiverse {
            branch,
            histories: Vec::new(),
        }
    }

    fn add_peer(
        &mut self,
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<(), error::FindDefaultBranch> {
        // If this peers tip is in the ancestors of any existing histories then we just
        // add the peer to those histories
        let mut found_descendant = false;
        for history in &mut self.histories {
            if history.ancestors.contains(&tip) {
                found_descendant = true;
                history.ancestor_peers.insert(peer);
            } else if history.tip == tip {
                found_descendant = true;
                history.tip_peers.insert(peer);
            }
        }
        if found_descendant {
            return Ok(());
        }

        // Otherwise we load a new history
        let mut history = History::load(storage, peer, tip)?;

        // Then we go through existing histories and check if any of them are ancestors
        // of the new history. If they are then we incorporate them as ancestors
        // of the new history and remove them from the multiverse
        let mut i = 0;
        while i < self.histories.len() {
            let other_history = &self.histories[i];
            if history.ancestors.contains(&other_history.tip) {
                let other_history = self.histories.remove(i);
                history.ancestor_peers.extend(other_history.ancestor_peers);
                history.ancestor_peers.extend(other_history.tip_peers);
            } else {
                i += 1;
            }
        }
        self.histories.push(history);

        Ok(())
    }

    fn finish(self) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
        if self.histories.is_empty() {
            Err(error::FindDefaultBranch::NoTips)
        } else if self.histories.len() == 1 {
            Ok(DefaultBranchHead::Head {
                target: self.histories[0].tip,
                branch: self.branch,
            })
        } else {
            Ok(DefaultBranchHead::Forked(
                self.histories
                    .into_iter()
                    .map(|h| Fork {
                        ancestor_peers: h.ancestor_peers,
                        tip_peers: h.tip_peers,
                        tip: h.tip,
                    })
                    .collect(),
            ))
        }
    }
}

#[derive(Debug)]
struct History {
    tip: git2::Oid,
    tip_peers: BTreeSet<PeerId>,
    ancestor_peers: BTreeSet<PeerId>,
    ancestors: BTreeSet<git2::Oid>,
}

impl History {
    fn load(
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<History, storage::Error> {
        let repo = storage.as_raw();
        let mut walk = repo.revwalk()?;
        walk.set_sorting(git2::Sort::TOPOLOGICAL)?;
        walk.push(tip)?;
        let mut ancestors = walk.collect::<Result<BTreeSet<git2::Oid>, _>>()?;
        ancestors.remove(&tip);
        let mut peers = BTreeSet::new();
        peers.insert(peer);
        let mut tip_peers = BTreeSet::new();
        tip_peers.insert(peer);
        Ok(History {
            tip,
            tip_peers,
            ancestors,
            ancestor_peers: BTreeSet::new(),
        })
    }
}
diff --git a/librad/t/src/integration/scenario.rs b/librad/t/src/integration/scenario.rs
index 9bfdd2ad..c47720a0 100644
--- a/librad/t/src/integration/scenario.rs
+++ b/librad/t/src/integration/scenario.rs
@@ -5,6 +5,7 @@

mod collaboration;
mod collaborative_objects;
mod default_branch_head;
mod menage;
mod passive_replication;
#[cfg(feature = "replication-v3")]
diff --git a/librad/t/src/integration/scenario/default_branch_head.rs b/librad/t/src/integration/scenario/default_branch_head.rs
new file mode 100644
index 00000000..2e7048c2
--- /dev/null
+++ b/librad/t/src/integration/scenario/default_branch_head.rs
@@ -0,0 +1,387 @@
// Copyright © 2019-2020 The Radicle Foundation <hello@radicle.foundation>
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

use std::{convert::TryFrom, ops::Index as _};

use tempfile::tempdir;

use git_ref_format::{lit, name, Namespaced, Qualified, RefString};
use it_helpers::{
    fixed::{TestPerson, TestProject},
    testnet::{self, RunningTestPeer},
    working_copy::{WorkingCopy, WorkingRemote as Remote},
};
use librad::git::{
    identities::{self, local, project::heads},
    storage::ReadOnlyStorage,
    tracking,
    types::{Namespace, Reference},
    Urn,
};
use link_identities::payload;
use test_helpers::logging;

fn config() -> testnet::Config {
    testnet::Config {
        num_peers: nonzero!(2usize),
        min_connected: 2,
        bootstrap: testnet::Bootstrap::from_env(),
    }
}

/// This test checks that the logic of `librad::git::identities::project::heads`
/// is correct. To do this we need to set up various scenarios where the
/// delegates of a project agree or disagree on the default branch of a project.
#[test]
fn default_branch_head() {
    logging::init();

    let net = testnet::run(config()).unwrap();
    net.enter(async {
        // Setup  a testnet with two peers and create a `Person` on each peer
        let peer1 = net.peers().index(0);
        let peer2 = net.peers().index(1);

        let id1 = peer1
            .using_storage::<_, anyhow::Result<TestPerson>>(|s| {
                let person = TestPerson::create(s)?;
                let local = local::load(s, person.owner.urn()).unwrap();
                s.config()?.set_user(local)?;
                Ok(person)
            })
            .await
            .unwrap()
            .unwrap();

        let id2 = peer2
            .using_storage::<_, anyhow::Result<TestPerson>>(|s| {
                let person = TestPerson::create(s)?;
                let local = local::load(s, person.owner.urn()).unwrap();
                s.config()?.set_user(local)?;
                Ok(person)
            })
            .await
            .unwrap()
            .unwrap();

        id2.pull(peer2, peer1).await.unwrap();
        id1.pull(peer1, peer2).await.unwrap();

        // Create a project on peer1 with both `Person`s as delegates
        let proj = peer1
            .using_storage({
                let owner = id1.owner.clone();
                move |s| {
                    TestProject::from_project_payload(
                        s,
                        owner,
                        payload::Project {
                            name: "venus".into(),
                            description: None,
                            default_branch: Some(name::MASTER.to_string().into()),
                        },
                    )
                }
            })
            .await
            .unwrap()
            .unwrap();

        // Track peer2 on peer1
        peer1
            .using_storage::<_, anyhow::Result<()>>({
                let urn = proj.project.urn();
                let peer2_id = peer2.peer_id();
                move |s| {
                    tracking::track(
                        s,
                        &urn,
                        Some(peer2_id),
                        tracking::Config::default(),
                        tracking::policy::Track::Any,
                    )??;
                    Ok(())
                }
            })
            .await
            .unwrap()
            .unwrap();

        proj.pull(peer1, peer2).await.unwrap();

        // Add peer2
        peer1
            .using_storage({
                let urn = proj.project.urn();
                let owner1 = id1.owner.clone();
                let owner2 = id2.owner.clone();
                move |storage| -> Result<(), anyhow::Error> {
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        librad::identities::delegation::Indirect::try_from_iter(
                            vec![either::Either::Right(owner1), either::Either::Right(owner2)]
                                .into_iter(),
                        )
                        .unwrap(),
                    )?;
                    identities::project::verify(storage, &urn)?;
                    Ok(())
                }
            })
            .await
            .unwrap()
            .unwrap();

        proj.pull(peer1, peer2).await.unwrap();

        // Sign the project document using peer2
        peer2
            .using_storage({
                let urn = proj.project.urn();
                let peer_id = peer1.peer_id();
                let rad =
                    Urn::try_from(Reference::rad_id(Namespace::from(&urn)).with_remote(peer_id))
                        .unwrap();
                move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                    let project = identities::project::get(&storage, &rad)?.unwrap();
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        project.delegations().clone(),
                    )?;
                    identities::project::merge(storage, &urn, peer_id)?;
                    Ok(identities::project::verify(storage, &urn)?)
                }
            })
            .await
            .unwrap()
            .unwrap();

        proj.pull(peer2, peer1).await.unwrap();

        // Merge the signed update into peer1
        peer1
            .using_storage({
                let urn = proj.project.urn();
                let peer_id = peer2.peer_id();
                move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                    identities::project::merge(storage, &urn, peer_id)?;
                    Ok(identities::project::verify(storage, &urn)?)
                }
            })
            .await
            .unwrap()
            .unwrap();

        id2.pull(peer2, peer1).await.unwrap();

        // Okay, now we have a running testnet with two Peers, each of which has a
        // `Person` who is a delegate on the `TestProject`

        // Create a commit in peer 1 and pull to peer2, then pull those changes into
        // peer2, create a new commit on top of the original commit and pull
        // that back to peer1. Then in peer1 pull the commit, fast forward, and
        // push.
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            working_copy1
                .commit("peer 1 initial", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();
            proj.pull(peer1, peer2).await.unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            let tip = working_copy2
                .commit("peer 2 initial", mastor.clone())
                .unwrap();
            working_copy2.push().unwrap();
            proj.pull(peer2, peer1).await.unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .fast_forward_to(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch = branch_head(peer1, &proj).await.unwrap();
        // The two peers hsould have the same view of the default branch
        assert_eq!(
            default_branch,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now update peer1 and push to peer 1s monorepo, we should get the tip of peer1
        // as the head (because peer2 can be fast forwarded)
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let tip = working_copy1.commit("peer 1 fork", mastor.clone()).unwrap();
            working_copy1.push().unwrap();

            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now create an alternate commit on peer2 and sync with peer1, on peer1 we
        // should get a fork
        let tmp = tempdir().unwrap();
        let forked_tip = {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let forked_tip = working_copy2.commit("peer 2 fork", mastor.clone()).unwrap();
            working_copy2.push().unwrap();

            forked_tip
        };

        proj.pull(peer2, peer1).await.unwrap();

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Forked(
                vec![
                    identities::project::heads::Fork {
                        ancestor_peers: std::collections::BTreeSet::new(),
                        tip_peers: std::iter::once(peer1.peer_id()).collect(),
                        tip,
                    },
                    identities::project::heads::Fork {
                        ancestor_peers: std::collections::BTreeSet::new(),
                        tip_peers: std::iter::once(peer2.peer_id()).collect(),
                        tip: forked_tip,
                    }
                ]
                .into_iter()
                .collect()
            )
        );

        // now update peer1 to match peer2
        let tmp = tempdir().unwrap();
        let fixed_tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            let tip = working_copy1
                .merge_remote(peer2.peer_id(), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: fixed_tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now set the head in the monorepo and check that the HEAD reference exists
        let updated_tip = peer1
            .using_storage::<_, anyhow::Result<_>>({
                let urn = proj.project.urn();
                move |s| {
                    let vp = identities::project::verify(s, &urn)?.ok_or_else(|| {
                        anyhow::anyhow!("failed to get project for default branch")
                    })?;
                    identities::project::heads::set_default_head(s, vp).map_err(anyhow::Error::from)
                }
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(updated_tip, fixed_tip);

        let head_ref = RefString::try_from(format!(
            "refs/namespaces/{}/refs/HEAD",
            proj.project.urn().encode_id()
        ))
        .unwrap();
        let master_ref = Namespaced::from(lit::refs_namespaces(
            &proj.project.urn(),
            Qualified::from(lit::refs_heads(name::MASTER)),
        ));
        let (master_oid, head_oid) = peer1
            .using_storage::<_, anyhow::Result<_>>(move |s| {
                let master_oid = s
                    .reference(&master_ref.into_qualified().into_refstring())?
                    .ok_or_else(|| anyhow::anyhow!("master ref not found"))?
                    .peel_to_commit()?
                    .id();
                let head_oid = s
                    .reference(&head_ref)?
                    .ok_or_else(|| anyhow::anyhow!("head ref not found"))?
                    .peel_to_commit()?
                    .id();
                Ok((master_oid, head_oid))
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(master_oid, updated_tip);
        assert_eq!(head_oid, updated_tip);
    });
}

async fn branch_head(
    peer: &RunningTestPeer,
    proj: &TestProject,
) -> anyhow::Result<heads::DefaultBranchHead> {
    peer.using_storage::<_, anyhow::Result<_>>({
        let urn = proj.project.urn();
        move |s| {
            let vp = identities::project::verify(s, &urn)?
                .ok_or_else(|| anyhow::anyhow!("failed to get project for default branch"))?;
            heads::default_branch_head(s, vp).map_err(anyhow::Error::from)
        }
    })
    .await?
}
diff --git a/test/it-helpers/Cargo.toml b/test/it-helpers/Cargo.toml
index 32c789cd..119d5aa1 100644
--- a/test/it-helpers/Cargo.toml
+++ b/test/it-helpers/Cargo.toml
@@ -40,5 +40,8 @@ path = "../../link-async"
[dependencies.lnk-clib]
path = "../../cli/lnk-clib"

[dependencies.radicle-git-ext]
path = "../../git-ext"

[dependencies.test-helpers]
path = "../test-helpers"
diff --git a/test/it-helpers/src/lib.rs b/test/it-helpers/src/lib.rs
index 981b922d..5012de39 100644
--- a/test/it-helpers/src/lib.rs
+++ b/test/it-helpers/src/lib.rs
@@ -7,3 +7,4 @@ pub mod layout;
pub mod ssh;
pub mod testnet;
pub mod tmp;
pub mod working_copy;
diff --git a/test/it-helpers/src/working_copy.rs b/test/it-helpers/src/working_copy.rs
new file mode 100644
index 00000000..5fbef0dd
--- /dev/null
+++ b/test/it-helpers/src/working_copy.rs
@@ -0,0 +1,291 @@
use std::path::Path;

use git_ref_format::{lit, name, refspec, Qualified, RefStr, RefString};

use librad::{
    git::{
        local::url::LocalUrl,
        types::{
            remote::{LocalFetchspec, LocalPushspec},
            Fetchspec,
            Force,
            Refspec,
            Remote,
        },
    },
    git_ext as ext,
    net::{peer::Peer, protocol::RequestPullGuard},
    refspec_pattern,
    PeerId,
    Signer,
};

use crate::fixed::TestProject;

/// A remote in the working copy
pub enum WorkingRemote {
    /// A remote representing a remote peer, named `PeerId::encode_id`
    Peer(PeerId),
    /// A remote representing the local peer, named "rad"
    Rad,
}

impl From<PeerId> for WorkingRemote {
    fn from(p: PeerId) -> Self {
        WorkingRemote::Peer(p)
    }
}

impl WorkingRemote {
    fn fetchspec(&self) -> Fetchspec {
        match self {
            Self::Peer(peer_id) => {
                let name = RefString::try_from(format!("{}", peer_id)).expect("peer is refstring");
                let dst = RefString::from(Qualified::from(lit::refs_remotes(name.clone())))
                    .with_pattern(refspec::STAR);
                let src = RefString::from(Qualified::from(lit::refs_remotes(name)))
                    .and(name::HEADS)
                    .with_pattern(refspec::STAR);
                let refspec = Refspec {
                    src,
                    dst,
                    force: Force::True,
                };
                refspec.into_fetchspec()
            },
            Self::Rad => {
                let name = RefString::try_from("rad").unwrap();
                let src =
                    RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR);
                Refspec {
                    src,
                    dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                        .with_pattern(refspec::STAR),
                    force: Force::True,
                }
                .into_fetchspec()
            },
        }
    }

    fn remote_ref(&self, branch: &RefStr) -> RefString {
        let name = match self {
            Self::Rad => name::RAD.to_owned(),
            Self::Peer(peer_id) => {
                RefString::try_from(peer_id.to_string()).expect("peer id is refstring")
            },
        };
        RefString::from(Qualified::from(lit::refs_remotes(name))).join(branch)
    }
}

/// A `WorkingCopy` for test driving interactions with the monorepo where one
/// needs to update the tree of a project.
///
/// Remotes are named after the peer ID, except in the case of the remote
/// representing the local Peer ID - which is called "rad".
pub struct WorkingCopy<'a, S, G> {
    repo: git2::Repository,
    peer: &'a Peer<S, G>,
    project: &'a TestProject,
}

impl<'a, S, G> WorkingCopy<'a, S, G>
where
    S: Signer + Clone,
    G: RequestPullGuard,
{
    /// Create a new working copy. This initializes a git repository and then
    /// fetches the state of the local peer into `refs/remotes/rad/*`.
    pub fn new<P: AsRef<Path>>(
        project: &'a TestProject,
        repo_path: P,
        peer: &'a Peer<S, G>,
    ) -> Result<WorkingCopy<'a, S, G>, anyhow::Error> {
        let repo = git2::Repository::init(repo_path.as_ref())?;

        let mut copy = WorkingCopy {
            peer,
            project,
            repo,
        };
        copy.fetch(WorkingRemote::Rad)?;
        Ok(copy)
    }

    /// Fetch changes from the monorepo into the working copy. The fetchspec
    /// used depends on the peer ID.
    ///
    /// * If `from` is `WorkingRemote::Peer` then `refs/remotes/<peer
    ///   ID>/refs/*:refs/remotes/<peer ID>/heads/*`
    /// * If `from` is `WorkingRemote::Rad` then
    ///   `refs/heads/*:refs/remotes/rad/*`
    ///
    /// I.e. changes from remote peers end up in a remote called
    /// `PeerId::encode_id` whilst changes from the local peer end up in a
    /// remote called "rad".
    pub fn fetch(&mut self, from: WorkingRemote) -> Result<(), anyhow::Error> {
        let fetchspec = from.fetchspec();
        let url = LocalUrl::from(self.project.project.urn());
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.fetch(self.peer.clone(), &self.repo, LocalFetchspec::Configured)?;
        Ok(())
    }

    /// Push changes from `refs/heads/*` to the local peer
    pub fn push(&mut self) -> Result<(), anyhow::Error> {
        let url = LocalUrl::from(self.project.project.urn());
        let name = RefString::try_from("rad").unwrap();
        let fetchspec = Refspec {
            src: RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR),
            dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                .with_pattern(refspec::STAR),
            force: Force::True,
        }
        .into_fetchspec();
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.push(
            self.peer.clone(),
            &self.repo,
            LocalPushspec::Matching {
                pattern: refspec_pattern!("refs/heads/*"),
                force: Force::True,
            },
        )?;
        Ok(())
    }

    /// Create a new commit on top of whichever commit is the head of
    /// `on_branch`. If the branch does not exist this will create it.
    pub fn commit(
        &mut self,
        message: &str,
        on_branch: Qualified,
    ) -> Result<git2::Oid, anyhow::Error> {
        let branch_name = on_branch.non_empty_components().2;
        let parent = match self.repo.find_branch(&branch_name, git2::BranchType::Local) {
            Ok(b) => b.get().target().and_then(|o| self.repo.find_commit(o).ok()),
            Err(e) if ext::error::is_not_found_err(&e) => None,
            Err(e) => return Err(anyhow::Error::from(e)),
        };
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = match &parent {
            Some(p) => vec![p],
            None => Vec::new(),
        };
        self.repo
            .commit(
                Some(&on_branch),
                &author,
                &author,
                message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }

    /// Create a branch at `refs/heads/<branch>` which tracks the given remote.
    /// The remote branch name depends on `from`.
    ///
    /// * If `from` is `WorkingCopy::Rad` then `refs/remotes/rad/<branch>`
    /// * If `from` is `WorkingCopy::Peer(peer_id)` then `refs/remotes/<peer
    ///   id>/<branch>`
    pub fn create_remote_tracking_branch(
        &self,
        from: WorkingRemote,
        branch: &RefStr,
    ) -> Result<(), anyhow::Error> {
        let target = self
            .repo
            .find_reference(from.remote_ref(branch).as_str())?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref is not a direct reference"))?;
        let commit = self.repo.find_commit(target)?;
        self.repo.branch(branch.as_str(), &commit, false)?;
        Ok(())
    }

    /// Fast forward the local branch `refs/heads/<branch>` to whatever is
    /// pointed to by `refs/remotes/<remote>/<branch>`
    ///
    /// * If `from` is `WorkingRemote::Peer(peer_id)` then `remote` is
    ///   `peer_id.encode_id()`
    /// * If `from` is `WorkingRemote::Rad` then `remote` is `"rad"`
    ///
    /// # Errors
    ///
    /// * If the local branch does not exist
    /// * If the remote branch does not exist
    /// * If either of the branches does not point at a commit
    /// * If the remote branch is not a descendant of the local branch
    pub fn fast_forward_to(&self, from: WorkingRemote, branch: &RefStr) -> anyhow::Result<()> {
        let remote_ref = from.remote_ref(branch);
        let remote_target = self
            .repo
            .find_reference(&remote_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref had no target"))?;
        let local_ref = RefString::from(Qualified::from(lit::refs_heads(branch)));
        let local_target = self
            .repo
            .find_reference(&local_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("local ref had no target"))?;
        if !self.repo.graph_descendant_of(remote_target, local_target)? {
            anyhow::bail!("remote ref was not a descendant of local ref");
        } else {
            self.repo
                .reference(&local_ref, remote_target, true, "fast forward")?;
        }
        Ok(())
    }

    /// Create a new commit which merges `refs/heads/<branch>` and
    /// `refs/remotes/<remote>/<branch>`
    ///
    /// this will create a new commit with two parents, one for the remote
    /// branch and one for the local branch
    ///
    /// # Errors
    ///
    /// * If the remote branch does not exist
    /// * If the local branch does not exist
    /// * If either of the references does not point to a commit
    pub fn merge_remote(&self, remote: PeerId, branch: &RefStr) -> anyhow::Result<git2::Oid> {
        let peer_branch = WorkingRemote::Peer(remote).remote_ref(branch);
        let peer_commit = self
            .repo
            .find_reference(&peer_branch.to_string())?
            .peel_to_commit()?;
        let local_branch = Qualified::from(lit::refs_heads(branch));
        let local_commit = self
            .repo
            .find_reference(&local_branch.to_string())?
            .peel_to_commit()?;

        let message = format!("merge {} into {}", peer_branch, local_branch);
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = vec![&peer_commit, &local_commit];
        self.repo
            .commit(
                Some(&local_branch),
                &author,
                &author,
                &message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }
}
-- 
2.36.1
Does it indeed work to set HEAD to an oid? I was under the impression that
`git-clone` would expect it to be a symref; if it is not, it doesn't know which
branch to check out. I believe it will leave the cloned repo in a detached head
state, which is gradually less scary than not checking out anything at all.

[PATCH v2 1/4] Add default_branch_head and set_default_branch Export this patch

When checking out projects from the monorepo it is useful to set the
`refs/namespaces/<urn>/HEAD` reference to the default branch of the
project so that the resulting working copy is in a useful state (namely
pointing at the latest commit for the default branch).

In general this is not possible because delegates may have diverging
views of the project, but often they do not disagree. Add
`librad::git::identities::project::heads::default_branch_head` to
determine if there is an agreed on default branch commit and
`librad::git::identities::project::heads::set_default_branch` to set the
local `HEAD` ref where possible.

Signed-off-by: Alex Good <alex@memoryandthought.me>
---
 librad/src/git/identities/project.rs          |   2 +
 librad/src/git/identities/project/heads.rs    | 312 ++++++++++++++++++
 librad/t/src/integration/scenario.rs          |   1 +
 .../scenario/default_branch_head.rs           | 286 ++++++++++++++++
 test/it-helpers/Cargo.toml                    |   4 +
 test/it-helpers/src/fixed.rs                  |   2 +-
 test/it-helpers/src/fixed/project.rs          | 150 ++++++++-
 test/it-helpers/src/lib.rs                    |   1 +
 test/it-helpers/src/testnet.rs                |  16 +-
 test/it-helpers/src/working_copy.rs           | 291 ++++++++++++++++
 10 files changed, 1062 insertions(+), 3 deletions(-)
 create mode 100644 librad/src/git/identities/project/heads.rs
 create mode 100644 librad/t/src/integration/scenario/default_branch_head.rs
 create mode 100644 test/it-helpers/src/working_copy.rs

diff --git a/librad/src/git/identities/project.rs b/librad/src/git/identities/project.rs
index 753358bf..ed9f003d 100644
--- a/librad/src/git/identities/project.rs
+++ b/librad/src/git/identities/project.rs
@@ -8,6 +8,8 @@ use std::{convert::TryFrom, fmt::Debug};
use either::Either;
use git_ext::{is_not_found_err, OneLevel};

pub mod heads;

use super::{
    super::{
        refs::Refs as Sigrefs,
diff --git a/librad/src/git/identities/project/heads.rs b/librad/src/git/identities/project/heads.rs
new file mode 100644
index 00000000..7f1e912e
--- /dev/null
+++ b/librad/src/git/identities/project/heads.rs
@@ -0,0 +1,312 @@
use std::{collections::BTreeSet, convert::TryFrom, fmt::Debug};

use crate::{
    git::{
        storage::{self, ReadOnlyStorage},
        Urn,
    },
    identities::git::VerifiedProject,
    PeerId,
};
use git_ext::RefLike;
use git_ref_format::{lit, name, Namespaced, Qualified, RefStr, RefString};

#[derive(Clone, Debug, PartialEq)]
pub enum DefaultBranchHead {
    /// Not all delegates agreed on an ancestry tree. Each set of diverging
    /// delegates is included as a `Fork`
    Forked(BTreeSet<Fork>),
    /// All the delegates agreed on an ancestry tree
    Head {
        /// The most recent commit for the tree
        target: git2::Oid,
        /// The branch name which is the default branch
        branch: RefString,
    },
}

#[derive(Clone, Debug, std::hash::Hash, PartialEq, Eq, PartialOrd, Ord)]
pub struct Fork {
    /// Peers which are in the ancestry set of this fork but not the tips. This
    /// means that these peers can appear in multiple forks
    pub ancestor_peers: BTreeSet<PeerId>,
    /// The peers pointing at the tip of this fork
    pub tip_peers: BTreeSet<PeerId>,
    /// The most recent tip
    pub tip: git2::Oid,
}

pub mod error {
    use git_ref_format as ref_format;
    use std::collections::BTreeSet;

    use crate::git::storage::read;

    #[derive(thiserror::Error, Debug)]
    pub enum FindDefaultBranch {
        #[error("the project payload does not define a default branch")]
        NoDefaultBranch,
        #[error("no peers had published anything for the default branch")]
        NoTips,
        #[error(transparent)]
        RefFormat(#[from] ref_format::Error),
        #[error(transparent)]
        Read(#[from] read::Error),
    }

    #[derive(thiserror::Error, Debug)]
    pub enum SetDefaultBranch {
        #[error(transparent)]
        Find(#[from] FindDefaultBranch),
        #[error(transparent)]
        Git(#[from] git2::Error),
        #[error("the delegates have forked")]
        Forked(BTreeSet<super::Fork>),
    }
}

/// Find the head of the default branch of `project`
///
/// In general there can be a different view of the default branch of a project
/// for each peer ID of each delegate and there is no reason that these would
/// all be compatible. It's quite possible that two peers publish entirely
/// unrelated ancestry trees for a given branch. In this case this function will
/// return [`DefaultBranchHead::Forked`].
///
/// However, often it's the case that delegates do agree on an ancestry tree for
/// a particular branch and the difference between peers is just that some are
/// ahead of others. In this case this function will return
/// [`DefaultBranchHead::Head`].
///
/// # Errors
///
/// * If the project contains no default branch definition
/// * No peers had published anything for the default branch
pub fn default_branch_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
    if let Some(default_branch) = &project.payload().subject.default_branch {
        let local = storage.peer_id();
        let branch_refstring = RefString::try_from(default_branch.to_string())?;
        let mut multiverse = Multiverse::new(branch_refstring.clone());
        let peers =
            project
                .delegations()
                .into_iter()
                .flat_map(|d| -> Box<dyn Iterator<Item = PeerId>> {
                    use either::Either::*;
                    match d {
                        Left(key) => Box::new(std::iter::once(PeerId::from(*key))),
                        Right(person) => Box::new(
                            person
                                .delegations()
                                .into_iter()
                                .map(|key| PeerId::from(*key)),
                        ),
                    }
                });
        for peer_id in peers {
            let tip = peer_commit(storage, project.urn(), peer_id, local, &branch_refstring)?;
            if let Some(tip) = tip {
                multiverse.add_peer(storage, peer_id, tip)?;
            } else {
                tracing::warn!(%peer_id, %default_branch, "no default branch commit found for peer");
            }
        }
        multiverse.finish()
    } else {
        Err(error::FindDefaultBranch::NoDefaultBranch)
    }
}

/// Determine the default branch for a project and set the local HEAD to this
/// branch
///
/// In more detail, this function determines the local head using
/// [`default_branch_head`] and then sets the following references to the
/// `DefaultBranchHead::target` returned:
///
/// * `refs/namespaces/<URN>/refs/HEAD`
/// * `refs/namespaces/<URN>/refs/<default branch name>`
///
/// # Why do this?
///
/// When cloning from a namespace representing a project to a working copy we
/// would like, if possible, to omit the specification of which particular peer
/// we want to clone. Specifically we would like to clone
/// `refs/namespaces/<URN>/`. This does work, but the working copy we end up
/// with does not have any contents because git uses `refs/HEAD` of the source
/// repository to figure out what branch to set the new working copy to.
/// Therefore, by setting `refs/HEAD` and `refs/<default branch name>` of the
/// namespace `git clone` (and any other clone based workflows) does something
/// sensible and we end up with a working copy which is looking at the default
/// branch of the project.
///
/// # Errors
///
/// * If no default branch could be determined
pub fn set_default_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<git2::Oid, error::SetDefaultBranch> {
    let urn = project.urn();
    let default_head = default_branch_head(storage, project)?;
    match default_head {
        DefaultBranchHead::Head { target, branch } => {
            // Note that we can't use `Namespaced` because `refs/HEAD` is not a `Qualified`
            let head =
                RefString::try_from(format!("refs/namespaces/{}/refs/HEAD", urn.encode_id()))
                    .expect("urn is valid namespace");
            let branch_head = Namespaced::from(lit::refs_namespaces(
                &urn,
                Qualified::from(lit::refs_heads(branch)),
            ));

            let repo = storage.as_raw();
            repo.reference(
                &branch_head.clone().into_qualified(),
                target,
                true,
                "set default branch head",
            )?;
            repo.reference_symbolic(head.as_str(), branch_head.as_str(), true, "set head")?;
            Ok(target)
        },
        DefaultBranchHead::Forked(forks) => Err(error::SetDefaultBranch::Forked(forks)),
    }
}

fn peer_commit(
    storage: &storage::Storage,
    urn: Urn,
    peer_id: PeerId,
    local: &PeerId,
    branch: &RefStr,
) -> Result<Option<git2::Oid>, error::FindDefaultBranch> {
    let remote_name = RefString::try_from(peer_id.default_encoding())?;
    let reference = if local == &peer_id {
        RefString::from(Qualified::from(lit::refs_heads(branch)))
    } else {
        RefString::from(Qualified::from(lit::refs_remotes(remote_name)))
            .join(name::HEADS)
            .join(branch)
    };
    let urn = urn.with_path(Some(RefLike::from(reference)));
    let tip = storage.tip(&urn, git2::ObjectType::Commit)?;
    Ok(tip.map(|c| c.id()))
}

#[derive(Debug)]
struct Multiverse {
    branch: RefString,
    histories: Vec<History>,
}

impl Multiverse {
    fn new(branch: RefString) -> Multiverse {
        Multiverse {
            branch,
            histories: Vec::new(),
        }
    }

    fn add_peer(
        &mut self,
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<(), error::FindDefaultBranch> {
        // If this peers tip is in the ancestors of any existing histories then we just
        // add the peer to those histories
        let mut found_descendant = false;
        for history in &mut self.histories {
            if history.ancestors.contains(&tip) {
                found_descendant = true;
                history.ancestor_peers.insert(peer);
            } else if history.tip == tip {
                found_descendant = true;
                history.tip_peers.insert(peer);
            }
        }
        if found_descendant {
            return Ok(());
        }

        // Otherwise we load a new history
        let mut history = History::load(storage, peer, tip)?;

        // Then we go through existing histories and check if any of them are ancestors
        // of the new history. If they are then we incorporate them as ancestors
        // of the new history and remove them from the multiverse
        let mut i = 0;
        while i < self.histories.len() {
            let other_history = &self.histories[i];
            if history.ancestors.contains(&other_history.tip) {
                let other_history = self.histories.remove(i);
                history.ancestor_peers.extend(other_history.ancestor_peers);
                history.ancestor_peers.extend(other_history.tip_peers);
            } else {
                i += 1;
            }
        }
        self.histories.push(history);

        Ok(())
    }

    fn finish(self) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
        if self.histories.is_empty() {
            Err(error::FindDefaultBranch::NoTips)
        } else if self.histories.len() == 1 {
            Ok(DefaultBranchHead::Head {
                target: self.histories[0].tip,
                branch: self.branch,
            })
        } else {
            Ok(DefaultBranchHead::Forked(
                self.histories
                    .into_iter()
                    .map(|h| Fork {
                        ancestor_peers: h.ancestor_peers,
                        tip_peers: h.tip_peers,
                        tip: h.tip,
                    })
                    .collect(),
            ))
        }
    }
}

#[derive(Debug)]
struct History {
    tip: git2::Oid,
    tip_peers: BTreeSet<PeerId>,
    ancestor_peers: BTreeSet<PeerId>,
    ancestors: BTreeSet<git2::Oid>,
}

impl History {
    fn load(
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<Self, storage::Error> {
        let repo = storage.as_raw();
        let mut walk = repo.revwalk()?;
        walk.set_sorting(git2::Sort::TOPOLOGICAL)?;
        walk.push(tip)?;
        let mut ancestors = walk.collect::<Result<BTreeSet<git2::Oid>, _>>()?;
        ancestors.remove(&tip);
        let mut peers = BTreeSet::new();
        peers.insert(peer);
        let mut tip_peers = BTreeSet::new();
        tip_peers.insert(peer);
        Ok(Self {
            tip,
            tip_peers,
            ancestors,
            ancestor_peers: BTreeSet::new(),
        })
    }
}
diff --git a/librad/t/src/integration/scenario.rs b/librad/t/src/integration/scenario.rs
index 9bfdd2ad..c47720a0 100644
--- a/librad/t/src/integration/scenario.rs
+++ b/librad/t/src/integration/scenario.rs
@@ -5,6 +5,7 @@

mod collaboration;
mod collaborative_objects;
mod default_branch_head;
mod menage;
mod passive_replication;
#[cfg(feature = "replication-v3")]
diff --git a/librad/t/src/integration/scenario/default_branch_head.rs b/librad/t/src/integration/scenario/default_branch_head.rs
new file mode 100644
index 00000000..0dcb6578
--- /dev/null
+++ b/librad/t/src/integration/scenario/default_branch_head.rs
@@ -0,0 +1,286 @@
// Copyright © 2019-2020 The Radicle Foundation <hello@radicle.foundation>
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

use std::{convert::TryFrom, ops::Index as _};

use tempfile::tempdir;

use git_ref_format::{lit, name, Namespaced, Qualified, RefString};
use it_helpers::{
    fixed::{TestPerson, TestProject},
    testnet::{self, RunningTestPeer},
    working_copy::{WorkingCopy, WorkingRemote as Remote},
};
use librad::git::{
    identities::{self, local, project::heads},
    storage::ReadOnlyStorage,
};
use link_identities::payload;
use test_helpers::logging;

fn config() -> testnet::Config {
    testnet::Config {
        num_peers: nonzero!(2usize),
        min_connected: 2,
        bootstrap: testnet::Bootstrap::from_env(),
    }
}

/// This test checks that the logic of `librad::git::identities::project::heads`
/// is correct. To do this we need to set up various scenarios where the
/// delegates of a project agree or disagree on the default branch of a project.
#[test]
fn default_branch_head() {
    logging::init();

    let net = testnet::run(config()).unwrap();
    net.enter(async {
        // Setup  a testnet with two peers
        let peer1 = net.peers().index(0);
        let peer2 = net.peers().index(1);

        // Create an identity on peer2
        let peer2_id = peer2
            .using_storage::<_, anyhow::Result<TestPerson>>(|s| {
                let person = TestPerson::create(s)?;
                let local = local::load(s, person.owner.urn()).unwrap();
                s.config()?.set_user(local)?;
                Ok(person)
            })
            .await
            .unwrap()
            .unwrap();

        peer2_id.pull(peer2, peer1).await.unwrap();

        // Create a project on peer1
        let proj = peer1
            .using_storage(|s| {
                TestProject::create_with_payload(
                    s,
                    payload::Project {
                        name: "venus".into(),
                        description: None,
                        default_branch: Some(name::MASTER.to_string().into()),
                    },
                )
            })
            .await
            .unwrap()
            .unwrap();

        // Add peer2 as a maintainer
        proj.maintainers(peer1)
            .add(&peer2_id, peer2)
            .setup()
            .await
            .unwrap();

        //// Okay, now we have a running testnet with two Peers, each of which has a
        //// `Person` who is a delegate on the `TestProject`

        // Create a commit in peer 1 and pull to peer2, then pull those changes into
        // peer2, create a new commit on top of the original commit and pull
        // that back to peer1. Then in peer1 pull the commit, fast forward, and
        // push.
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            working_copy1
                .commit("peer 1 initial", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();
            proj.pull(peer1, peer2).await.unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            let tip = working_copy2
                .commit("peer 2 initial", mastor.clone())
                .unwrap();
            working_copy2.push().unwrap();
            proj.pull(peer2, peer1).await.unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .fast_forward_to(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch = branch_head(peer1, &proj).await.unwrap();
        // The two peers hsould have the same view of the default branch
        assert_eq!(
            default_branch,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now update peer1 and push to peer 1s monorepo, we should get the tip of peer1
        // as the head (because peer2 can be fast forwarded)
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let tip = working_copy1.commit("peer 1 fork", mastor.clone()).unwrap();
            working_copy1.push().unwrap();

            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now create an alternate commit on peer2 and sync with peer1, on peer1 we
        // should get a fork
        let tmp = tempdir().unwrap();
        let forked_tip = {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let forked_tip = working_copy2.commit("peer 2 fork", mastor.clone()).unwrap();
            working_copy2.push().unwrap();

            forked_tip
        };

        proj.pull(peer2, peer1).await.unwrap();

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Forked(
                vec![
                    identities::project::heads::Fork {
                        ancestor_peers: std::collections::BTreeSet::new(),
                        tip_peers: std::iter::once(peer1.peer_id()).collect(),
                        tip,
                    },
                    identities::project::heads::Fork {
                        ancestor_peers: std::collections::BTreeSet::new(),
                        tip_peers: std::iter::once(peer2.peer_id()).collect(),
                        tip: forked_tip,
                    }
                ]
                .into_iter()
                .collect()
            )
        );

        // now update peer1 to match peer2
        let tmp = tempdir().unwrap();
        let fixed_tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            let tip = working_copy1
                .merge_remote(peer2.peer_id(), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: fixed_tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now set the head in the monorepo and check that the HEAD reference exists
        let updated_tip = peer1
            .using_storage::<_, anyhow::Result<_>>({
                let urn = proj.project.urn();
                move |s| {
                    let vp = identities::project::verify(s, &urn)?.ok_or_else(|| {
                        anyhow::anyhow!("failed to get project for default branch")
                    })?;
                    identities::project::heads::set_default_head(s, vp).map_err(anyhow::Error::from)
                }
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(updated_tip, fixed_tip);

        let head_ref = RefString::try_from(format!(
            "refs/namespaces/{}/refs/HEAD",
            proj.project.urn().encode_id()
        ))
        .unwrap();
        let master_ref = Namespaced::from(lit::refs_namespaces(
            &proj.project.urn(),
            Qualified::from(lit::refs_heads(name::MASTER)),
        ));
        let (master_oid, head_target) = peer1
            .using_storage::<_, anyhow::Result<_>>({
                let master_ref = master_ref.clone();
                move |s| {
                    let master_oid = s
                        .reference(&master_ref.into_qualified().into_refstring())?
                        .ok_or_else(|| anyhow::anyhow!("master ref not found"))?
                        .peel_to_commit()?
                        .id();
                    let head_target = s
                        .reference(&head_ref)?
                        .ok_or_else(|| anyhow::anyhow!("head ref not found"))?
                        .symbolic_target()
                        .map(|s| s.to_string());
                    Ok((master_oid, head_target))
                }
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(master_oid, updated_tip);
        assert_eq!(head_target, Some(master_ref.to_string()));
    });
}

async fn branch_head(
    peer: &RunningTestPeer,
    proj: &TestProject,
) -> anyhow::Result<heads::DefaultBranchHead> {
    peer.using_storage::<_, anyhow::Result<_>>({
        let urn = proj.project.urn();
        move |s| {
            let vp = identities::project::verify(s, &urn)?
                .ok_or_else(|| anyhow::anyhow!("failed to get project for default branch"))?;
            heads::default_branch_head(s, vp).map_err(anyhow::Error::from)
        }
    })
    .await?
}
diff --git a/test/it-helpers/Cargo.toml b/test/it-helpers/Cargo.toml
index 32c789cd..04aa2166 100644
--- a/test/it-helpers/Cargo.toml
+++ b/test/it-helpers/Cargo.toml
@@ -18,6 +18,7 @@ once_cell = "1.10"
tempfile = "3.3"
tokio = "1.13"
tracing = "0.1"
either = "1.6"

[dependencies.git2]
version = "0.13.24"
@@ -40,5 +41,8 @@ path = "../../link-async"
[dependencies.lnk-clib]
path = "../../cli/lnk-clib"

[dependencies.radicle-git-ext]
path = "../../git-ext"

[dependencies.test-helpers]
path = "../test-helpers"
diff --git a/test/it-helpers/src/fixed.rs b/test/it-helpers/src/fixed.rs
index c36f5dd6..53006a31 100644
--- a/test/it-helpers/src/fixed.rs
+++ b/test/it-helpers/src/fixed.rs
@@ -2,7 +2,7 @@ mod person;
pub use person::TestPerson;

mod project;
pub use project::TestProject;
pub use project::{Maintainers, TestProject};

pub mod repository;
pub use repository::{commit, repository};
diff --git a/test/it-helpers/src/fixed/project.rs b/test/it-helpers/src/fixed/project.rs
index 4ce3a41e..37a48832 100644
--- a/test/it-helpers/src/fixed/project.rs
+++ b/test/it-helpers/src/fixed/project.rs
@@ -4,6 +4,8 @@ use librad::{
    git::{
        identities::{self, Person, Project},
        storage::Storage,
        types::{Namespace, Reference},
        Urn,
    },
    identities::{
        delegation::{self, Direct},
@@ -18,6 +20,8 @@ use librad::{
};
use tracing::{info, instrument};

use crate::testnet::RunningTestPeer;

use super::TestPerson;

pub struct TestProject {
@@ -27,6 +31,13 @@ pub struct TestProject {

impl TestProject {
    pub fn create(storage: &Storage) -> anyhow::Result<Self> {
        Self::create_with_payload(storage, Self::default_payload())
    }

    pub fn create_with_payload(
        storage: &Storage,
        payload: payload::Project,
    ) -> anyhow::Result<Self> {
        let peer_id = storage.peer_id();
        let alice = identities::person::create(
            storage,
@@ -40,7 +51,7 @@ impl TestProject {
        let proj = identities::project::create(
            storage,
            local_id,
            Self::default_payload(),
            payload,
            delegation::Indirect::from(alice.clone()),
        )?;

@@ -120,4 +131,141 @@ impl TestProject {
            .replicate((remote_peer, remote_addrs), urn, None)
            .await?)
    }

    /// Add maintainers to a TestProject
    ///
    /// The `home` argument must be a peer which is already a delegate of the
    /// project. The [`Maintainers`] struct which is returned can be used to
    /// add maintainers using [`Maintainers::add`] before calling
    /// [`Maintainers::setup`] to perform the cross signing which adds the
    /// delegates to the project.
    ///
    /// # Example
    ///
    /// ```rust,no_run
    /// # use it_helpers::{testnet::RunningTestPeer, fixed::{TestProject, TestPerson}};
    /// # async fn doit() {
    /// let peer: RunningTestPeer = unimplemented!();
    /// let peer2: RunningTestPeer = unimplemented!();
    ///
    /// let project = peer.using_storage(TestProject::create).await.unwrap().unwrap();
    /// let other_person = peer2.using_storage(TestPerson::create).await.unwrap().unwrap();
    /// project.maintainers(&peer).add(&other_person, &peer2).setup().await.unwrap()
    /// # }
    /// ```
    pub fn maintainers<'a>(&'a self, home: &'a RunningTestPeer) -> Maintainers<'a> {
        Maintainers {
            project: self,
            home,
            other_maintainers: Vec::new(),
        }
    }
}

pub struct Maintainers<'a> {
    project: &'a TestProject,
    home: &'a RunningTestPeer,
    other_maintainers: Vec<(&'a RunningTestPeer, &'a TestPerson)>,
}

impl<'a> Maintainers<'a> {
    pub fn add(mut self, person: &'a TestPerson, peer: &'a RunningTestPeer) -> Self {
        self.other_maintainers.push((peer, person));
        self
    }

    /// Perform the cross signing necessary to add all the maintainers to the
    /// project.
    ///
    /// What this does is the following:
    /// * Track each of the maintainers remotes for the given peer on the `home`
    ///   peer
    /// * Add all of the `Person` identities as indirect delegates of the
    ///   projects on the home peer
    /// * For each maintainer:
    ///     * Pull the updated document into the maintainers peer and `update`
    ///       the document
    ///     * Pull the updated document back into the home peer
    ///     * On the home peer `update` and `merge` the document
    /// * Finally pull the completed document back into each of the maintainer
    ///   peers
    pub async fn setup(self) -> anyhow::Result<()> {
        // make sure the home peer has all the other identities
        for (peer, testperson) in &self.other_maintainers {
            self.home
                .track(self.project.project.urn(), Some(peer.peer_id()))
                .await?;
            testperson.pull(*peer, self.home).await?;
        }
        // Add the other identities as delegates of the project
        self.home
            .using_storage({
                let urn = self.project.project.urn();
                let owners = std::iter::once(self.project.owner.clone())
                    .chain(self.other_maintainers.iter().map(|(_, m)| m.owner.clone()))
                    .map(either::Either::Right)
                    .collect::<Vec<_>>();
                move |storage| -> Result<(), anyhow::Error> {
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        librad::identities::delegation::Indirect::try_from_iter(owners).unwrap(),
                    )?;
                    identities::project::verify(storage, &urn)?;
                    Ok(())
                }
            })
            .await??;

        // For each maintainer, sign the updated document and merge it back into the
        // home peer
        for (peer, _) in &self.other_maintainers {
            // pull the document into the maintainer peer
            self.project.pull(self.home, *peer).await?;
            // Sign the project document using the maintiners peer
            peer.using_storage({
                let urn = self.project.project.urn();
                let peer_id = self.home.peer_id();
                let rad =
                    Urn::try_from(Reference::rad_id(Namespace::from(&urn)).with_remote(peer_id))
                        .unwrap();
                move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                    let project = identities::project::get(&storage, &rad)?.unwrap();
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        project.delegations().clone(),
                    )?;
                    identities::project::merge(storage, &urn, peer_id)?;
                    Ok(identities::project::verify(storage, &urn)?)
                }
            })
            .await??;

            // pull the signed update back into the home peer
            self.project.pull(*peer, self.home).await?;

            // Merge the signed update into peer1
            self.home
                .using_storage({
                    let urn = self.project.project.urn();
                    let peer_id = peer.peer_id();
                    move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                        identities::project::merge(storage, &urn, peer_id)?;
                        Ok(identities::project::verify(storage, &urn)?)
                    }
                })
                .await??;
        }

        // pull the finished document back to the maintainer peers
        for (peer, _) in self.other_maintainers {
            self.project.pull(self.home, peer).await?;
        }
        Ok(())
    }
}
diff --git a/test/it-helpers/src/lib.rs b/test/it-helpers/src/lib.rs
index 981b922d..5012de39 100644
--- a/test/it-helpers/src/lib.rs
+++ b/test/it-helpers/src/lib.rs
@@ -7,3 +7,4 @@ pub mod layout;
pub mod ssh;
pub mod testnet;
pub mod tmp;
pub mod working_copy;
diff --git a/test/it-helpers/src/testnet.rs b/test/it-helpers/src/testnet.rs
index 9274b3f7..4888335b 100644
--- a/test/it-helpers/src/testnet.rs
+++ b/test/it-helpers/src/testnet.rs
@@ -21,7 +21,7 @@ use once_cell::sync::Lazy;
use tempfile::{tempdir, TempDir};

use librad::{
    git,
    git::{self, tracking, Urn},
    net::{
        connection::{LocalAddr, LocalPeer},
        discovery::{self, Discovery as _},
@@ -138,6 +138,20 @@ impl RunningTestPeer {
    pub fn listen_addrs(&self) -> &[SocketAddr] {
        &self.listen_addrs
    }

    pub async fn track(&self, urn: Urn, peer: Option<PeerId>) -> anyhow::Result<()> {
        self.using_storage(move |s| {
            tracking::track(
                s,
                &urn,
                peer,
                tracking::Config::default(),
                tracking::policy::Track::Any,
            )??;
            Ok(())
        })
        .await?
    }
}

impl LocalPeer for RunningTestPeer {
diff --git a/test/it-helpers/src/working_copy.rs b/test/it-helpers/src/working_copy.rs
new file mode 100644
index 00000000..5fbef0dd
--- /dev/null
+++ b/test/it-helpers/src/working_copy.rs
@@ -0,0 +1,291 @@
use std::path::Path;

use git_ref_format::{lit, name, refspec, Qualified, RefStr, RefString};

use librad::{
    git::{
        local::url::LocalUrl,
        types::{
            remote::{LocalFetchspec, LocalPushspec},
            Fetchspec,
            Force,
            Refspec,
            Remote,
        },
    },
    git_ext as ext,
    net::{peer::Peer, protocol::RequestPullGuard},
    refspec_pattern,
    PeerId,
    Signer,
};

use crate::fixed::TestProject;

/// A remote in the working copy
pub enum WorkingRemote {
    /// A remote representing a remote peer, named `PeerId::encode_id`
    Peer(PeerId),
    /// A remote representing the local peer, named "rad"
    Rad,
}

impl From<PeerId> for WorkingRemote {
    fn from(p: PeerId) -> Self {
        WorkingRemote::Peer(p)
    }
}

impl WorkingRemote {
    fn fetchspec(&self) -> Fetchspec {
        match self {
            Self::Peer(peer_id) => {
                let name = RefString::try_from(format!("{}", peer_id)).expect("peer is refstring");
                let dst = RefString::from(Qualified::from(lit::refs_remotes(name.clone())))
                    .with_pattern(refspec::STAR);
                let src = RefString::from(Qualified::from(lit::refs_remotes(name)))
                    .and(name::HEADS)
                    .with_pattern(refspec::STAR);
                let refspec = Refspec {
                    src,
                    dst,
                    force: Force::True,
                };
                refspec.into_fetchspec()
            },
            Self::Rad => {
                let name = RefString::try_from("rad").unwrap();
                let src =
                    RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR);
                Refspec {
                    src,
                    dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                        .with_pattern(refspec::STAR),
                    force: Force::True,
                }
                .into_fetchspec()
            },
        }
    }

    fn remote_ref(&self, branch: &RefStr) -> RefString {
        let name = match self {
            Self::Rad => name::RAD.to_owned(),
            Self::Peer(peer_id) => {
                RefString::try_from(peer_id.to_string()).expect("peer id is refstring")
            },
        };
        RefString::from(Qualified::from(lit::refs_remotes(name))).join(branch)
    }
}

/// A `WorkingCopy` for test driving interactions with the monorepo where one
/// needs to update the tree of a project.
///
/// Remotes are named after the peer ID, except in the case of the remote
/// representing the local Peer ID - which is called "rad".
pub struct WorkingCopy<'a, S, G> {
    repo: git2::Repository,
    peer: &'a Peer<S, G>,
    project: &'a TestProject,
}

impl<'a, S, G> WorkingCopy<'a, S, G>
where
    S: Signer + Clone,
    G: RequestPullGuard,
{
    /// Create a new working copy. This initializes a git repository and then
    /// fetches the state of the local peer into `refs/remotes/rad/*`.
    pub fn new<P: AsRef<Path>>(
        project: &'a TestProject,
        repo_path: P,
        peer: &'a Peer<S, G>,
    ) -> Result<WorkingCopy<'a, S, G>, anyhow::Error> {
        let repo = git2::Repository::init(repo_path.as_ref())?;

        let mut copy = WorkingCopy {
            peer,
            project,
            repo,
        };
        copy.fetch(WorkingRemote::Rad)?;
        Ok(copy)
    }

    /// Fetch changes from the monorepo into the working copy. The fetchspec
    /// used depends on the peer ID.
    ///
    /// * If `from` is `WorkingRemote::Peer` then `refs/remotes/<peer
    ///   ID>/refs/*:refs/remotes/<peer ID>/heads/*`
    /// * If `from` is `WorkingRemote::Rad` then
    ///   `refs/heads/*:refs/remotes/rad/*`
    ///
    /// I.e. changes from remote peers end up in a remote called
    /// `PeerId::encode_id` whilst changes from the local peer end up in a
    /// remote called "rad".
    pub fn fetch(&mut self, from: WorkingRemote) -> Result<(), anyhow::Error> {
        let fetchspec = from.fetchspec();
        let url = LocalUrl::from(self.project.project.urn());
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.fetch(self.peer.clone(), &self.repo, LocalFetchspec::Configured)?;
        Ok(())
    }

    /// Push changes from `refs/heads/*` to the local peer
    pub fn push(&mut self) -> Result<(), anyhow::Error> {
        let url = LocalUrl::from(self.project.project.urn());
        let name = RefString::try_from("rad").unwrap();
        let fetchspec = Refspec {
            src: RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR),
            dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                .with_pattern(refspec::STAR),
            force: Force::True,
        }
        .into_fetchspec();
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.push(
            self.peer.clone(),
            &self.repo,
            LocalPushspec::Matching {
                pattern: refspec_pattern!("refs/heads/*"),
                force: Force::True,
            },
        )?;
        Ok(())
    }

    /// Create a new commit on top of whichever commit is the head of
    /// `on_branch`. If the branch does not exist this will create it.
    pub fn commit(
        &mut self,
        message: &str,
        on_branch: Qualified,
    ) -> Result<git2::Oid, anyhow::Error> {
        let branch_name = on_branch.non_empty_components().2;
        let parent = match self.repo.find_branch(&branch_name, git2::BranchType::Local) {
            Ok(b) => b.get().target().and_then(|o| self.repo.find_commit(o).ok()),
            Err(e) if ext::error::is_not_found_err(&e) => None,
            Err(e) => return Err(anyhow::Error::from(e)),
        };
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = match &parent {
            Some(p) => vec![p],
            None => Vec::new(),
        };
        self.repo
            .commit(
                Some(&on_branch),
                &author,
                &author,
                message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }

    /// Create a branch at `refs/heads/<branch>` which tracks the given remote.
    /// The remote branch name depends on `from`.
    ///
    /// * If `from` is `WorkingCopy::Rad` then `refs/remotes/rad/<branch>`
    /// * If `from` is `WorkingCopy::Peer(peer_id)` then `refs/remotes/<peer
    ///   id>/<branch>`
    pub fn create_remote_tracking_branch(
        &self,
        from: WorkingRemote,
        branch: &RefStr,
    ) -> Result<(), anyhow::Error> {
        let target = self
            .repo
            .find_reference(from.remote_ref(branch).as_str())?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref is not a direct reference"))?;
        let commit = self.repo.find_commit(target)?;
        self.repo.branch(branch.as_str(), &commit, false)?;
        Ok(())
    }

    /// Fast forward the local branch `refs/heads/<branch>` to whatever is
    /// pointed to by `refs/remotes/<remote>/<branch>`
    ///
    /// * If `from` is `WorkingRemote::Peer(peer_id)` then `remote` is
    ///   `peer_id.encode_id()`
    /// * If `from` is `WorkingRemote::Rad` then `remote` is `"rad"`
    ///
    /// # Errors
    ///
    /// * If the local branch does not exist
    /// * If the remote branch does not exist
    /// * If either of the branches does not point at a commit
    /// * If the remote branch is not a descendant of the local branch
    pub fn fast_forward_to(&self, from: WorkingRemote, branch: &RefStr) -> anyhow::Result<()> {
        let remote_ref = from.remote_ref(branch);
        let remote_target = self
            .repo
            .find_reference(&remote_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref had no target"))?;
        let local_ref = RefString::from(Qualified::from(lit::refs_heads(branch)));
        let local_target = self
            .repo
            .find_reference(&local_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("local ref had no target"))?;
        if !self.repo.graph_descendant_of(remote_target, local_target)? {
            anyhow::bail!("remote ref was not a descendant of local ref");
        } else {
            self.repo
                .reference(&local_ref, remote_target, true, "fast forward")?;
        }
        Ok(())
    }

    /// Create a new commit which merges `refs/heads/<branch>` and
    /// `refs/remotes/<remote>/<branch>`
    ///
    /// this will create a new commit with two parents, one for the remote
    /// branch and one for the local branch
    ///
    /// # Errors
    ///
    /// * If the remote branch does not exist
    /// * If the local branch does not exist
    /// * If either of the references does not point to a commit
    pub fn merge_remote(&self, remote: PeerId, branch: &RefStr) -> anyhow::Result<git2::Oid> {
        let peer_branch = WorkingRemote::Peer(remote).remote_ref(branch);
        let peer_commit = self
            .repo
            .find_reference(&peer_branch.to_string())?
            .peel_to_commit()?;
        let local_branch = Qualified::from(lit::refs_heads(branch));
        let local_commit = self
            .repo
            .find_reference(&local_branch.to_string())?
            .peel_to_commit()?;

        let message = format!("merge {} into {}", peer_branch, local_branch);
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = vec![&peer_commit, &local_commit];
        self.repo
            .commit(
                Some(&local_branch),
                &author,
                &author,
                &message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }
}
-- 
2.36.1

[PATCH v3 1/5] Add default_branch_head and set_default_branch Export this patch

When checking out projects from the monorepo it is useful to set the
`refs/namespaces/<urn>/HEAD` reference to the default branch of the
project so that the resulting working copy is in a useful state (namely
pointing at the latest commit for the default branch).

In general this is not possible because delegates may have diverging
views of the project, but often they do not disagree. Add
`librad::git::identities::project::heads::default_branch_head` to
determine if there is an agreed on default branch commit and
`librad::git::identities::project::heads::set_default_branch` to set the
local `HEAD` ref where possible.

Signed-off-by: Alex Good <alex@memoryandthought.me>
---
 librad/src/git/identities/project.rs          |   2 +
 librad/src/git/identities/project/heads.rs    | 282 +++++++++++++++
 librad/t/src/integration/scenario.rs          |   1 +
 .../scenario/default_branch_head.rs           | 338 ++++++++++++++++++
 test/it-helpers/Cargo.toml                    |   4 +
 test/it-helpers/src/fixed.rs                  |   2 +-
 test/it-helpers/src/fixed/project.rs          | 150 +++++++-
 test/it-helpers/src/lib.rs                    |   1 +
 test/it-helpers/src/testnet.rs                |  16 +-
 test/it-helpers/src/working_copy.rs           | 291 +++++++++++++++
 10 files changed, 1084 insertions(+), 3 deletions(-)
 create mode 100644 librad/src/git/identities/project/heads.rs
 create mode 100644 librad/t/src/integration/scenario/default_branch_head.rs
 create mode 100644 test/it-helpers/src/working_copy.rs

diff --git a/librad/src/git/identities/project.rs b/librad/src/git/identities/project.rs
index 753358bf..ed9f003d 100644
--- a/librad/src/git/identities/project.rs
+++ b/librad/src/git/identities/project.rs
@@ -8,6 +8,8 @@ use std::{convert::TryFrom, fmt::Debug};
use either::Either;
use git_ext::{is_not_found_err, OneLevel};

pub mod heads;

use super::{
    super::{
        refs::Refs as Sigrefs,
diff --git a/librad/src/git/identities/project/heads.rs b/librad/src/git/identities/project/heads.rs
new file mode 100644
index 00000000..79c98a92
--- /dev/null
+++ b/librad/src/git/identities/project/heads.rs
@@ -0,0 +1,282 @@
use std::{
    collections::{BTreeSet, HashMap},
    convert::TryFrom,
    fmt::Debug,
};

use crate::{
    git::{
        storage::{self, ReadOnlyStorage},
        Urn,
    },
    identities::git::VerifiedProject,
    PeerId,
};
use git_ext::{is_not_found_err, RefLike};
use git_ref_format::{lit, name, Namespaced, Qualified, RefStr, RefString};

#[derive(Clone, Debug, PartialEq)]
pub enum DefaultBranchHead {
    /// Not all delegates agreed on an ancestry tree. Each set of diverging
    /// delegates is included as a `Fork`
    Forked(BTreeSet<Fork>),
    /// All the delegates agreed on an ancestry tree
    Head {
        /// The most recent commit which all peers agree on
        target: git2::Oid,
        /// The branch name which is the default branch
        branch: RefString,
    },
}

#[derive(Clone, Debug, std::hash::Hash, PartialEq, Eq, PartialOrd, Ord)]
pub struct Fork {
    /// Peers which have a commit in this fork. Peers can appear in multiple
    /// forks if they are ancestors before the fork point.
    pub peers: BTreeSet<PeerId>,
    /// The most recent tip
    pub tip: git2::Oid,
}

pub mod error {
    use git_ref_format as ref_format;
    use std::collections::BTreeSet;

    use crate::git::storage::read;

    #[derive(thiserror::Error, Debug)]
    pub enum FindDefaultBranch {
        #[error("the project payload does not define a default branch")]
        NoDefaultBranch,
        #[error("no peers had published anything for the default branch")]
        NoTips,
        #[error(transparent)]
        RefFormat(#[from] ref_format::Error),
        #[error(transparent)]
        Read(#[from] read::Error),
        #[error(transparent)]
        Git2(#[from] git2::Error),
    }

    #[derive(thiserror::Error, Debug)]
    pub enum SetDefaultBranch {
        #[error(transparent)]
        Find(#[from] FindDefaultBranch),
        #[error(transparent)]
        Git(#[from] git2::Error),
        #[error("the delegates have forked")]
        Forked(BTreeSet<super::Fork>),
    }
}

/// Find the head of the default branch of `project`
///
/// In general there can be a different view of the default branch of a project
/// for each peer ID of each delegate and there is no reason that these would
/// all be compatible. It's quite possible that two peers publish entirely
/// unrelated ancestry trees for a given branch. In this case this function will
/// return [`DefaultBranchHead::Forked`].
///
/// However, often it's the case that delegates do agree on an ancestry tree for
/// a particular branch and the difference between peers is just that some are
/// ahead of others. In this case this function will return
/// [`DefaultBranchHead::Head`].
///
/// # Errors
///
/// * If the project contains no default branch definition
/// * No peers had published anything for the default branch
pub fn default_branch_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
    if let Some(default_branch) = &project.payload().subject.default_branch {
        let local = storage.peer_id();
        let branch_refstring = RefString::try_from(default_branch.to_string())?;
        let mut multiverse = Multiverse::new(branch_refstring.clone());
        let peers =
            project
                .delegations()
                .into_iter()
                .flat_map(|d| -> Box<dyn Iterator<Item = PeerId>> {
                    use either::Either::*;
                    match d {
                        Left(key) => Box::new(std::iter::once(PeerId::from(*key))),
                        Right(person) => Box::new(
                            person
                                .delegations()
                                .into_iter()
                                .map(|key| PeerId::from(*key)),
                        ),
                    }
                });
        for peer_id in peers {
            let tip = peer_commit(storage, project.urn(), peer_id, local, &branch_refstring)?;
            if let Some(tip) = tip {
                multiverse.add_peer(storage, peer_id, tip)?;
            } else {
                tracing::warn!(%peer_id, %default_branch, "no default branch commit found for peer");
            }
        }
        multiverse.finish()
    } else {
        Err(error::FindDefaultBranch::NoDefaultBranch)
    }
}

/// Determine the default branch for a project and set the local HEAD to this
/// branch
///
/// In more detail, this function determines the local head using
/// [`default_branch_head`] and then sets the following references to the
/// `DefaultBranchHead::target` returned:
///
/// * `refs/namespaces/<URN>/refs/HEAD`
/// * `refs/namespaces/<URN>/refs/<default branch name>`
///
/// # Why do this?
///
/// When cloning from a namespace representing a project to a working copy we
/// would like, if possible, to omit the specification of which particular peer
/// we want to clone. Specifically we would like to clone
/// `refs/namespaces/<URN>/`. This does work, but the working copy we end up
/// with does not have any contents because git uses `refs/HEAD` of the source
/// repository to figure out what branch to set the new working copy to.
/// Therefore, by setting `refs/HEAD` and `refs/<default branch name>` of the
/// namespace `git clone` (and any other clone based workflows) does something
/// sensible and we end up with a working copy which is looking at the default
/// branch of the project.
///
/// # Errors
///
/// * If no default branch could be determined
pub fn set_default_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<git2::Oid, error::SetDefaultBranch> {
    let urn = project.urn();
    let default_head = default_branch_head(storage, project)?;
    match default_head {
        DefaultBranchHead::Head { target, branch } => {
            // Note that we can't use `Namespaced` because `refs/HEAD` is not a `Qualified`
            let head =
                RefString::try_from(format!("refs/namespaces/{}/refs/HEAD", urn.encode_id()))
                    .expect("urn is valid namespace");
            let branch_head = Namespaced::from(lit::refs_namespaces(
                &urn,
                Qualified::from(lit::refs_heads(branch)),
            ));

            let repo = storage.as_raw();
            repo.reference(
                &branch_head.clone().into_qualified(),
                target,
                true,
                "set default branch head",
            )?;
            repo.reference_symbolic(head.as_str(), branch_head.as_str(), true, "set head")?;
            Ok(target)
        },
        DefaultBranchHead::Forked(forks) => Err(error::SetDefaultBranch::Forked(forks)),
    }
}

fn peer_commit(
    storage: &storage::Storage,
    urn: Urn,
    peer_id: PeerId,
    local: &PeerId,
    branch: &RefStr,
) -> Result<Option<git2::Oid>, error::FindDefaultBranch> {
    let remote_name = RefString::try_from(peer_id.default_encoding())?;
    let reference = if local == &peer_id {
        RefString::from(Qualified::from(lit::refs_heads(branch)))
    } else {
        RefString::from(Qualified::from(lit::refs_remotes(remote_name)))
            .join(name::HEADS)
            .join(branch)
    };
    let urn = urn.with_path(Some(RefLike::from(reference)));
    let tip = storage.tip(&urn, git2::ObjectType::Commit)?;
    Ok(tip.map(|c| c.id()))
}

#[derive(Debug)]
struct Multiverse {
    branch: RefString,
    histories: Vec<History>,
}

impl Multiverse {
    fn new(branch: RefString) -> Multiverse {
        Multiverse {
            branch,
            histories: Vec::new(),
        }
    }

    fn add_peer(
        &mut self,
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<(), git2::Error> {
        let repo = storage.as_raw();
        let mut found_history = false;
        for History { merge_base, peers } in self.histories.iter_mut() {
            if *merge_base == tip {
                found_history = true;
                peers.insert(peer, tip);
            } else {
                match repo.merge_base(*merge_base, tip) {
                    Err(e) if is_not_found_err(&e) => {},
                    Err(e) => return Err(e),
                    Ok(b) => {
                        found_history = true;
                        peers.insert(peer, tip);
                        *merge_base = b;
                    },
                };
            }
        }
        if !found_history {
            self.histories.push(History::new(tip, peer));
        }
        Ok(())
    }

    fn finish(self) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
        if self.histories.is_empty() {
            Err(error::FindDefaultBranch::NoTips)
        } else if self.histories.len() == 1 {
            Ok(DefaultBranchHead::Head {
                target: self.histories[0].merge_base,
                branch: self.branch,
            })
        } else {
            Ok(DefaultBranchHead::Forked(
                self.histories
                    .into_iter()
                    .map(|h| Fork {
                        peers: h.peers.into_keys().collect(),
                        tip: h.merge_base,
                    })
                    .collect(),
            ))
        }
    }
}

#[derive(Debug, Clone)]
struct History {
    merge_base: git2::Oid,
    peers: HashMap<PeerId, git2::Oid>,
}

impl History {
    fn new(merge_base: git2::Oid, peer: PeerId) -> Self {
        let mut peers = HashMap::new();
        peers.insert(peer, merge_base);
        Self { merge_base, peers }
    }
}
diff --git a/librad/t/src/integration/scenario.rs b/librad/t/src/integration/scenario.rs
index 9bfdd2ad..c47720a0 100644
--- a/librad/t/src/integration/scenario.rs
+++ b/librad/t/src/integration/scenario.rs
@@ -5,6 +5,7 @@

mod collaboration;
mod collaborative_objects;
mod default_branch_head;
mod menage;
mod passive_replication;
#[cfg(feature = "replication-v3")]
diff --git a/librad/t/src/integration/scenario/default_branch_head.rs b/librad/t/src/integration/scenario/default_branch_head.rs
new file mode 100644
index 00000000..07399eed
--- /dev/null
+++ b/librad/t/src/integration/scenario/default_branch_head.rs
@@ -0,0 +1,338 @@
// Copyright © 2019-2020 The Radicle Foundation <hello@radicle.foundation>
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

use std::{convert::TryFrom, ops::Index as _};

use tempfile::tempdir;

use git_ref_format::{lit, name, Namespaced, Qualified, RefString};
use it_helpers::{
    fixed::{TestPerson, TestProject},
    testnet::{self, RunningTestPeer},
    working_copy::{WorkingCopy, WorkingRemote as Remote},
};
use librad::git::{
    identities::{self, local, project::heads},
    storage::ReadOnlyStorage,
};
use link_identities::payload;
use test_helpers::logging;

fn config() -> testnet::Config {
    testnet::Config {
        num_peers: nonzero!(2usize),
        min_connected: 2,
        bootstrap: testnet::Bootstrap::from_env(),
    }
}

/// This test checks that the logic of `librad::git::identities::project::heads`
/// is correct. To do this we need to set up various scenarios where the
/// delegates of a project agree or disagree on the default branch of a project.
#[test]
fn default_branch_head() {
    logging::init();

    let net = testnet::run(config()).unwrap();
    net.enter(async {
        // Setup  a testnet with two peers
        let peer1 = net.peers().index(0);
        let peer2 = net.peers().index(1);

        // Create an identity on peer2
        let peer2_id = peer2
            .using_storage::<_, anyhow::Result<TestPerson>>(|s| {
                let person = TestPerson::create(s)?;
                let local = local::load(s, person.owner.urn()).unwrap();
                s.config()?.set_user(local)?;
                Ok(person)
            })
            .await
            .unwrap()
            .unwrap();

        peer2_id.pull(peer2, peer1).await.unwrap();

        // Create a project on peer1
        let proj = peer1
            .using_storage(|s| {
                TestProject::create_with_payload(
                    s,
                    payload::Project {
                        name: "venus".into(),
                        description: None,
                        default_branch: Some(name::MASTER.to_string().into()),
                    },
                )
            })
            .await
            .unwrap()
            .unwrap();

        // Add peer2 as a maintainer
        proj.maintainers(peer1)
            .add(&peer2_id, peer2)
            .setup()
            .await
            .unwrap();

        //// Okay, now we have a running testnet with two Peers, each of which has a
        //// `Person` who is a delegate on the `TestProject`

        // Create a basic history which contains two commits, one from peer1 and one
        // from peer2
        //
        // * Create a commit in peer 1
        // * pull to peer2
        // * create a new commit on top of the original commit in peer2
        // * pull peer2 back to peer1
        // * in peer1 fast forward and push
        //
        // At this point both peers should have a history like
        //
        //     peer1 commit
        //           ↓
        //     peer2 commit
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            working_copy1
                .commit("peer 1 initial", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();
            proj.pull(peer1, peer2).await.unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            let tip = working_copy2
                .commit("peer 2 initial", mastor.clone())
                .unwrap();
            working_copy2.push().unwrap();
            proj.pull(peer2, peer1).await.unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .fast_forward_to(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch = branch_head(peer1, &proj).await.unwrap();
        // The two peers should have the same view of the default branch
        assert_eq!(
            default_branch,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now update peer1 and push to peer 1s monorepo, we should still get the old
        // tip because peer2 is behind
        let tmp = tempdir().unwrap();
        let new_tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let tip = working_copy1
                .commit("peer 1 update", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();

            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // fast forward peer2 and pull the update back into peer1
        let tmp = tempdir().unwrap();
        proj.pull(peer1, peer2).await.unwrap();
        {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .fast_forward_to(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            working_copy2.push().unwrap();
        }
        proj.pull(peer2, peer1).await.unwrap();

        // Now we should be pointing at the latest tip because both peer1 and peer2
        // agree
        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: new_tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now create an alternate commit on peer2 and sync with peer1, on peer1 we
        // should get a fork
        let tmp = tempdir().unwrap();
        let forked_tip = {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let forked_tip = working_copy2.commit("peer 2 fork", mastor.clone()).unwrap();
            working_copy2.push().unwrap();

            forked_tip
        };
        proj.pull(peer2, peer1).await.unwrap();

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Forked(
                vec![
                    identities::project::heads::Fork {
                        peers: std::iter::once(peer1.peer_id()).collect(),
                        tip: new_tip,
                    },
                    identities::project::heads::Fork {
                        peers: std::iter::once(peer2.peer_id()).collect(),
                        tip: forked_tip,
                    }
                ]
                .into_iter()
                .collect()
            )
        );

        // now merge the fork into peer1
        let tmp = tempdir().unwrap();
        let fixed_tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            let tip = working_copy1
                .merge_remote(peer2.peer_id(), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        // pull the merge into peer2
        proj.pull(peer1, peer2).await.unwrap();
        {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .fast_forward_to(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            working_copy2.push().unwrap();
        }
        proj.pull(peer2, peer1).await.unwrap();

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: fixed_tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now set the head in the monorepo and check that the HEAD reference exists
        let updated_tip = peer1
            .using_storage::<_, anyhow::Result<_>>({
                let urn = proj.project.urn();
                move |s| {
                    let vp = identities::project::verify(s, &urn)?.ok_or_else(|| {
                        anyhow::anyhow!("failed to get project for default branch")
                    })?;
                    identities::project::heads::set_default_head(s, vp).map_err(anyhow::Error::from)
                }
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(updated_tip, fixed_tip);

        let head_ref = RefString::try_from(format!(
            "refs/namespaces/{}/refs/HEAD",
            proj.project.urn().encode_id()
        ))
        .unwrap();
        let master_ref = Namespaced::from(lit::refs_namespaces(
            &proj.project.urn(),
            Qualified::from(lit::refs_heads(name::MASTER)),
        ));
        let (master_oid, head_target) = peer1
            .using_storage::<_, anyhow::Result<_>>({
                let master_ref = master_ref.clone();
                move |s| {
                    let master_oid = s
                        .reference(&master_ref.into_qualified().into_refstring())?
                        .ok_or_else(|| anyhow::anyhow!("master ref not found"))?
                        .peel_to_commit()?
                        .id();
                    let head_target = s
                        .reference(&head_ref)?
                        .ok_or_else(|| anyhow::anyhow!("head ref not found"))?
                        .symbolic_target()
                        .map(|s| s.to_string());
                    Ok((master_oid, head_target))
                }
            })
            .await
            .unwrap()
            .unwrap();
        assert_eq!(master_oid, updated_tip);
        assert_eq!(head_target, Some(master_ref.to_string()));
    });
}

async fn branch_head(
    peer: &RunningTestPeer,
    proj: &TestProject,
) -> anyhow::Result<heads::DefaultBranchHead> {
    peer.using_storage::<_, anyhow::Result<_>>({
        let urn = proj.project.urn();
        move |s| {
            let vp = identities::project::verify(s, &urn)?
                .ok_or_else(|| anyhow::anyhow!("failed to get project for default branch"))?;
            heads::default_branch_head(s, vp).map_err(anyhow::Error::from)
        }
    })
    .await?
}
diff --git a/test/it-helpers/Cargo.toml b/test/it-helpers/Cargo.toml
index 32c789cd..04aa2166 100644
--- a/test/it-helpers/Cargo.toml
+++ b/test/it-helpers/Cargo.toml
@@ -18,6 +18,7 @@ once_cell = "1.10"
tempfile = "3.3"
tokio = "1.13"
tracing = "0.1"
either = "1.6"

[dependencies.git2]
version = "0.13.24"
@@ -40,5 +41,8 @@ path = "../../link-async"
[dependencies.lnk-clib]
path = "../../cli/lnk-clib"

[dependencies.radicle-git-ext]
path = "../../git-ext"

[dependencies.test-helpers]
path = "../test-helpers"
diff --git a/test/it-helpers/src/fixed.rs b/test/it-helpers/src/fixed.rs
index c36f5dd6..53006a31 100644
--- a/test/it-helpers/src/fixed.rs
+++ b/test/it-helpers/src/fixed.rs
@@ -2,7 +2,7 @@ mod person;
pub use person::TestPerson;

mod project;
pub use project::TestProject;
pub use project::{Maintainers, TestProject};

pub mod repository;
pub use repository::{commit, repository};
diff --git a/test/it-helpers/src/fixed/project.rs b/test/it-helpers/src/fixed/project.rs
index 4ce3a41e..37a48832 100644
--- a/test/it-helpers/src/fixed/project.rs
+++ b/test/it-helpers/src/fixed/project.rs
@@ -4,6 +4,8 @@ use librad::{
    git::{
        identities::{self, Person, Project},
        storage::Storage,
        types::{Namespace, Reference},
        Urn,
    },
    identities::{
        delegation::{self, Direct},
@@ -18,6 +20,8 @@ use librad::{
};
use tracing::{info, instrument};

use crate::testnet::RunningTestPeer;

use super::TestPerson;

pub struct TestProject {
@@ -27,6 +31,13 @@ pub struct TestProject {

impl TestProject {
    pub fn create(storage: &Storage) -> anyhow::Result<Self> {
        Self::create_with_payload(storage, Self::default_payload())
    }

    pub fn create_with_payload(
        storage: &Storage,
        payload: payload::Project,
    ) -> anyhow::Result<Self> {
        let peer_id = storage.peer_id();
        let alice = identities::person::create(
            storage,
@@ -40,7 +51,7 @@ impl TestProject {
        let proj = identities::project::create(
            storage,
            local_id,
            Self::default_payload(),
            payload,
            delegation::Indirect::from(alice.clone()),
        )?;

@@ -120,4 +131,141 @@ impl TestProject {
            .replicate((remote_peer, remote_addrs), urn, None)
            .await?)
    }

    /// Add maintainers to a TestProject
    ///
    /// The `home` argument must be a peer which is already a delegate of the
    /// project. The [`Maintainers`] struct which is returned can be used to
    /// add maintainers using [`Maintainers::add`] before calling
    /// [`Maintainers::setup`] to perform the cross signing which adds the
    /// delegates to the project.
    ///
    /// # Example
    ///
    /// ```rust,no_run
    /// # use it_helpers::{testnet::RunningTestPeer, fixed::{TestProject, TestPerson}};
    /// # async fn doit() {
    /// let peer: RunningTestPeer = unimplemented!();
    /// let peer2: RunningTestPeer = unimplemented!();
    ///
    /// let project = peer.using_storage(TestProject::create).await.unwrap().unwrap();
    /// let other_person = peer2.using_storage(TestPerson::create).await.unwrap().unwrap();
    /// project.maintainers(&peer).add(&other_person, &peer2).setup().await.unwrap()
    /// # }
    /// ```
    pub fn maintainers<'a>(&'a self, home: &'a RunningTestPeer) -> Maintainers<'a> {
        Maintainers {
            project: self,
            home,
            other_maintainers: Vec::new(),
        }
    }
}

pub struct Maintainers<'a> {
    project: &'a TestProject,
    home: &'a RunningTestPeer,
    other_maintainers: Vec<(&'a RunningTestPeer, &'a TestPerson)>,
}

impl<'a> Maintainers<'a> {
    pub fn add(mut self, person: &'a TestPerson, peer: &'a RunningTestPeer) -> Self {
        self.other_maintainers.push((peer, person));
        self
    }

    /// Perform the cross signing necessary to add all the maintainers to the
    /// project.
    ///
    /// What this does is the following:
    /// * Track each of the maintainers remotes for the given peer on the `home`
    ///   peer
    /// * Add all of the `Person` identities as indirect delegates of the
    ///   projects on the home peer
    /// * For each maintainer:
    ///     * Pull the updated document into the maintainers peer and `update`
    ///       the document
    ///     * Pull the updated document back into the home peer
    ///     * On the home peer `update` and `merge` the document
    /// * Finally pull the completed document back into each of the maintainer
    ///   peers
    pub async fn setup(self) -> anyhow::Result<()> {
        // make sure the home peer has all the other identities
        for (peer, testperson) in &self.other_maintainers {
            self.home
                .track(self.project.project.urn(), Some(peer.peer_id()))
                .await?;
            testperson.pull(*peer, self.home).await?;
        }
        // Add the other identities as delegates of the project
        self.home
            .using_storage({
                let urn = self.project.project.urn();
                let owners = std::iter::once(self.project.owner.clone())
                    .chain(self.other_maintainers.iter().map(|(_, m)| m.owner.clone()))
                    .map(either::Either::Right)
                    .collect::<Vec<_>>();
                move |storage| -> Result<(), anyhow::Error> {
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        librad::identities::delegation::Indirect::try_from_iter(owners).unwrap(),
                    )?;
                    identities::project::verify(storage, &urn)?;
                    Ok(())
                }
            })
            .await??;

        // For each maintainer, sign the updated document and merge it back into the
        // home peer
        for (peer, _) in &self.other_maintainers {
            // pull the document into the maintainer peer
            self.project.pull(self.home, *peer).await?;
            // Sign the project document using the maintiners peer
            peer.using_storage({
                let urn = self.project.project.urn();
                let peer_id = self.home.peer_id();
                let rad =
                    Urn::try_from(Reference::rad_id(Namespace::from(&urn)).with_remote(peer_id))
                        .unwrap();
                move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                    let project = identities::project::get(&storage, &rad)?.unwrap();
                    identities::project::update(
                        storage,
                        &urn,
                        None,
                        None,
                        project.delegations().clone(),
                    )?;
                    identities::project::merge(storage, &urn, peer_id)?;
                    Ok(identities::project::verify(storage, &urn)?)
                }
            })
            .await??;

            // pull the signed update back into the home peer
            self.project.pull(*peer, self.home).await?;

            // Merge the signed update into peer1
            self.home
                .using_storage({
                    let urn = self.project.project.urn();
                    let peer_id = peer.peer_id();
                    move |storage| -> Result<Option<identities::VerifiedProject>, anyhow::Error> {
                        identities::project::merge(storage, &urn, peer_id)?;
                        Ok(identities::project::verify(storage, &urn)?)
                    }
                })
                .await??;
        }

        // pull the finished document back to the maintainer peers
        for (peer, _) in self.other_maintainers {
            self.project.pull(self.home, peer).await?;
        }
        Ok(())
    }
}
diff --git a/test/it-helpers/src/lib.rs b/test/it-helpers/src/lib.rs
index 981b922d..5012de39 100644
--- a/test/it-helpers/src/lib.rs
+++ b/test/it-helpers/src/lib.rs
@@ -7,3 +7,4 @@ pub mod layout;
pub mod ssh;
pub mod testnet;
pub mod tmp;
pub mod working_copy;
diff --git a/test/it-helpers/src/testnet.rs b/test/it-helpers/src/testnet.rs
index 9274b3f7..4888335b 100644
--- a/test/it-helpers/src/testnet.rs
+++ b/test/it-helpers/src/testnet.rs
@@ -21,7 +21,7 @@ use once_cell::sync::Lazy;
use tempfile::{tempdir, TempDir};

use librad::{
    git,
    git::{self, tracking, Urn},
    net::{
        connection::{LocalAddr, LocalPeer},
        discovery::{self, Discovery as _},
@@ -138,6 +138,20 @@ impl RunningTestPeer {
    pub fn listen_addrs(&self) -> &[SocketAddr] {
        &self.listen_addrs
    }

    pub async fn track(&self, urn: Urn, peer: Option<PeerId>) -> anyhow::Result<()> {
        self.using_storage(move |s| {
            tracking::track(
                s,
                &urn,
                peer,
                tracking::Config::default(),
                tracking::policy::Track::Any,
            )??;
            Ok(())
        })
        .await?
    }
}

impl LocalPeer for RunningTestPeer {
diff --git a/test/it-helpers/src/working_copy.rs b/test/it-helpers/src/working_copy.rs
new file mode 100644
index 00000000..5fbef0dd
--- /dev/null
+++ b/test/it-helpers/src/working_copy.rs
@@ -0,0 +1,291 @@
use std::path::Path;

use git_ref_format::{lit, name, refspec, Qualified, RefStr, RefString};

use librad::{
    git::{
        local::url::LocalUrl,
        types::{
            remote::{LocalFetchspec, LocalPushspec},
            Fetchspec,
            Force,
            Refspec,
            Remote,
        },
    },
    git_ext as ext,
    net::{peer::Peer, protocol::RequestPullGuard},
    refspec_pattern,
    PeerId,
    Signer,
};

use crate::fixed::TestProject;

/// A remote in the working copy
pub enum WorkingRemote {
    /// A remote representing a remote peer, named `PeerId::encode_id`
    Peer(PeerId),
    /// A remote representing the local peer, named "rad"
    Rad,
}

impl From<PeerId> for WorkingRemote {
    fn from(p: PeerId) -> Self {
        WorkingRemote::Peer(p)
    }
}

impl WorkingRemote {
    fn fetchspec(&self) -> Fetchspec {
        match self {
            Self::Peer(peer_id) => {
                let name = RefString::try_from(format!("{}", peer_id)).expect("peer is refstring");
                let dst = RefString::from(Qualified::from(lit::refs_remotes(name.clone())))
                    .with_pattern(refspec::STAR);
                let src = RefString::from(Qualified::from(lit::refs_remotes(name)))
                    .and(name::HEADS)
                    .with_pattern(refspec::STAR);
                let refspec = Refspec {
                    src,
                    dst,
                    force: Force::True,
                };
                refspec.into_fetchspec()
            },
            Self::Rad => {
                let name = RefString::try_from("rad").unwrap();
                let src =
                    RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR);
                Refspec {
                    src,
                    dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                        .with_pattern(refspec::STAR),
                    force: Force::True,
                }
                .into_fetchspec()
            },
        }
    }

    fn remote_ref(&self, branch: &RefStr) -> RefString {
        let name = match self {
            Self::Rad => name::RAD.to_owned(),
            Self::Peer(peer_id) => {
                RefString::try_from(peer_id.to_string()).expect("peer id is refstring")
            },
        };
        RefString::from(Qualified::from(lit::refs_remotes(name))).join(branch)
    }
}

/// A `WorkingCopy` for test driving interactions with the monorepo where one
/// needs to update the tree of a project.
///
/// Remotes are named after the peer ID, except in the case of the remote
/// representing the local Peer ID - which is called "rad".
pub struct WorkingCopy<'a, S, G> {
    repo: git2::Repository,
    peer: &'a Peer<S, G>,
    project: &'a TestProject,
}

impl<'a, S, G> WorkingCopy<'a, S, G>
where
    S: Signer + Clone,
    G: RequestPullGuard,
{
    /// Create a new working copy. This initializes a git repository and then
    /// fetches the state of the local peer into `refs/remotes/rad/*`.
    pub fn new<P: AsRef<Path>>(
        project: &'a TestProject,
        repo_path: P,
        peer: &'a Peer<S, G>,
    ) -> Result<WorkingCopy<'a, S, G>, anyhow::Error> {
        let repo = git2::Repository::init(repo_path.as_ref())?;

        let mut copy = WorkingCopy {
            peer,
            project,
            repo,
        };
        copy.fetch(WorkingRemote::Rad)?;
        Ok(copy)
    }

    /// Fetch changes from the monorepo into the working copy. The fetchspec
    /// used depends on the peer ID.
    ///
    /// * If `from` is `WorkingRemote::Peer` then `refs/remotes/<peer
    ///   ID>/refs/*:refs/remotes/<peer ID>/heads/*`
    /// * If `from` is `WorkingRemote::Rad` then
    ///   `refs/heads/*:refs/remotes/rad/*`
    ///
    /// I.e. changes from remote peers end up in a remote called
    /// `PeerId::encode_id` whilst changes from the local peer end up in a
    /// remote called "rad".
    pub fn fetch(&mut self, from: WorkingRemote) -> Result<(), anyhow::Error> {
        let fetchspec = from.fetchspec();
        let url = LocalUrl::from(self.project.project.urn());
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.fetch(self.peer.clone(), &self.repo, LocalFetchspec::Configured)?;
        Ok(())
    }

    /// Push changes from `refs/heads/*` to the local peer
    pub fn push(&mut self) -> Result<(), anyhow::Error> {
        let url = LocalUrl::from(self.project.project.urn());
        let name = RefString::try_from("rad").unwrap();
        let fetchspec = Refspec {
            src: RefString::from_iter([name::REFS, name::HEADS]).with_pattern(refspec::STAR),
            dst: RefString::from(Qualified::from(lit::refs_remotes(name)))
                .with_pattern(refspec::STAR),
            force: Force::True,
        }
        .into_fetchspec();
        let mut remote = Remote::rad_remote(url, fetchspec);
        let _ = remote.push(
            self.peer.clone(),
            &self.repo,
            LocalPushspec::Matching {
                pattern: refspec_pattern!("refs/heads/*"),
                force: Force::True,
            },
        )?;
        Ok(())
    }

    /// Create a new commit on top of whichever commit is the head of
    /// `on_branch`. If the branch does not exist this will create it.
    pub fn commit(
        &mut self,
        message: &str,
        on_branch: Qualified,
    ) -> Result<git2::Oid, anyhow::Error> {
        let branch_name = on_branch.non_empty_components().2;
        let parent = match self.repo.find_branch(&branch_name, git2::BranchType::Local) {
            Ok(b) => b.get().target().and_then(|o| self.repo.find_commit(o).ok()),
            Err(e) if ext::error::is_not_found_err(&e) => None,
            Err(e) => return Err(anyhow::Error::from(e)),
        };
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = match &parent {
            Some(p) => vec![p],
            None => Vec::new(),
        };
        self.repo
            .commit(
                Some(&on_branch),
                &author,
                &author,
                message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }

    /// Create a branch at `refs/heads/<branch>` which tracks the given remote.
    /// The remote branch name depends on `from`.
    ///
    /// * If `from` is `WorkingCopy::Rad` then `refs/remotes/rad/<branch>`
    /// * If `from` is `WorkingCopy::Peer(peer_id)` then `refs/remotes/<peer
    ///   id>/<branch>`
    pub fn create_remote_tracking_branch(
        &self,
        from: WorkingRemote,
        branch: &RefStr,
    ) -> Result<(), anyhow::Error> {
        let target = self
            .repo
            .find_reference(from.remote_ref(branch).as_str())?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref is not a direct reference"))?;
        let commit = self.repo.find_commit(target)?;
        self.repo.branch(branch.as_str(), &commit, false)?;
        Ok(())
    }

    /// Fast forward the local branch `refs/heads/<branch>` to whatever is
    /// pointed to by `refs/remotes/<remote>/<branch>`
    ///
    /// * If `from` is `WorkingRemote::Peer(peer_id)` then `remote` is
    ///   `peer_id.encode_id()`
    /// * If `from` is `WorkingRemote::Rad` then `remote` is `"rad"`
    ///
    /// # Errors
    ///
    /// * If the local branch does not exist
    /// * If the remote branch does not exist
    /// * If either of the branches does not point at a commit
    /// * If the remote branch is not a descendant of the local branch
    pub fn fast_forward_to(&self, from: WorkingRemote, branch: &RefStr) -> anyhow::Result<()> {
        let remote_ref = from.remote_ref(branch);
        let remote_target = self
            .repo
            .find_reference(&remote_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("remote ref had no target"))?;
        let local_ref = RefString::from(Qualified::from(lit::refs_heads(branch)));
        let local_target = self
            .repo
            .find_reference(&local_ref)?
            .target()
            .ok_or_else(|| anyhow::anyhow!("local ref had no target"))?;
        if !self.repo.graph_descendant_of(remote_target, local_target)? {
            anyhow::bail!("remote ref was not a descendant of local ref");
        } else {
            self.repo
                .reference(&local_ref, remote_target, true, "fast forward")?;
        }
        Ok(())
    }

    /// Create a new commit which merges `refs/heads/<branch>` and
    /// `refs/remotes/<remote>/<branch>`
    ///
    /// this will create a new commit with two parents, one for the remote
    /// branch and one for the local branch
    ///
    /// # Errors
    ///
    /// * If the remote branch does not exist
    /// * If the local branch does not exist
    /// * If either of the references does not point to a commit
    pub fn merge_remote(&self, remote: PeerId, branch: &RefStr) -> anyhow::Result<git2::Oid> {
        let peer_branch = WorkingRemote::Peer(remote).remote_ref(branch);
        let peer_commit = self
            .repo
            .find_reference(&peer_branch.to_string())?
            .peel_to_commit()?;
        let local_branch = Qualified::from(lit::refs_heads(branch));
        let local_commit = self
            .repo
            .find_reference(&local_branch.to_string())?
            .peel_to_commit()?;

        let message = format!("merge {} into {}", peer_branch, local_branch);
        let empty_tree = {
            let mut index = self.repo.index()?;
            let oid = index.write_tree()?;
            self.repo.find_tree(oid).unwrap()
        };
        let author = git2::Signature::now("The Animal", "animal@muppets.com").unwrap();
        let parents = vec![&peer_commit, &local_commit];
        self.repo
            .commit(
                Some(&local_branch),
                &author,
                &author,
                &message,
                &empty_tree,
                &parents,
            )
            .map_err(anyhow::Error::from)
    }
}
-- 
2.36.1

[PATCH v4 1/5] Add default_branch_head and set_default_branch Export this patch

When checking out projects from the monorepo it is useful to set the
`refs/namespaces/<urn>/HEAD` reference to the default branch of the
project so that the resulting working copy is in a useful state (namely
pointing at the latest commit for the default branch).

In general this is not possible because delegates may have diverging
views of the project, but often they do not disagree. Add
`librad::git::identities::project::heads::default_branch_head` to
determine if there is an agreed on default branch commit and
`librad::git::identities::project::heads::set_default_branch` to set the
local `HEAD` ref where possible.

Signed-off-by: Alex Good <alex@memoryandthought.me>
---
 librad/src/git/identities/project.rs          |   2 +
 librad/src/git/identities/project/heads.rs    | 283 +++++++++++++++
 librad/t/src/integration/scenario.rs          |   1 +
 .../scenario/default_branch_head.rs           | 338 ++++++++++++++++++
 test/it-helpers/Cargo.toml                    |   4 +
 test/it-helpers/src/fixed.rs                  |   2 +-
 test/it-helpers/src/fixed/project.rs          | 150 +++++++-
 test/it-helpers/src/lib.rs                    |   1 +
 test/it-helpers/src/testnet.rs                |  16 +-
 test/it-helpers/src/working_copy.rs           | 291 +++++++++++++++
 10 files changed, 1085 insertions(+), 3 deletions(-)
 create mode 100644 librad/src/git/identities/project/heads.rs
 create mode 100644 librad/t/src/integration/scenario/default_branch_head.rs
 create mode 100644 test/it-helpers/src/working_copy.rs

diff --git a/librad/src/git/identities/project.rs b/librad/src/git/identities/project.rs
index 753358bf..ed9f003d 100644
--- a/librad/src/git/identities/project.rs
+++ b/librad/src/git/identities/project.rs
@@ -8,6 +8,8 @@ use std::{convert::TryFrom, fmt::Debug};
use either::Either;
use git_ext::{is_not_found_err, OneLevel};

pub mod heads;

use super::{
    super::{
        refs::Refs as Sigrefs,
diff --git a/librad/src/git/identities/project/heads.rs b/librad/src/git/identities/project/heads.rs
new file mode 100644
index 00000000..1fd444f7
--- /dev/null
+++ b/librad/src/git/identities/project/heads.rs
@@ -0,0 +1,283 @@
use std::{
    collections::{BTreeSet, HashMap},
    convert::TryFrom,
    fmt::Debug,
};

use crate::{
    git::{
        refs::Refs,
        storage::{self, ReadOnlyStorage},
        Urn,
    },
    identities::git::VerifiedProject,
    PeerId,
};
use git_ext::{is_not_found_err, RefLike};
use git_ref_format::{lit, name, Namespaced, Qualified, RefStr, RefString};

#[derive(Clone, Debug, PartialEq)]
pub enum DefaultBranchHead {
    /// Not all delegates agreed on an ancestry tree. Each set of diverging
    /// delegates is included as a `Fork`
    Forked(BTreeSet<Fork>),
    /// All the delegates agreed on an ancestry tree
    Head {
        /// The most recent commit which all peers agree on
        target: git2::Oid,
        /// The branch name which is the default branch
        branch: RefString,
    },
}

#[derive(Clone, Debug, std::hash::Hash, PartialEq, Eq, PartialOrd, Ord)]
pub struct Fork {
    /// Peers which have a commit in this fork. Peers can appear in multiple
    /// forks if they are ancestors before the fork point.
    pub peers: BTreeSet<PeerId>,
    /// The most recent tip
    pub tip: git2::Oid,
}

pub mod error {
    use git_ref_format as ref_format;
    use std::collections::BTreeSet;

    use crate::git::{refs, storage::read};

    #[derive(thiserror::Error, Debug)]
    pub enum FindDefaultBranch {
        #[error("the project payload does not define a default branch")]
        NoDefaultBranch,
        #[error("no peers had published anything for the default branch")]
        NoTips,
        #[error(transparent)]
        RefFormat(#[from] ref_format::Error),
        #[error(transparent)]
        Read(#[from] read::Error),
        #[error(transparent)]
        Git2(#[from] git2::Error),
    }

    #[derive(thiserror::Error, Debug)]
    pub enum SetDefaultBranch {
        #[error(transparent)]
        Find(#[from] FindDefaultBranch),
        #[error(transparent)]
        Git(#[from] git2::Error),
        #[error("the delegates have forked")]
        Forked(BTreeSet<super::Fork>),
        #[error(transparent)]
        UpdateRefs(#[from] refs::stored::Error),
    }
}

/// Find the head of the default branch of `project`
///
/// In general there can be a different view of the default branch of a project
/// for each peer ID of each delegate and there is no reason that these would
/// all be compatible. It's quite possible that two peers publish entirely
/// unrelated ancestry trees for a given branch. In this case this function will
/// return [`DefaultBranchHead::Forked`].
///
/// However, often it's the case that delegates do agree on an ancestry tree for
/// a particular branch and the difference between peers is just that some are
/// ahead of others. In this case this function will return
/// [`DefaultBranchHead::Head`].
///
/// # Errors
///
/// * If the project contains no default branch definition
/// * No peers had published anything for the default branch
pub fn default_branch_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
    if let Some(default_branch) = &project.payload().subject.default_branch {
        let local = storage.peer_id();
        let branch_refstring = RefString::try_from(default_branch.to_string())?;
        let mut multiverse = Multiverse::new(branch_refstring.clone());
        let peers =
            project
                .delegations()
                .into_iter()
                .flat_map(|d| -> Box<dyn Iterator<Item = PeerId>> {
                    use either::Either::*;
                    match d {
                        Left(key) => Box::new(std::iter::once(PeerId::from(*key))),
                        Right(person) => Box::new(
                            person
                                .delegations()
                                .into_iter()
                                .map(|key| PeerId::from(*key)),
                        ),
                    }
                });
        for peer_id in peers {
            let tip = peer_commit(storage, project.urn(), peer_id, local, &branch_refstring)?;
            if let Some(tip) = tip {
                multiverse.add_peer(storage, peer_id, tip)?;
            } else {
                tracing::warn!(%peer_id, %default_branch, "no default branch commit found for peer");
            }
        }
        multiverse.finish()
    } else {
        Err(error::FindDefaultBranch::NoDefaultBranch)
    }
}

/// Determine the default branch for a project and set the local HEAD to this
/// branch
///
/// In more detail, this function determines the local head using
/// [`default_branch_head`] and then sets the following references
///
/// * `refs/namespaces/<URN>/refs/heads/<default branch name>`
/// * `refs/namespaces/<URN>/refs/HEAD`
///
/// WARNING: this will update the local <default branch> ref and recompute
/// sigrefs
///
/// # Why do this?
///
/// When cloning from a namespace representing a project to a working copy we
/// would like, if possible, to omit the specification of which particular peer
/// we want to clone. Specifically we would like to clone
/// `refs/namespaces/<URN>/`. This does work, but the working copy we end up
/// with does not have any contents because git uses `refs/HEAD` of the source
/// repository to figure out what branch to set the new working copy to.
/// Therefore, by setting `refs/HEAD` and `refs/<default branch name>` of the
/// namespace `git clone` (and any other clone based workflows) does something
/// sensible and we end up with a working copy which is looking at the default
/// branch of the project.
///
/// # Errors
///
/// * If no default branch could be determined
pub fn set_default_head(
    storage: &storage::Storage,
    project: VerifiedProject,
) -> Result<git2::Oid, error::SetDefaultBranch> {
    let urn = project.urn();
    let default_head = default_branch_head(storage, project)?;
    match default_head {
        DefaultBranchHead::Head { target, branch } => {
            // Note that we can't use `Namespaced` because `refs/HEAD` is not a `Qualified`
            let head =
                RefString::try_from(format!("refs/namespaces/{}/refs/HEAD", urn.encode_id()))
                    .expect("urn is valid namespace");
            let branch_head = Namespaced::from(lit::refs_namespaces(
                &urn,
                Qualified::from(lit::refs_heads(branch)),
            ));

            let repo = storage.as_raw();
            repo.reference(&branch_head, target, true, "set default branch head")?;
            repo.reference_symbolic(head.as_str(), branch_head.as_str(), true, "set head")?;
            Refs::update(storage, &urn)?;
            Ok(target)
        },
        DefaultBranchHead::Forked(forks) => Err(error::SetDefaultBranch::Forked(forks)),
    }
}

fn peer_commit(
    storage: &storage::Storage,
    urn: Urn,
    peer_id: PeerId,
    local: &PeerId,
    branch: &RefStr,
) -> Result<Option<git2::Oid>, error::FindDefaultBranch> {
    let remote_name = RefString::try_from(peer_id.default_encoding())?;
    let reference = if local == &peer_id {
        RefString::from(Qualified::from(lit::refs_heads(branch)))
    } else {
        RefString::from(Qualified::from(lit::refs_remotes(remote_name)))
            .join(name::HEADS)
            .join(branch)
    };
    let urn = urn.with_path(Some(RefLike::from(reference)));
    let tip = storage.tip(&urn, git2::ObjectType::Commit)?;
    Ok(tip.map(|c| c.id()))
}

#[derive(Debug)]
struct Multiverse {
    branch: RefString,
    histories: Vec<History>,
}

impl Multiverse {
    fn new(branch: RefString) -> Multiverse {
        Multiverse {
            branch,
            histories: Vec::new(),
        }
    }

    fn add_peer(
        &mut self,
        storage: &storage::Storage,
        peer: PeerId,
        tip: git2::Oid,
    ) -> Result<(), git2::Error> {
        let repo = storage.as_raw();
        let mut found_history = false;
        for History { merge_base, peers } in self.histories.iter_mut() {
            if *merge_base == tip {
                found_history = true;
                peers.insert(peer, tip);
            } else {
                match repo.merge_base(*merge_base, tip) {
                    Err(e) if is_not_found_err(&e) => {},
                    Err(e) => return Err(e),
                    Ok(b) => {
                        found_history = true;
                        peers.insert(peer, tip);
                        *merge_base = b;
                    },
                };
            }
        }
        if !found_history {
            self.histories.push(History::new(tip, peer));
        }
        Ok(())
    }

    fn finish(self) -> Result<DefaultBranchHead, error::FindDefaultBranch> {
        if self.histories.is_empty() {
            Err(error::FindDefaultBranch::NoTips)
        } else if self.histories.len() == 1 {
            Ok(DefaultBranchHead::Head {
                target: self.histories[0].merge_base,
                branch: self.branch,
            })
        } else {
            Ok(DefaultBranchHead::Forked(
                self.histories
                    .into_iter()
                    .map(|h| Fork {
                        peers: h.peers.into_keys().collect(),
                        tip: h.merge_base,
                    })
                    .collect(),
            ))
        }
    }
}

#[derive(Debug, Clone)]
struct History {
    merge_base: git2::Oid,
    peers: HashMap<PeerId, git2::Oid>,
}

impl History {
    fn new(merge_base: git2::Oid, peer: PeerId) -> Self {
        let mut peers = HashMap::new();
        peers.insert(peer, merge_base);
        Self { merge_base, peers }
    }
}
diff --git a/librad/t/src/integration/scenario.rs b/librad/t/src/integration/scenario.rs
index 9bfdd2ad..c47720a0 100644
--- a/librad/t/src/integration/scenario.rs
+++ b/librad/t/src/integration/scenario.rs
@@ -5,6 +5,7 @@

mod collaboration;
mod collaborative_objects;
mod default_branch_head;
mod menage;
mod passive_replication;
#[cfg(feature = "replication-v3")]
diff --git a/librad/t/src/integration/scenario/default_branch_head.rs b/librad/t/src/integration/scenario/default_branch_head.rs
new file mode 100644
index 00000000..07399eed
--- /dev/null
+++ b/librad/t/src/integration/scenario/default_branch_head.rs
@@ -0,0 +1,338 @@
// Copyright © 2019-2020 The Radicle Foundation <hello@radicle.foundation>
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

use std::{convert::TryFrom, ops::Index as _};

use tempfile::tempdir;

use git_ref_format::{lit, name, Namespaced, Qualified, RefString};
use it_helpers::{
    fixed::{TestPerson, TestProject},
    testnet::{self, RunningTestPeer},
    working_copy::{WorkingCopy, WorkingRemote as Remote},
};
use librad::git::{
    identities::{self, local, project::heads},
    storage::ReadOnlyStorage,
};
use link_identities::payload;
use test_helpers::logging;

fn config() -> testnet::Config {
    testnet::Config {
        num_peers: nonzero!(2usize),
        min_connected: 2,
        bootstrap: testnet::Bootstrap::from_env(),
    }
}

/// This test checks that the logic of `librad::git::identities::project::heads`
/// is correct. To do this we need to set up various scenarios where the
/// delegates of a project agree or disagree on the default branch of a project.
#[test]
fn default_branch_head() {
    logging::init();

    let net = testnet::run(config()).unwrap();
    net.enter(async {
        // Setup  a testnet with two peers
        let peer1 = net.peers().index(0);
        let peer2 = net.peers().index(1);

        // Create an identity on peer2
        let peer2_id = peer2
            .using_storage::<_, anyhow::Result<TestPerson>>(|s| {
                let person = TestPerson::create(s)?;
                let local = local::load(s, person.owner.urn()).unwrap();
                s.config()?.set_user(local)?;
                Ok(person)
            })
            .await
            .unwrap()
            .unwrap();

        peer2_id.pull(peer2, peer1).await.unwrap();

        // Create a project on peer1
        let proj = peer1
            .using_storage(|s| {
                TestProject::create_with_payload(
                    s,
                    payload::Project {
                        name: "venus".into(),
                        description: None,
                        default_branch: Some(name::MASTER.to_string().into()),
                    },
                )
            })
            .await
            .unwrap()
            .unwrap();

        // Add peer2 as a maintainer
        proj.maintainers(peer1)
            .add(&peer2_id, peer2)
            .setup()
            .await
            .unwrap();

        //// Okay, now we have a running testnet with two Peers, each of which has a
        //// `Person` who is a delegate on the `TestProject`

        // Create a basic history which contains two commits, one from peer1 and one
        // from peer2
        //
        // * Create a commit in peer 1
        // * pull to peer2
        // * create a new commit on top of the original commit in peer2
        // * pull peer2 back to peer1
        // * in peer1 fast forward and push
        //
        // At this point both peers should have a history like
        //
        //     peer1 commit
        //           ↓
        //     peer2 commit
        let tmp = tempdir().unwrap();
        let tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            working_copy1
                .commit("peer 1 initial", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();
            proj.pull(peer1, peer2).await.unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            let tip = working_copy2
                .commit("peer 2 initial", mastor.clone())
                .unwrap();
            working_copy2.push().unwrap();
            proj.pull(peer2, peer1).await.unwrap();

            working_copy1.fetch(Remote::Peer(peer2.peer_id())).unwrap();
            working_copy1
                .fast_forward_to(Remote::Peer(peer2.peer_id()), name::MASTER)
                .unwrap();
            working_copy1.push().unwrap();
            tip
        };

        let default_branch = branch_head(peer1, &proj).await.unwrap();
        // The two peers should have the same view of the default branch
        assert_eq!(
            default_branch,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now update peer1 and push to peer 1s monorepo, we should still get the old
        // tip because peer2 is behind
        let tmp = tempdir().unwrap();
        let new_tip = {
            let mut working_copy1 =
                WorkingCopy::new(&proj, tmp.path().join("peer1"), peer1).unwrap();
            working_copy1
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let tip = working_copy1
                .commit("peer 1 update", mastor.clone())
                .unwrap();
            working_copy1.push().unwrap();

            tip
        };

        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // fast forward peer2 and pull the update back into peer1
        let tmp = tempdir().unwrap();
        proj.pull(peer1, peer2).await.unwrap();
        {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();
            working_copy2
                .create_remote_tracking_branch(Remote::Rad, name::MASTER)
                .unwrap();

            working_copy2.fetch(Remote::Peer(peer1.peer_id())).unwrap();
            working_copy2
                .fast_forward_to(Remote::Peer(peer1.peer_id()), name::MASTER)
                .unwrap();
            working_copy2.push().unwrap();
        }
        proj.pull(peer2, peer1).await.unwrap();

        // Now we should be pointing at the latest tip because both peer1 and peer2
        // agree
        let default_branch_peer1 = branch_head(peer1, &proj).await.unwrap();
        assert_eq!(
            default_branch_peer1,
            identities::project::heads::DefaultBranchHead::Head {
                target: new_tip,
                branch: name::MASTER.to_owned(),
            }
        );

        // now create an alternate commit on peer2 and sync with peer1, on peer1 we
        // should get a fork
        let tmp = tempdir().unwrap();
        let forked_tip = {
            let mut working_copy2 =
                WorkingCopy::new(&proj, tmp.path().join("peer2"), peer2).unwrap();

            let mastor = Qualified::from(lit::refs_heads(name::MASTER));
            let forked_tip = working_copy2.commit("peer 2 fork", mastor.clone()).unwrap();
            working_copy2.push().unwrap();

            forked_tip