~radicle-link/dev

radicle-link: add integration test for bins package v1 PROPOSED

This is the first version patch to add an integration test for bins package. The test 
is following the steps in https://github.com/alexjg/linkd-playground. This version covers the steps
before we git push the local changes and do git clone.

This test is not embedded in CI workflow yet. Run `cargo test` (recommend `cargo test -- --nocapture`)
under bins to execute the test.


Han Xu (1):
  basic integration test for bins

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 ++
 bins/tests/integration_test.rs | 392 +++++++++++++++++++++++++++++++++
 4 files changed, 435 insertions(+)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
#820143 nixos-latest.yml failed
radicle-link/patches/nixos-latest.yml: FAILED in 23m57s

[add integration test for bins package][0] from [Han Xu][1]

[0]: https://lists.sr.ht/~radicle-link/dev/patches/34576
[1]: mailto:keepsimple@gmail.com

✗ #820143 FAILED radicle-link/patches/nixos-latest.yml https://builds.sr.ht/~radicle-link/job/820143
This is the 3rd version (re-rolling) patch to add an integration test for `bins` package.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

The only non-test code change is in git version checks. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version checked against.

This patch also addresses the comments from the 2nd version, and I opted to re-rolling
instead of adding new commits.

Han Xu (1):
  Add integration test for bins and fix git version checks

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 449 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 500 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
This patch is an update on the previous v3 patch, with the following changes:

- Added a check for refs/heads/master between peers. This is because I noticed
a bug where `git push` from peer1 only pushs an older commit as `heads/master`
to the seed node.

- Found the root cause of the above bug and included a fix in 
cli/gitd-lib/src/hooks.rs. The root cause is that currently a peer does 
request_pull before updating the signed refs, hence the request pull only
has an old signed refs for heads/master.

Han Xu (3):
  Add integration test for bins and fix git version checks
  check peer1 and peer2 ref/heads/master
  fix post_receive to send updated refs in requst_pull

 bins/Cargo.lock                |  29 ++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 468 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 7 files changed, 530 insertions(+), 10 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
This is the 5th version (re-rolling) patch to add an integration test for `bins` package.

It is built on top of version 4 patch and addressed review comments (see new commits):
 - created a new macro to simplify the test code.
 - removed the use of systemd-socket-activate.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

We also found two bugs along the way: 

1. Git version checks are off. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version checked against.

2. `request_pull` to the seed fails to pull the latest commit. The reason being that
the signed_refs for `heads/master` is not the latest. A fix is provided.

Han Xu (5):
  Add integration test for bins and fix git version checks
  check peer1 and peer2 ref/heads/master
  fix post_receive to send updated refs in requst_pull
  Use macro to refactor run_lnk
  remove the use of systemd-socket-activate

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 417 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 7 files changed, 479 insertions(+), 10 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
So I am trying to get some collaboration on testing going between testing CLI type binaries across

Is there a reason none of these could be used ?
https://crates.io/crates/trycmd
https://crates.io/crates/term-transcript
out of date: https://github.com/rust-rspec/rspec
https://crates.io/crates/trybuild
https://nexte.st/book/leaky-tests.html

compiletest seems to have detached yay https://github.com/Manishearth/compiletest-rs
and ofc insta https://crates.io/crates/insta
https://crates.io/crates/snapbox
https://crates.io/crates/assert_cmd, https://crates.io/crates/assert_fs
https://crates.io/crates/duct
https://crates.io/crates/rexpect
https://crates.io/crates/dir-diff

I use insta a lot myself
Rust idiomatic way is to use small crates from the ecosystem

If we are still going ahead of using Rust to do integration tests.
------- Original Message -------
On Friday, August 26th, 2022 at 1:44 AM, Alex Good <alex@memoryandthought.me> wrote:
Next
This is the 6th version (re-rolling) patch to add an integration test for `bins` package.

It is built on top of version 5 patch and addressed review comments:
 - refactor the code styling.
 - squash the patch into two commits only.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

We also found two bugs along the way: 

1. Git version checks are off. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version checked against.

2. `request_pull` to the seed fails to pull the latest commit. The reason being that
the signed_refs for `heads/master` is not the latest. A fix is provided.


Han Xu (2):
  Add integration test for bins and fix git version checks
  fix post_receive to send updated refs in request_pull

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 7 files changed, 518 insertions(+), 10 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
This is the 8th version (re-rolling) patch to add an integration test for `bins` package.
It was refactored the patch into 3 commits so that it is clearer to understand the changes.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

We also found two bugs along the way: 

1. Git version checks are off. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version check in ls-refs.
2. `request_pull` to the seed fails to pull the latest commit. The reason being that
the signed_refs for `heads/master` is not the latest. A fix is provided.


Han Xu (3):
  fix post_receive to send updated refs in request_pull
  Add integration test for bins
  fix Git version check for ls-refs

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 514 insertions(+), 9 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
This is the 9th version (re-rolling) patch to add an integration test for `bins` package.
The only change from the 8th version is to add sign-off lines in commit messages.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

We also found two bugs along the way:

1. Git version checks are off. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version check in ls-refs.

2. `request_pull` to the seed fails to pull the latest commit. The reason being that
the signed_refs for `heads/master` is not the latest. A fix is provided.

Han Xu (3):
  fix post_receive to send updated refs in request_pull
  Add integration test for bins
  fix Git version check for ls-refs

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 514 insertions(+), 9 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.1 (Apple Git-133)
No, 


As a developer -

It is our job to choose the right tool for the job -

Re-using is one of the core competencies of ours -

I don't see even attempts to evaluate the options I gave you.
Don't expect anyone else to effectively do your work after for you.
Correcting mistakes is one but not even looking at the list and engaging in technical merits is another.

It is a massive stress having to clean up after someone when they were told of options to do it properly without extra effort.

We don't really need a strategy to do the right thing (tm) 


This fallacy goes two ways - 


Now we are adding code that should not exists without strategy -

We are now adding some low level code that is wholly redundant to test other low level code.. 


I mean what are we testing here.. the tests or the code ?

Like why are we dealing with pty's and the logistics of it ?

When e.g. trycmd - first on the list - does that in minimal manner ? 

Think of it like this:

   Every line of code is liability.

Just pick the right tool for the job (tm)

------- Original Message -------
On Friday, August 26th, 2022 at 7:15 AM, Han <keepsimple@gmail.com> wrote:
> On Thu, Aug 25, 2022 at 9:10 AM MissM_signed@protonmail.ch wrote:
Could you publish the v6 tag, please? :)
This is the 7th version (re-rolling) patch to add an integration test for `bins` package.
It has been rebased on master branch 622c1bcd59a6ce584f957ffe6b874b2af0b207fd.

The test case scenario follows the steps in https://github.com/alexjg/linkd-playground ,
except it is in Rust now and we can just run `cargo test`. To see a more helpful output,
please run `cargo test -- --nocapture` from the `bins/tests` directory.

We also found two bugs along the way: 

1. Git version checks are off. In our testing, it shows that
git 2.25.1 on Ubuntu 20.04 and git 2.30.2 on Debian 11 do not need the workaround that
adds "namespaces" manually. Hence fixing the git version checked against.

2. `request_pull` to the seed fails to pull the latest commit. The reason being that
the signed_refs for `heads/master` is not the latest. A fix is provided.

Han Xu (2):
  Add integration test for bins and fix git version checks
  fix post_receive to send updated refs in request_pull

 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 cli/gitd-lib/src/hooks.rs      |  19 +-
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 7 files changed, 518 insertions(+), 10 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

-- 
2.32.0 (Apple Git-132)
Here it is: https://github.com/keepsimple1/radicle-link/releases/tag/patches%2Fintegration-test-bins%2Fv6

Sorry I forgot it.

Han
On Tue, Aug 30, 2022 at 1:50 AM Fintan Halpenny
<fintan.halpenny@gmail.com> wrote:
Could you publish your tag? :D
Sorry, I always forgot to publish the tag. Here it is:
https://github.com/keepsimple1/radicle-link/releases/tag/patches%2Fintegration-test-bins%2Fv7


On Thu, Sep 1, 2022 at 2:38 AM Fintan Halpenny
<fintan.halpenny@gmail.com> wrote:
We have changed the Git version checks in both
link-git/src/protocol/ls.rs and link-git/src/protocol/fetch.rs. I
reverted the fetch.rs change and now link-git tests are passing,
without updating Git in the system. (My system has Git 2.25.1). Our
new Integration Test uses `ls.rs` to do `ls-refs` to obtain refs path,
so that without the fetch.rs change, Integration Test still passes.
Given these observations, I am changing only the `ls.rs` for Git
version check.

I will submit a new patch version, and separate out the Git version
check in its own commit so it is easier to see.

thanks
Han

On Thu, Sep 1, 2022 at 4:23 AM Fintan Halpenny
<fintan.halpenny@gmail.com> wrote:
I have submitted the new (8th) version of the patch, also published
the tag : https://github.com/keepsimple1/radicle-link/releases/tag/patches%2Fintegration-test-bins%2Fv8

Cheers,
Han

On Thu, Sep 1, 2022 at 9:47 AM Han <keepsimple@gmail.com> wrote:
Also, I noticed that your commits are missing your sign-off, ie. name
and email, as per our DCO policy[0].

[0]: https://github.com/radicle-dev/radicle-link/blob/master/CONTRIBUTING.md#certificate-of-origin


LGTM! Can you publish the tag? :)
sorry I never remember to publish a tag ;-) Here it is:

https://github.com/keepsimple1/radicle-link/releases/tag/patches%2Fintegration-test-bins%2Fv9

On Tue, Sep 6, 2022 at 2:24 AM Fintan Halpenny
<fintan.halpenny@gmail.com> wrote:
Export patchset (mbox)
How do I use this?

Copy & paste the following snippet into your terminal to import this patchset into git:

curl -s https://lists.sr.ht/~radicle-link/dev/patches/34576/mbox | git am -3
Learn more about email & git

Re: [PATCH radicle-link v5 0/5] Add integration test for bins and fix bugs found Export this patch

I have some stylistic suggestions for test file below.

I think you can also squash the changes to that same file into a
single commit. So, the patch should look like the addition of the
integration_test.rs file and the fix to the signed_refs update.
---
 bins/tests/integration_test.rs | 165 ++++++++++++++++++++-------------
 1 file changed, 100 insertions(+), 65 deletions(-)

diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
index d245748d..91d4d423 100644
--- a/bins/tests/integration_test.rs
@@ -34,59 +34,79 @@ fn two_peers_and_a_seed() {
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let cmd = LnkCmd::ProfileCreate;
    run_lnk!(&cmd, peer1_home, passphrase);
    run_lnk!(&cmd, peer2_home, passphrase);
    run_lnk!(&cmd, seed_home, passphrase);
    {
        println!("\n== create lnk homes for two peers and one seed ==\n");
        let cmd = LnkCmd::ProfileCreate;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let cmd = LnkCmd::ProfileSshAdd;
    run_lnk!(&cmd, peer1_home, passphrase);
    run_lnk!(&cmd, peer2_home, passphrase);
    run_lnk!(&cmd, seed_home, passphrase);

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let cmd = LnkCmd::IdPersonCreate(peer1_name);
    let output = run_lnk!(&cmd, peer1_home, passphrase);
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let cmd = LnkCmd::IdLocalSet(urn1);
    run_lnk!(&cmd, peer1_home, passphrase);

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let cmd = LnkCmd::IdPersonCreate(peer2_name);
    let output = run_lnk!(&cmd, peer2_home, passphrase);
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let cmd = LnkCmd::IdLocalSet(urn2);
    run_lnk!(&cmd, peer2_home, passphrase);
    {
        println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
        let cmd = LnkCmd::ProfileSshAdd;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let cmd = LnkCmd::IdProjectCreate(peer1_proj.clone());
    let output = run_lnk!(&cmd, peer1_home, passphrase);
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);
    {
        println!("\n== Creating local link 1 identity ==\n");
        let name = "sockpuppet1".to_string();
        let cmd = LnkCmd::IdPersonCreate { name };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn: urn };
        run_lnk!(&cmd, peer1_home, passphrase);
    }

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let cmd = LnkCmd::ProfilePeer;
    let seed_peer_id = run_lnk!(&cmd, seed_home, passphrase);
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);
    {
        println!("\n== Creating local link 2 identity ==\n");
        let name = "sockpuppet2".to_string();
        let cmd = LnkCmd::IdPersonCreate { name };
        let output = run_lnk!(&cmd, peer2_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn };
        run_lnk!(&cmd, peer2_home, passphrase);
    }

    let urn = {
        println!("\n== Create a local repository ==\n");
        let peer1_proj = format!("peer1_proj_{}", timestamp());
        let cmd = LnkCmd::IdProjectCreate {
            name: peer1_proj.clone(),
        };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let proj_urn = v["urn"].as_str().unwrap().to_string();
        println!("our project URN: {}", &proj_urn);
        proj_urn
    };

    let cmd = LnkCmd::ProfileGet;
    let peer1_profile = run_lnk!(&cmd, peer1_home, passphrase);
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();
    let seed_endpoint = {
        println!("\n== Add the seed to the local peer seed configs ==\n");
        let cmd = LnkCmd::ProfilePeer;
        let seed_peer_id = run_lnk!(&cmd, seed_home, passphrase);
        format!("{}@127.0.0.1:8799", &seed_peer_id)
    };

    let peer2_profile = run_lnk!(&cmd, peer2_home, passphrase);
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();
    {
        let cmd = LnkCmd::ProfileGet;
        let peer1_profile = run_lnk!(&cmd, peer1_home, passphrase);
        let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
        let mut peer1_f = File::create(peer1_seed).unwrap();
        peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    {
        let cmd = LnkCmd::ProfileGet;
        let peer2_profile = run_lnk!(&cmd, peer2_home, passphrase);
        let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
        let mut peer2_f = File::create(peer2_seed).unwrap();
        peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
@@ -99,7 +119,7 @@ fn two_peers_and_a_seed() {
    let mut gitd = spawn_lnk_gitd(peer1_home, &manifest_path, gitd_addr);

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    env::set_current_dir(&urn).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
@@ -116,7 +136,7 @@ fn two_peers_and_a_seed() {
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@{}/{}.git", gitd_addr, &proj_urn);
    let remote_url = format!("ssh://rad@{}/{}.git", gitd_addr, &urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
@@ -138,11 +158,12 @@ fn two_peers_and_a_seed() {

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let peer2_proj = format!("peer2_proj_{}", timestamp());
    let cmd = LnkCmd::Clone(proj_urn, peer1_peer_id, peer2_proj.clone());
    let cmd = LnkCmd::Clone {
        urn,
        peer_id: peer1_peer_id,
        path: peer2_proj.clone(),
    };
    run_lnk!(&cmd, peer2_home, passphrase);
    if !is_parent {
        return;
    }

    env::set_current_dir(peer2_proj).unwrap();
    let peer2_last_commit = git_last_commit();
@@ -162,10 +183,20 @@ enum LnkCmd {
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
    Clone(String, String, String), // the associated string is "urn", "peer_id", "path"
    IdPersonCreate {
        name: String,
    },
    IdLocalSet {
        urn: String,
    },
    IdProjectCreate {
        name: String,
    },
    Clone {
        urn: String,
        peer_id: String,
        path: String,
    },
}

/// Runs a `lnk` command of `$cmd` using `$lnk_home` as the node home.
@@ -199,12 +230,12 @@ fn process_lnk_output(lnk_home: &str, lnk_process: &mut Master, cmd: &LnkCmd) ->
        // Print the output and decode them if necessary.
        println!("{}: {}", lnk_home, line);
        match cmd {
            LnkCmd::IdPersonCreate(ref _name) => {
            LnkCmd::IdPersonCreate { .. } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
            },
            LnkCmd::IdProjectCreate(ref _name) => {
            LnkCmd::IdProjectCreate { .. } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
@@ -215,7 +246,7 @@ fn process_lnk_output(lnk_home: &str, lnk_process: &mut Master, cmd: &LnkCmd) ->
            LnkCmd::ProfilePeer => {
                output = line; // get the last line for peer id.
            },
            LnkCmd::Clone(ref _urn, ref _peer, ref _path) => {
            LnkCmd::Clone { .. } => {
                output = line;
            },
            _ => {},
@@ -245,7 +276,7 @@ fn start_lnk_cmd(lnk_home: &str, cmd: &LnkCmd) {
        LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
        LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
        LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
        LnkCmd::IdPersonCreate(name) => {
        LnkCmd::IdPersonCreate { name } => {
            let payload = json!({ "name": name });
            lnk_cmd
                .arg("identities")
@@ -255,13 +286,13 @@ fn start_lnk_cmd(lnk_home: &str, cmd: &LnkCmd) {
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::IdLocalSet(urn) => lnk_cmd
        LnkCmd::IdLocalSet { urn } => lnk_cmd
            .arg("identities")
            .arg("local")
            .arg("set")
            .arg("--urn")
            .arg(urn),
        LnkCmd::IdProjectCreate(name) => {
        LnkCmd::IdProjectCreate { name } => {
            let payload = json!({"name": name, "default_branch": "master"});
            let project_path = format!("./{}", name);
            lnk_cmd
@@ -274,7 +305,11 @@ fn start_lnk_cmd(lnk_home: &str, cmd: &LnkCmd) {
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::Clone(urn, peer_id, peer2_proj) => lnk_cmd
        LnkCmd::Clone {
            urn,
            peer_id,
            path: peer2_proj,
        } => lnk_cmd
            .arg("clone")
            .arg("--urn")
            .arg(urn)
-- 
2.31.1

[PATCH radicle-link v1 1/1] basic integration test for bins Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 ++
 bins/tests/integration_test.rs | 392 +++++++++++++++++++++++++++++++++
 4 files changed, 435 insertions(+)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..4ff581a0
--- /dev/null
@@ -0,0 +1,392 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
#[test]
fn happy_path_to_push_changes() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer1_name), peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn1), peer1_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer2_name), peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn2), peer2_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let (is_parent, output) = run_lnk(
        LnkCmd::IdProjectCreate(peer1_proj.clone()),
        peer1_home,
        passphrase,
    );
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let (is_parent, seed_peer_id) = run_lnk(LnkCmd::ProfilePeer, seed_home, passphrase);
    if !is_parent {
        return;
    }
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);

    let (is_parent, peer1_profile) = run_lnk(LnkCmd::ProfileGet, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();

    let (is_parent, peer2_profile) = run_lnk(LnkCmd::ProfileGet, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let (is_parent, peer1_peer_id) = run_lnk(LnkCmd::ProfilePeer, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let mut lnk_gitd = spawn_lnk_gitd(peer1_home, &manifest_path, &peer1_peer_id);

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@127.0.0.1:9987/{}.git", &proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    linkd.kill().ok();
    lnk_gitd.kill().ok();
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
}

/// Runs a `cmd` for `lnk_home`. Rebuilds `lnk` if necessary.
/// Return.0: true if this is the parent (i.e. test) process,
///           false if this is the child (i.e. lnk) process.
/// Return.1: an output that depends on the `cmd`.
fn run_lnk(cmd: LnkCmd, lnk_home: &str, passphrase: &[u8]) -> (bool, String) {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        // Input the passphrase if necessary.
        match cmd {
            LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                parent.write_all(passphrase).unwrap();
                println!("{}: wrote passphase", lnk_home);
            },
            _ => {},
        }

        // Print the output and decode them if necessary.
        let buf_reader = BufReader::new(parent);
        let mut output = String::new();
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("{}: {}", lnk_home, line);

            match cmd {
                LnkCmd::IdPersonCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::IdProjectCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::ProfileGet => {
                    output = line; // get the last line for profile id.
                },
                LnkCmd::ProfilePeer => {
                    output = line; // get the last line for peer id.
                },
                _ => {},
            }
        }

        (true, output)
    } else {
        // Child process is to run `lnk`.
        let manifest_path = manifest_path();

        // cargo run \
        // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
        // -p lnk -- "$@"
        let mut lnk_cmd = Command::new("cargo");
        lnk_cmd
            .env("LNK_HOME", lnk_home)
            .arg("run")
            .arg("--manifest-path")
            .arg(manifest_path)
            .arg("-p")
            .arg("lnk")
            .arg("--");
        let full_cmd = match cmd {
            LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
            LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
            LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
            LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
            LnkCmd::IdPersonCreate(name) => {
                let payload = json!({ "name": name });
                lnk_cmd
                    .arg("identities")
                    .arg("person")
                    .arg("create")
                    .arg("new")
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::IdLocalSet(urn) => lnk_cmd
                .arg("identities")
                .arg("local")
                .arg("set")
                .arg("--urn")
                .arg(urn),
            LnkCmd::IdProjectCreate(name) => {
                let payload = json!({"name": name, "default_branch": "master"});
                let project_path = format!("./{}", name);
                lnk_cmd
                    .arg("identities")
                    .arg("project")
                    .arg("create")
                    .arg("new")
                    .arg("--path")
                    .arg(project_path)
                    .arg("--payload")
                    .arg(payload.to_string())
            },
        };
        full_cmd.status().expect("lnk cmd failed:");

        (false, String::new())
    }
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let child = Command::new("cargo")
        .arg("run")
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .arg("--")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, peer_id: &str) -> Child {
    let log_name = format!("lnk-gitd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let port = "9987";
    let xdg_runtime_dir = env!("XDG_RUNTIME_DIR");
    let rpc_socket = format!("{}/link-peer-{}-rpc.socket", xdg_runtime_dir, peer_id);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg("./target")
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .output()
        .expect("cargo build lnk-gitd failed");

    let child = Command::new("systemd-socket-activate")
        .arg("-l")
        .arg(port)
        .arg("--fdname=ssh")
        .arg("-E")
        .arg("SSH_AUTH_SOCK")
        .arg("-E")
        .arg("RUST_BACKTRACE")
        .arg("./target/debug/lnk-gitd")
        .arg(lnk_home)
        .arg("--linkd-rpc-socket")
        .arg(rpc_socket)
        .arg("--push-seeds")
        .arg("--fetch-seeds")
        .arg("--linger-timeout")
        .arg("10000")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("lnk-gitd failed to start");
    println!("lnk-gitd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));

    child
}

/// Returns true if this is the parent process,
/// returns false if this is the child process.
fn _run_git_push() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        parent.write_all(yes).unwrap();

        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
        }

        true
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        false
    }
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
-- 
2.32.0 (Apple Git-132)
radicle-link/patches/nixos-latest.yml: FAILED in 23m57s

[add integration test for bins package][0] from [Han Xu][1]

[0]: https://lists.sr.ht/~radicle-link/dev/patches/34576
[1]: mailto:keepsimple@gmail.com

✗ #820143 FAILED radicle-link/patches/nixos-latest.yml https://builds.sr.ht/~radicle-link/job/820143

[PATCH radicle-link v2 1/1] Add integration test for bins Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 458 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   2 +-
 link-git/src/protocol/ls.rs    |   2 +-
 6 files changed, 503 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..d6d3b4ae
--- /dev/null
@@ -0,0 +1,458 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer1_name), peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn1), peer1_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer2_name), peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn2), peer2_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let (is_parent, output) = run_lnk(
        LnkCmd::IdProjectCreate(peer1_proj.clone()),
        peer1_home,
        passphrase,
    );
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let (is_parent, seed_peer_id) = run_lnk(LnkCmd::ProfilePeer, seed_home, passphrase);
    if !is_parent {
        return;
    }
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);

    let (is_parent, peer1_profile) = run_lnk(LnkCmd::ProfileGet, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();

    let (is_parent, peer2_profile) = run_lnk(LnkCmd::ProfileGet, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let (is_parent, peer1_peer_id) = run_lnk(LnkCmd::ProfilePeer, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let is_parent = spawn_lnk_gitd(peer1_home, &manifest_path, &peer1_peer_id);
    if !is_parent {
        return;
    }

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@127.0.0.1:9987/{}.git", &proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    let is_parent = run_git_push();
    if !is_parent {
        return;
    }

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let (is_parent, _) = run_lnk(
        LnkCmd::Clone(proj_urn, peer1_peer_id),
        peer2_home,
        passphrase,
    );
    if !is_parent {
        return;
    }

    println!("\n== Kill linkd (seed) ==\n");

    linkd.kill().ok();
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
    Clone(String, String),   // the associated string is "urn", "peer_id"
}

/// Runs a `cmd` for `lnk_home`. Rebuilds `lnk` if necessary.
/// Return.0: true if this is the parent (i.e. test) process,
///           false if this is the child (i.e. lnk) process.
/// Return.1: an output that depends on the `cmd`.
fn run_lnk(cmd: LnkCmd, lnk_home: &str, passphrase: &[u8]) -> (bool, String) {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        // Input the passphrase if necessary.
        match cmd {
            LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                parent.write_all(passphrase).unwrap();
                println!("{}: wrote passphase", lnk_home);
            },
            _ => {},
        }

        // Print the output and decode them if necessary.
        let buf_reader = BufReader::new(parent);
        let mut output = String::new();
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("{}: {}", lnk_home, line);

            match cmd {
                LnkCmd::IdPersonCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::IdProjectCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::ProfileGet => {
                    output = line; // get the last line for profile id.
                },
                LnkCmd::ProfilePeer => {
                    output = line; // get the last line for peer id.
                },
                LnkCmd::Clone(ref _urn, ref _peer) => {
                    output = line;
                },
                _ => {},
            }
        }

        (true, output)
    } else {
        // Child process is to run `lnk`.
        let manifest_path = manifest_path();

        // cargo run \
        // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
        // -p lnk -- "$@"
        let mut lnk_cmd = Command::new("cargo");
        lnk_cmd
            .env("LNK_HOME", lnk_home)
            .arg("run")
            .arg("--manifest-path")
            .arg(manifest_path)
            .arg("-p")
            .arg("lnk")
            .arg("--");
        let full_cmd = match cmd {
            LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
            LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
            LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
            LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
            LnkCmd::IdPersonCreate(name) => {
                let payload = json!({ "name": name });
                lnk_cmd
                    .arg("identities")
                    .arg("person")
                    .arg("create")
                    .arg("new")
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::IdLocalSet(urn) => lnk_cmd
                .arg("identities")
                .arg("local")
                .arg("set")
                .arg("--urn")
                .arg(urn),
            LnkCmd::IdProjectCreate(name) => {
                let payload = json!({"name": name, "default_branch": "master"});
                let project_path = format!("./{}", name);
                lnk_cmd
                    .arg("identities")
                    .arg("project")
                    .arg("create")
                    .arg("new")
                    .arg("--path")
                    .arg(project_path)
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::Clone(urn, peer_id) => {
                let peer2_proj = format!("peer2_proj_{}", timestamp());
                lnk_cmd
                    .arg("clone")
                    .arg("--urn")
                    .arg(urn)
                    .arg("--path")
                    .arg(peer2_proj)
                    .arg("--peer")
                    .arg(peer_id)
            },
        };
        full_cmd.status().expect("lnk cmd failed:");

        (false, String::new())
    }
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, peer_id: &str) -> bool {
    let port = "9987";
    let xdg_runtime_dir = env!("XDG_RUNTIME_DIR");
    let rpc_socket = format!("{}/link-peer-{}-rpc.socket", xdg_runtime_dir, peer_id);
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

    let fork = Fork::from_ptmx().unwrap();

    if let Some(_parent) = fork.is_parent().ok() {
        println!("started lnk-gitd");
        true
    } else {
        Command::new("systemd-socket-activate")
            .arg("-l")
            .arg(port)
            .arg("--fdname=ssh")
            .arg("-E")
            .arg("SSH_AUTH_SOCK")
            .arg("-E")
            .arg("RUST_BACKTRACE")
            .arg(&exec_path)
            .arg(lnk_home)
            .arg("--linkd-rpc-socket")
            .arg(rpc_socket)
            .arg("--push-seeds")
            .arg("--fetch-seeds")
            .arg("--linger-timeout")
            .arg("10000")
            .output()
            .expect("lnk-gitd failed to start");
        false
    }
}

/// Returns true if this is the parent process,
/// returns false if this is the child process.
fn run_git_push() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        true
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        false
    }
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..562e433d 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -39,7 +39,7 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..9dafa538 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -24,7 +24,7 @@ use super::{remote_git_version, transport};
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v3 1/1] Add integration test for bins and fix git version checks Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 449 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 500 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..770f1b5e
--- /dev/null
@@ -0,0 +1,449 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer1_name), peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn1), peer1_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer2_name), peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn2), peer2_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let (is_parent, output) = run_lnk(
        LnkCmd::IdProjectCreate(peer1_proj.clone()),
        peer1_home,
        passphrase,
    );
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let (is_parent, seed_peer_id) = run_lnk(LnkCmd::ProfilePeer, seed_home, passphrase);
    if !is_parent {
        return;
    }
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);

    let (is_parent, peer1_profile) = run_lnk(LnkCmd::ProfileGet, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();

    let (is_parent, peer2_profile) = run_lnk(LnkCmd::ProfileGet, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let (is_parent, peer1_peer_id) = run_lnk(LnkCmd::ProfilePeer, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    spawn_lnk_gitd(peer1_home, &manifest_path, &peer1_peer_id);

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@127.0.0.1:9987/{}.git", &proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    let is_parent = run_git_push();
    if !is_parent {
        return;
    }

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let (is_parent, _) = run_lnk(
        LnkCmd::Clone(proj_urn, peer1_peer_id),
        peer2_home,
        passphrase,
    );
    if !is_parent {
        return;
    }

    println!("\n== Kill linkd (seed) ==\n");

    linkd.kill().ok();
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
    Clone(String, String),   // the associated string is "urn", "peer_id"
}

/// Runs a `cmd` for `lnk_home`. Rebuilds `lnk` if necessary.
/// Return.0: true if this is the parent (i.e. test) process,
///           false if this is the child (i.e. lnk) process.
/// Return.1: an output that depends on the `cmd`.
fn run_lnk(cmd: LnkCmd, lnk_home: &str, passphrase: &[u8]) -> (bool, String) {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        // Input the passphrase if necessary.
        match cmd {
            LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                parent.write_all(passphrase).unwrap();
                println!("{}: wrote passphase", lnk_home);
            },
            _ => {},
        }

        // Print the output and decode them if necessary.
        let buf_reader = BufReader::new(parent);
        let mut output = String::new();
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("{}: {}", lnk_home, line);

            match cmd {
                LnkCmd::IdPersonCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::IdProjectCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::ProfileGet => {
                    output = line; // get the last line for profile id.
                },
                LnkCmd::ProfilePeer => {
                    output = line; // get the last line for peer id.
                },
                LnkCmd::Clone(ref _urn, ref _peer) => {
                    output = line;
                },
                _ => {},
            }
        }

        (true, output)
    } else {
        // Child process is to run `lnk`.
        let manifest_path = manifest_path();

        // cargo run \
        // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
        // -p lnk -- "$@"
        let mut lnk_cmd = Command::new("cargo");
        lnk_cmd
            .env("LNK_HOME", lnk_home)
            .arg("run")
            .arg("--manifest-path")
            .arg(manifest_path)
            .arg("-p")
            .arg("lnk")
            .arg("--");
        let full_cmd = match cmd {
            LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
            LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
            LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
            LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
            LnkCmd::IdPersonCreate(name) => {
                let payload = json!({ "name": name });
                lnk_cmd
                    .arg("identities")
                    .arg("person")
                    .arg("create")
                    .arg("new")
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::IdLocalSet(urn) => lnk_cmd
                .arg("identities")
                .arg("local")
                .arg("set")
                .arg("--urn")
                .arg(urn),
            LnkCmd::IdProjectCreate(name) => {
                let payload = json!({"name": name, "default_branch": "master"});
                let project_path = format!("./{}", name);
                lnk_cmd
                    .arg("identities")
                    .arg("project")
                    .arg("create")
                    .arg("new")
                    .arg("--path")
                    .arg(project_path)
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::Clone(urn, peer_id) => {
                let peer2_proj = format!("peer2_proj_{}", timestamp());
                lnk_cmd
                    .arg("clone")
                    .arg("--urn")
                    .arg(urn)
                    .arg("--path")
                    .arg(peer2_proj)
                    .arg("--peer")
                    .arg(peer_id)
            },
        };
        full_cmd.status().expect("lnk cmd failed:");

        (false, String::new())
    }
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, peer_id: &str) {
    let port = "9987";
    let xdg_runtime_dir = env!("XDG_RUNTIME_DIR");
    let rpc_socket = format!("{}/link-peer-{}-rpc.socket", xdg_runtime_dir, peer_id);
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

        Command::new("systemd-socket-activate")
            .arg("-l")
            .arg(port)
            .arg("--fdname=ssh")
            .arg("-E")
            .arg("SSH_AUTH_SOCK")
            .arg("-E")
            .arg("RUST_BACKTRACE")
            .arg(&exec_path)
            .arg(lnk_home)
            .arg("--linkd-rpc-socket")
            .arg(rpc_socket)
            .arg("--push-seeds")
            .arg("--fetch-seeds")
            .arg("--linger-timeout")
            .arg("10000")
            .spawn()
            .expect("lnk-gitd failed to start");
            println!("started lnk-gitd");

}

/// Returns true if this is the parent process,
/// returns false if this is the child process.
fn run_git_push() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        true
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        false
    }
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..98b92b51 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -38,8 +38,11 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
//
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..de95b4a3 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -22,9 +22,12 @@ use super::{remote_git_version, transport};
// Work around `git-upload-pack` not handling namespaces properly
//
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v4 1/3] Add integration test for bins and fix git version checks Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 449 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 500 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..770f1b5e
--- /dev/null
@@ -0,0 +1,449 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer1_name), peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn1), peer1_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer2_name), peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn2), peer2_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let (is_parent, output) = run_lnk(
        LnkCmd::IdProjectCreate(peer1_proj.clone()),
        peer1_home,
        passphrase,
    );
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let (is_parent, seed_peer_id) = run_lnk(LnkCmd::ProfilePeer, seed_home, passphrase);
    if !is_parent {
        return;
    }
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);

    let (is_parent, peer1_profile) = run_lnk(LnkCmd::ProfileGet, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();

    let (is_parent, peer2_profile) = run_lnk(LnkCmd::ProfileGet, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let (is_parent, peer1_peer_id) = run_lnk(LnkCmd::ProfilePeer, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    spawn_lnk_gitd(peer1_home, &manifest_path, &peer1_peer_id);

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@127.0.0.1:9987/{}.git", &proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    let is_parent = run_git_push();
    if !is_parent {
        return;
    }

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let (is_parent, _) = run_lnk(
        LnkCmd::Clone(proj_urn, peer1_peer_id),
        peer2_home,
        passphrase,
    );
    if !is_parent {
        return;
    }

    println!("\n== Kill linkd (seed) ==\n");

    linkd.kill().ok();
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
    Clone(String, String),   // the associated string is "urn", "peer_id"
}

/// Runs a `cmd` for `lnk_home`. Rebuilds `lnk` if necessary.
/// Return.0: true if this is the parent (i.e. test) process,
///           false if this is the child (i.e. lnk) process.
/// Return.1: an output that depends on the `cmd`.
fn run_lnk(cmd: LnkCmd, lnk_home: &str, passphrase: &[u8]) -> (bool, String) {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        // Input the passphrase if necessary.
        match cmd {
            LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                parent.write_all(passphrase).unwrap();
                println!("{}: wrote passphase", lnk_home);
            },
            _ => {},
        }

        // Print the output and decode them if necessary.
        let buf_reader = BufReader::new(parent);
        let mut output = String::new();
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("{}: {}", lnk_home, line);

            match cmd {
                LnkCmd::IdPersonCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::IdProjectCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::ProfileGet => {
                    output = line; // get the last line for profile id.
                },
                LnkCmd::ProfilePeer => {
                    output = line; // get the last line for peer id.
                },
                LnkCmd::Clone(ref _urn, ref _peer) => {
                    output = line;
                },
                _ => {},
            }
        }

        (true, output)
    } else {
        // Child process is to run `lnk`.
        let manifest_path = manifest_path();

        // cargo run \
        // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
        // -p lnk -- "$@"
        let mut lnk_cmd = Command::new("cargo");
        lnk_cmd
            .env("LNK_HOME", lnk_home)
            .arg("run")
            .arg("--manifest-path")
            .arg(manifest_path)
            .arg("-p")
            .arg("lnk")
            .arg("--");
        let full_cmd = match cmd {
            LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
            LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
            LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
            LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
            LnkCmd::IdPersonCreate(name) => {
                let payload = json!({ "name": name });
                lnk_cmd
                    .arg("identities")
                    .arg("person")
                    .arg("create")
                    .arg("new")
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::IdLocalSet(urn) => lnk_cmd
                .arg("identities")
                .arg("local")
                .arg("set")
                .arg("--urn")
                .arg(urn),
            LnkCmd::IdProjectCreate(name) => {
                let payload = json!({"name": name, "default_branch": "master"});
                let project_path = format!("./{}", name);
                lnk_cmd
                    .arg("identities")
                    .arg("project")
                    .arg("create")
                    .arg("new")
                    .arg("--path")
                    .arg(project_path)
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::Clone(urn, peer_id) => {
                let peer2_proj = format!("peer2_proj_{}", timestamp());
                lnk_cmd
                    .arg("clone")
                    .arg("--urn")
                    .arg(urn)
                    .arg("--path")
                    .arg(peer2_proj)
                    .arg("--peer")
                    .arg(peer_id)
            },
        };
        full_cmd.status().expect("lnk cmd failed:");

        (false, String::new())
    }
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, peer_id: &str) {
    let port = "9987";
    let xdg_runtime_dir = env!("XDG_RUNTIME_DIR");
    let rpc_socket = format!("{}/link-peer-{}-rpc.socket", xdg_runtime_dir, peer_id);
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

        Command::new("systemd-socket-activate")
            .arg("-l")
            .arg(port)
            .arg("--fdname=ssh")
            .arg("-E")
            .arg("SSH_AUTH_SOCK")
            .arg("-E")
            .arg("RUST_BACKTRACE")
            .arg(&exec_path)
            .arg(lnk_home)
            .arg("--linkd-rpc-socket")
            .arg(rpc_socket)
            .arg("--push-seeds")
            .arg("--fetch-seeds")
            .arg("--linger-timeout")
            .arg("10000")
            .spawn()
            .expect("lnk-gitd failed to start");
            println!("started lnk-gitd");

}

/// Returns true if this is the parent process,
/// returns false if this is the child process.
fn run_git_push() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        true
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        false
    }
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..98b92b51 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -38,8 +38,11 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
//
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..de95b4a3 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -22,9 +22,12 @@ use super::{remote_git_version, transport};
// Work around `git-upload-pack` not handling namespaces properly
//
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v5 1/5] Add integration test for bins and fix git version checks Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 449 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 500 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..770f1b5e
--- /dev/null
@@ -0,0 +1,449 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    println!("\n== create lnk homes for two peers and one seed ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileCreate, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let (is_parent, _) = run_lnk(LnkCmd::ProfileSshAdd, seed_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 1 identity ==\n");
    let peer1_name = "sockpuppet1".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer1_name), peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn1 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn1), peer1_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Creating local link 2 identity ==\n");
    let peer2_name = "sockpuppet2".to_string();
    let (is_parent, output) = run_lnk(LnkCmd::IdPersonCreate(peer2_name), peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let urn2 = v["urn"].as_str().unwrap().to_string();
    let (is_parent, _) = run_lnk(LnkCmd::IdLocalSet(urn2), peer2_home, passphrase);
    if !is_parent {
        return;
    }

    println!("\n== Create a local repository ==\n");
    let peer1_proj = format!("peer1_proj_{}", timestamp());
    let (is_parent, output) = run_lnk(
        LnkCmd::IdProjectCreate(peer1_proj.clone()),
        peer1_home,
        passphrase,
    );
    if !is_parent {
        return;
    }
    let v: Value = serde_json::from_str(&output).unwrap();
    let proj_urn = v["urn"].as_str().unwrap().to_string();
    println!("our project URN: {}", &proj_urn);

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let (is_parent, seed_peer_id) = run_lnk(LnkCmd::ProfilePeer, seed_home, passphrase);
    if !is_parent {
        return;
    }
    let seed_endpoint = format!("{}@127.0.0.1:8799", &seed_peer_id);

    let (is_parent, peer1_profile) = run_lnk(LnkCmd::ProfileGet, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
    let mut peer1_f = File::create(peer1_seed).unwrap();
    peer1_f.write_all(seed_endpoint.as_bytes()).unwrap();

    let (is_parent, peer2_profile) = run_lnk(LnkCmd::ProfileGet, peer2_home, passphrase);
    if !is_parent {
        return;
    }
    let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
    let mut peer2_f = File::create(peer2_seed).unwrap();
    peer2_f.write_all(seed_endpoint.as_bytes()).unwrap();

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let (is_parent, peer1_peer_id) = run_lnk(LnkCmd::ProfilePeer, peer1_home, passphrase);
    if !is_parent {
        return;
    }
    spawn_lnk_gitd(peer1_home, &manifest_path, &peer1_peer_id);

    println!("\n== Make some changes in the repo ==\n");
    env::set_current_dir(&peer1_proj).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@127.0.0.1:9987/{}.git", &proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    let is_parent = run_git_push();
    if !is_parent {
        return;
    }

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let (is_parent, _) = run_lnk(
        LnkCmd::Clone(proj_urn, peer1_peer_id),
        peer2_home,
        passphrase,
    );
    if !is_parent {
        return;
    }

    println!("\n== Kill linkd (seed) ==\n");

    linkd.kill().ok();
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate(String),  // the associated string is "the person's name".
    IdLocalSet(String),      // the associated string is "urn".
    IdProjectCreate(String), // the associated string is "the project name".
    Clone(String, String),   // the associated string is "urn", "peer_id"
}

/// Runs a `cmd` for `lnk_home`. Rebuilds `lnk` if necessary.
/// Return.0: true if this is the parent (i.e. test) process,
///           false if this is the child (i.e. lnk) process.
/// Return.1: an output that depends on the `cmd`.
fn run_lnk(cmd: LnkCmd, lnk_home: &str, passphrase: &[u8]) -> (bool, String) {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        // Input the passphrase if necessary.
        match cmd {
            LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                parent.write_all(passphrase).unwrap();
                println!("{}: wrote passphase", lnk_home);
            },
            _ => {},
        }

        // Print the output and decode them if necessary.
        let buf_reader = BufReader::new(parent);
        let mut output = String::new();
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("{}: {}", lnk_home, line);

            match cmd {
                LnkCmd::IdPersonCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::IdProjectCreate(ref _name) => {
                    if line.find("\"urn\":").is_some() {
                        output = line; // get the line with URN.
                    }
                },
                LnkCmd::ProfileGet => {
                    output = line; // get the last line for profile id.
                },
                LnkCmd::ProfilePeer => {
                    output = line; // get the last line for peer id.
                },
                LnkCmd::Clone(ref _urn, ref _peer) => {
                    output = line;
                },
                _ => {},
            }
        }

        (true, output)
    } else {
        // Child process is to run `lnk`.
        let manifest_path = manifest_path();

        // cargo run \
        // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
        // -p lnk -- "$@"
        let mut lnk_cmd = Command::new("cargo");
        lnk_cmd
            .env("LNK_HOME", lnk_home)
            .arg("run")
            .arg("--manifest-path")
            .arg(manifest_path)
            .arg("-p")
            .arg("lnk")
            .arg("--");
        let full_cmd = match cmd {
            LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
            LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
            LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
            LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
            LnkCmd::IdPersonCreate(name) => {
                let payload = json!({ "name": name });
                lnk_cmd
                    .arg("identities")
                    .arg("person")
                    .arg("create")
                    .arg("new")
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::IdLocalSet(urn) => lnk_cmd
                .arg("identities")
                .arg("local")
                .arg("set")
                .arg("--urn")
                .arg(urn),
            LnkCmd::IdProjectCreate(name) => {
                let payload = json!({"name": name, "default_branch": "master"});
                let project_path = format!("./{}", name);
                lnk_cmd
                    .arg("identities")
                    .arg("project")
                    .arg("create")
                    .arg("new")
                    .arg("--path")
                    .arg(project_path)
                    .arg("--payload")
                    .arg(payload.to_string())
            },
            LnkCmd::Clone(urn, peer_id) => {
                let peer2_proj = format!("peer2_proj_{}", timestamp());
                lnk_cmd
                    .arg("clone")
                    .arg("--urn")
                    .arg(urn)
                    .arg("--path")
                    .arg(peer2_proj)
                    .arg("--peer")
                    .arg(peer_id)
            },
        };
        full_cmd.status().expect("lnk cmd failed:");

        (false, String::new())
    }
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, peer_id: &str) {
    let port = "9987";
    let xdg_runtime_dir = env!("XDG_RUNTIME_DIR");
    let rpc_socket = format!("{}/link-peer-{}-rpc.socket", xdg_runtime_dir, peer_id);
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

        Command::new("systemd-socket-activate")
            .arg("-l")
            .arg(port)
            .arg("--fdname=ssh")
            .arg("-E")
            .arg("SSH_AUTH_SOCK")
            .arg("-E")
            .arg("RUST_BACKTRACE")
            .arg(&exec_path)
            .arg(lnk_home)
            .arg("--linkd-rpc-socket")
            .arg(rpc_socket)
            .arg("--push-seeds")
            .arg("--fetch-seeds")
            .arg("--linger-timeout")
            .arg("10000")
            .spawn()
            .expect("lnk-gitd failed to start");
            println!("started lnk-gitd");

}

/// Returns true if this is the parent process,
/// returns false if this is the child process.
fn run_git_push() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        true
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        false
    }
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..98b92b51 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -38,8 +38,11 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
//
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..de95b4a3 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -22,9 +22,12 @@ use super::{remote_git_version, transport};
// Work around `git-upload-pack` not handling namespaces properly
//
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v6 1/2] Add integration test for bins and fix git version checks Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 507 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index d5756144..4d0ea413 100644
--- a/bins/Cargo.lock
@@ -838,6 +838,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.2"
@@ -2945,6 +2956,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3567,6 +3588,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..2d2a4a92
--- /dev/null
@@ -0,0 +1,456 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
///
/// This test only works in Linux at this moment.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    {
        println!("\n== create lnk homes for two peers and one seed ==\n");
        let cmd = LnkCmd::ProfileCreate;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    {
        println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
        let cmd = LnkCmd::ProfileSshAdd;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    {
        println!("\n== Creating local link 1 identity ==\n");
        let peer1_name = "sockpuppet1".to_string();
        let cmd = LnkCmd::IdPersonCreate { name: peer1_name };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn };
        run_lnk!(&cmd, peer1_home, passphrase);
    }

    {
        println!("\n== Creating local link 2 identity ==\n");
        let peer2_name = "sockpuppet2".to_string();
        let cmd = LnkCmd::IdPersonCreate { name: peer2_name };
        let output = run_lnk!(&cmd, peer2_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn };
        run_lnk!(&cmd, peer2_home, passphrase);
    }

    let peer1_proj_dir = format!("peer1_proj_{}", timestamp());
    let peer1_proj_urn = {
        println!("\n== Create a local repository for peer1 ==\n");
        let cmd = LnkCmd::IdProjectCreate {
            name: peer1_proj_dir.clone(),
        };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let proj_urn = v["urn"].as_str().unwrap().to_string();
        println!("our project URN: {}", &proj_urn);
        proj_urn
    };

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let seed_endpoint = {
        let cmd = LnkCmd::ProfilePeer;
        let seed_peer_id = run_lnk!(&cmd, seed_home, passphrase);
        format!("{}@127.0.0.1:8799", &seed_peer_id)
    };

    {
        // Create seed file for peer1
        let cmd = LnkCmd::ProfileGet;
        let peer1_profile = run_lnk!(&cmd, peer1_home, passphrase);
        let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
        let mut peer1_seed_f = File::create(peer1_seed).unwrap();
        peer1_seed_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    {
        // Create seed file for peer2
        let cmd = LnkCmd::ProfileGet;
        let peer2_profile = run_lnk!(&cmd, peer2_home, passphrase);
        let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
        let mut peer2_seed_f = File::create(peer2_seed).unwrap();
        peer2_seed_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let cmd = LnkCmd::ProfilePeer;
    let gitd_addr = "127.0.0.1:9987";
    let peer1_peer_id = run_lnk!(&cmd, peer1_home, passphrase);
    let mut gitd = spawn_lnk_gitd(peer1_home, &manifest_path, gitd_addr);

    println!("\n== Make some changes in the repo: add and commit a test file ==\n");
    env::set_current_dir(&peer1_proj_dir).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@{}/{}.git", gitd_addr, &peer1_proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    if run_git_push_in_child_process() {
        // The child process is done with git push.
        return;
    }

    let peer1_last_commit = git_last_commit();

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let peer2_proj = format!("peer2_proj_{}", timestamp());
    let cmd = LnkCmd::Clone {
        urn: peer1_proj_urn,
        peer_id: peer1_peer_id,
        path: peer2_proj.clone(),
    };
    run_lnk!(&cmd, peer2_home, passphrase);

    env::set_current_dir(peer2_proj).unwrap();
    let peer2_last_commit = git_last_commit();
    println!("\n== peer1 proj last commit: {}", &peer1_last_commit);
    println!("\n== peer2 proj last commit: {}", &peer2_last_commit);

    println!("\n== Cleanup: kill linkd (seed) and gitd (peer1) ==\n");

    linkd.kill().ok();
    gitd.kill().ok();

    assert_eq!(peer1_last_commit, peer2_last_commit);
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate {
        name: String,
    },
    IdLocalSet {
        urn: String,
    },
    IdProjectCreate {
        name: String,
    },
    Clone {
        urn: String,
        peer_id: String,
        path: String,
    },
}

/// Runs a `lnk` command of `$cmd` using `$lnk_home` as the node home.
/// Also support the passphrase input for commands that need it.
#[macro_export]
macro_rules! run_lnk {
    ( $cmd:expr, $lnk_home:ident, $passphrase:ident ) => {{
        let fork = Fork::from_ptmx().unwrap();
        if let Some(mut parent) = fork.is_parent().ok() {
            match $cmd {
                LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                    // Input the passphrase if necessary.
                    parent.write_all($passphrase).unwrap();
                },
                _ => {},
            }
            process_lnk_output($lnk_home, &mut parent, $cmd)
        } else {
            start_lnk_cmd($lnk_home, $cmd);
            return;
        }
    }};
}

fn process_lnk_output(lnk_home: &str, lnk_process: &mut Master, cmd: &LnkCmd) -> String {
    let buf_reader = BufReader::new(lnk_process);
    let mut output = String::new();
    for line in buf_reader.lines() {
        let line = line.unwrap();

        // Print the output and decode them if necessary.
        println!("{}: {}", lnk_home, line);
        match cmd {
            LnkCmd::IdPersonCreate { name: _ } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
            },
            LnkCmd::IdProjectCreate { name: _ } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
            },
            LnkCmd::ProfileGet => {
                output = line; // get the last line for profile id.
            },
            LnkCmd::ProfilePeer => {
                output = line; // get the last line for peer id.
            },
            LnkCmd::Clone {
                urn: _,
                peer_id: _,
                path: _,
            } => {
                output = line;
            },
            _ => {},
        }
    }

    output
}

fn start_lnk_cmd(lnk_home: &str, cmd: &LnkCmd) {
    let manifest_path = manifest_path();

    // cargo run \
    // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
    // -p lnk -- "$@"
    let mut lnk_cmd = Command::new("cargo");
    lnk_cmd
        .env("LNK_HOME", lnk_home)
        .arg("run")
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk")
        .arg("--");
    let full_cmd = match cmd {
        LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
        LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
        LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
        LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
        LnkCmd::IdPersonCreate { name } => {
            let payload = json!({ "name": name });
            lnk_cmd
                .arg("identities")
                .arg("person")
                .arg("create")
                .arg("new")
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::IdLocalSet { urn } => lnk_cmd
            .arg("identities")
            .arg("local")
            .arg("set")
            .arg("--urn")
            .arg(urn),
        LnkCmd::IdProjectCreate { name } => {
            let payload = json!({"name": name, "default_branch": "master"});
            let project_path = format!("./{}", name);
            lnk_cmd
                .arg("identities")
                .arg("project")
                .arg("create")
                .arg("new")
                .arg("--path")
                .arg(project_path)
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::Clone { urn, peer_id, path } => lnk_cmd
            .arg("clone")
            .arg("--urn")
            .arg(urn)
            .arg("--path")
            .arg(path)
            .arg("--peer")
            .arg(peer_id),
    };
    full_cmd.status().expect("lnk cmd failed:");
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, addr: &str) -> Child {
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

    let child = Command::new(&exec_path)
        .arg(lnk_home)
        .arg("--push-seeds")
        .arg("--fetch-seeds")
        .arg("-a")
        .arg(addr)
        .spawn()
        .expect("lnk-gitd failed to start");
    println!("started lnk-gitd");
    child
}

/// Returns true if runs in the forked child process for git push.
/// returns false if runs in the parent process.
fn run_git_push_in_child_process() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        false // This is not the child process.
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        true // This is the child process.
    }
}

fn git_last_commit() -> String {
    let output = Command::new("git")
        .arg("rev-parse")
        .arg("HEAD")
        .output()
        .expect("failed to run git rev-parse");
    String::from_utf8_lossy(&output.stdout).to_string()
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..98b92b51 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -38,8 +38,11 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
//
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..de95b4a3 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -22,9 +22,12 @@ use super::{remote_git_version, transport};
// Work around `git-upload-pack` not handling namespaces properly
//
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v7 1/2] Add integration test for bins and fix git version checks Export this patch

---
 bins/Cargo.lock                |  29 +++
 bins/Cargo.toml                |   1 +
 bins/tests/Cargo.toml          |  13 +
 bins/tests/integration_test.rs | 456 +++++++++++++++++++++++++++++++++
 link-git/src/protocol/fetch.rs |   5 +-
 link-git/src/protocol/ls.rs    |   5 +-
 6 files changed, 507 insertions(+), 2 deletions(-)
 create mode 100644 bins/tests/Cargo.toml
 create mode 100644 bins/tests/integration_test.rs

diff --git a/bins/Cargo.lock b/bins/Cargo.lock
index ca8d4229..25d0898b 100644
--- a/bins/Cargo.lock
@@ -840,6 +840,17 @@ dependencies = [
 "termcolor",
]

[[package]]
name = "errno"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e2b2decb0484e15560df3210cf0d78654bb0864b2c138977c07e377a1bae0e2"
dependencies = [
 "kernel32-sys",
 "libc",
 "winapi 0.2.8",
]

[[package]]
name = "event-listener"
version = "2.5.3"
@@ -2943,6 +2954,16 @@ dependencies = [
 "human_format",
]

[[package]]
name = "pty"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f50f3d255966981eb4e4c5df3e983e6f7d163221f547406d83b6a460ff5c5ee8"
dependencies = [
 "errno",
 "libc",
]

[[package]]
name = "quanta"
version = "0.4.1"
@@ -3568,6 +3589,14 @@ dependencies = [
 "winapi-util",
]

[[package]]
name = "tests"
version = "0.1.0"
dependencies = [
 "pty",
 "serde_json",
]

[[package]]
name = "textwrap"
version = "0.15.0"
diff --git a/bins/Cargo.toml b/bins/Cargo.toml
index 3d1786f4..99b3c69f 100644
--- a/bins/Cargo.toml
@@ -5,4 +5,5 @@ members = [
  "lnk-gitd",
  "lnk-identities-dev",
  "lnk-profile-dev",
  "tests",
]
diff --git a/bins/tests/Cargo.toml b/bins/tests/Cargo.toml
new file mode 100644
index 00000000..3ae82769
--- /dev/null
@@ -0,0 +1,13 @@
[package]
name = "tests"
version = "0.1.0"
authors = ["Han Xu <keepsimple@gmail.com>"]
edition = "2021"

[dev-dependencies]
pty = "0.2"
serde_json = "1.0"

[[test]]
name = "integration_test"
path = "integration_test.rs"
diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
new file mode 100644
index 00000000..2d2a4a92
--- /dev/null
@@ -0,0 +1,456 @@
// Copyright © 2022 The Radicle Link Contributors
//
// This file is part of radicle-link, distributed under the GPLv3 with Radicle
// Linking Exception. For full terms see the included LICENSE file.

//! Integration test to exercise the programs in `bins`.

use pty::fork::*;
use serde_json::{json, Value};
use std::{
    env,
    fs::File,
    io::{BufRead, BufReader, Write},
    process::{Child, Command, Stdio},
    thread,
    time::{Duration, SystemTime},
};

/// This test is inspired by https://github.com/alexjg/linkd-playground
///
/// Tests a typical scenario: there are two peer nodes and one seed node.
/// The main steps are:
///   - Setup a profile for each node in tmp home directories.
///   - Setup SSH keys for each profile.
///   - Create identities.
///   - Create a local repo for peer 1.
///   - Start `linkd` for the seed, and `lnk-gitd` for peer 1.
///   - Push peer 1 repo to its monorepo and to the seed.
///   - Clone the peer 1 repo to peer 2 via seed.
///
/// This test only works in Linux at this moment.
#[test]
fn two_peers_and_a_seed() {
    let peer1_home = "/tmp/link-local-1";
    let peer2_home = "/tmp/link-local-2";
    let seed_home = "/tmp/seed-home";
    let passphrase = b"play\n";

    {
        println!("\n== create lnk homes for two peers and one seed ==\n");
        let cmd = LnkCmd::ProfileCreate;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    {
        println!("\n== add ssh keys for each profile to the ssh-agent ==\n");
        let cmd = LnkCmd::ProfileSshAdd;
        run_lnk!(&cmd, peer1_home, passphrase);
        run_lnk!(&cmd, peer2_home, passphrase);
        run_lnk!(&cmd, seed_home, passphrase);
    }

    {
        println!("\n== Creating local link 1 identity ==\n");
        let peer1_name = "sockpuppet1".to_string();
        let cmd = LnkCmd::IdPersonCreate { name: peer1_name };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn };
        run_lnk!(&cmd, peer1_home, passphrase);
    }

    {
        println!("\n== Creating local link 2 identity ==\n");
        let peer2_name = "sockpuppet2".to_string();
        let cmd = LnkCmd::IdPersonCreate { name: peer2_name };
        let output = run_lnk!(&cmd, peer2_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let urn = v["urn"].as_str().unwrap().to_string();
        let cmd = LnkCmd::IdLocalSet { urn };
        run_lnk!(&cmd, peer2_home, passphrase);
    }

    let peer1_proj_dir = format!("peer1_proj_{}", timestamp());
    let peer1_proj_urn = {
        println!("\n== Create a local repository for peer1 ==\n");
        let cmd = LnkCmd::IdProjectCreate {
            name: peer1_proj_dir.clone(),
        };
        let output = run_lnk!(&cmd, peer1_home, passphrase);
        let v: Value = serde_json::from_str(&output).unwrap();
        let proj_urn = v["urn"].as_str().unwrap().to_string();
        println!("our project URN: {}", &proj_urn);
        proj_urn
    };

    println!("\n== Add the seed to the local peer seed configs ==\n");
    let seed_endpoint = {
        let cmd = LnkCmd::ProfilePeer;
        let seed_peer_id = run_lnk!(&cmd, seed_home, passphrase);
        format!("{}@127.0.0.1:8799", &seed_peer_id)
    };

    {
        // Create seed file for peer1
        let cmd = LnkCmd::ProfileGet;
        let peer1_profile = run_lnk!(&cmd, peer1_home, passphrase);
        let peer1_seed = format!("{}/{}/seeds", peer1_home, peer1_profile);
        let mut peer1_seed_f = File::create(peer1_seed).unwrap();
        peer1_seed_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    {
        // Create seed file for peer2
        let cmd = LnkCmd::ProfileGet;
        let peer2_profile = run_lnk!(&cmd, peer2_home, passphrase);
        let peer2_seed = format!("{}/{}/seeds", peer2_home, peer2_profile);
        let mut peer2_seed_f = File::create(peer2_seed).unwrap();
        peer2_seed_f.write_all(seed_endpoint.as_bytes()).unwrap();
    }

    println!("\n== Start the seed linkd ==\n");
    let manifest_path = manifest_path();
    let mut linkd = spawn_linkd(seed_home, &manifest_path);

    println!("\n== Start the peer 1 gitd ==\n");
    let cmd = LnkCmd::ProfilePeer;
    let gitd_addr = "127.0.0.1:9987";
    let peer1_peer_id = run_lnk!(&cmd, peer1_home, passphrase);
    let mut gitd = spawn_lnk_gitd(peer1_home, &manifest_path, gitd_addr);

    println!("\n== Make some changes in the repo: add and commit a test file ==\n");
    env::set_current_dir(&peer1_proj_dir).unwrap();
    let mut test_file = File::create("test").unwrap();
    test_file.write_all(b"test").unwrap();
    Command::new("git")
        .arg("add")
        .arg("test")
        .output()
        .expect("failed to do git add");
    let output = Command::new("git")
        .arg("commit")
        .arg("-m")
        .arg("test commit")
        .output()
        .expect("failed to do git commit");
    println!("git-commit: {:?}", &output);

    println!("\n== Add the linkd remote to the repo ==\n");
    let remote_url = format!("ssh://rad@{}/{}.git", gitd_addr, &peer1_proj_urn);
    Command::new("git")
        .arg("remote")
        .arg("add")
        .arg("linkd")
        .arg(remote_url)
        .output()
        .expect("failed to do git remote add");

    clean_up_known_hosts();

    if run_git_push_in_child_process() {
        // The child process is done with git push.
        return;
    }

    let peer1_last_commit = git_last_commit();

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let peer2_proj = format!("peer2_proj_{}", timestamp());
    let cmd = LnkCmd::Clone {
        urn: peer1_proj_urn,
        peer_id: peer1_peer_id,
        path: peer2_proj.clone(),
    };
    run_lnk!(&cmd, peer2_home, passphrase);

    env::set_current_dir(peer2_proj).unwrap();
    let peer2_last_commit = git_last_commit();
    println!("\n== peer1 proj last commit: {}", &peer1_last_commit);
    println!("\n== peer2 proj last commit: {}", &peer2_last_commit);

    println!("\n== Cleanup: kill linkd (seed) and gitd (peer1) ==\n");

    linkd.kill().ok();
    gitd.kill().ok();

    assert_eq!(peer1_last_commit, peer2_last_commit);
}

enum LnkCmd {
    ProfileCreate,
    ProfileGet,
    ProfilePeer,
    ProfileSshAdd,
    IdPersonCreate {
        name: String,
    },
    IdLocalSet {
        urn: String,
    },
    IdProjectCreate {
        name: String,
    },
    Clone {
        urn: String,
        peer_id: String,
        path: String,
    },
}

/// Runs a `lnk` command of `$cmd` using `$lnk_home` as the node home.
/// Also support the passphrase input for commands that need it.
#[macro_export]
macro_rules! run_lnk {
    ( $cmd:expr, $lnk_home:ident, $passphrase:ident ) => {{
        let fork = Fork::from_ptmx().unwrap();
        if let Some(mut parent) = fork.is_parent().ok() {
            match $cmd {
                LnkCmd::ProfileCreate | LnkCmd::ProfileSshAdd => {
                    // Input the passphrase if necessary.
                    parent.write_all($passphrase).unwrap();
                },
                _ => {},
            }
            process_lnk_output($lnk_home, &mut parent, $cmd)
        } else {
            start_lnk_cmd($lnk_home, $cmd);
            return;
        }
    }};
}

fn process_lnk_output(lnk_home: &str, lnk_process: &mut Master, cmd: &LnkCmd) -> String {
    let buf_reader = BufReader::new(lnk_process);
    let mut output = String::new();
    for line in buf_reader.lines() {
        let line = line.unwrap();

        // Print the output and decode them if necessary.
        println!("{}: {}", lnk_home, line);
        match cmd {
            LnkCmd::IdPersonCreate { name: _ } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
            },
            LnkCmd::IdProjectCreate { name: _ } => {
                if line.find("\"urn\":").is_some() {
                    output = line; // get the line with URN.
                }
            },
            LnkCmd::ProfileGet => {
                output = line; // get the last line for profile id.
            },
            LnkCmd::ProfilePeer => {
                output = line; // get the last line for peer id.
            },
            LnkCmd::Clone {
                urn: _,
                peer_id: _,
                path: _,
            } => {
                output = line;
            },
            _ => {},
        }
    }

    output
}

fn start_lnk_cmd(lnk_home: &str, cmd: &LnkCmd) {
    let manifest_path = manifest_path();

    // cargo run \
    // --manifest-path $LINK_CHECKOUT/bins/Cargo.toml \
    // -p lnk -- "$@"
    let mut lnk_cmd = Command::new("cargo");
    lnk_cmd
        .env("LNK_HOME", lnk_home)
        .arg("run")
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk")
        .arg("--");
    let full_cmd = match cmd {
        LnkCmd::ProfileCreate => lnk_cmd.arg("profile").arg("create"),
        LnkCmd::ProfileGet => lnk_cmd.arg("profile").arg("get"),
        LnkCmd::ProfilePeer => lnk_cmd.arg("profile").arg("peer"),
        LnkCmd::ProfileSshAdd => lnk_cmd.arg("profile").arg("ssh").arg("add"),
        LnkCmd::IdPersonCreate { name } => {
            let payload = json!({ "name": name });
            lnk_cmd
                .arg("identities")
                .arg("person")
                .arg("create")
                .arg("new")
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::IdLocalSet { urn } => lnk_cmd
            .arg("identities")
            .arg("local")
            .arg("set")
            .arg("--urn")
            .arg(urn),
        LnkCmd::IdProjectCreate { name } => {
            let payload = json!({"name": name, "default_branch": "master"});
            let project_path = format!("./{}", name);
            lnk_cmd
                .arg("identities")
                .arg("project")
                .arg("create")
                .arg("new")
                .arg("--path")
                .arg(project_path)
                .arg("--payload")
                .arg(payload.to_string())
        },
        LnkCmd::Clone { urn, peer_id, path } => lnk_cmd
            .arg("clone")
            .arg("--urn")
            .arg(urn)
            .arg("--path")
            .arg(path)
            .arg("--peer")
            .arg(peer_id),
    };
    full_cmd.status().expect("lnk cmd failed:");
}

fn spawn_linkd(lnk_home: &str, manifest_path: &str) -> Child {
    let log_name = format!("linkd_{}.log", &timestamp());
    let log_file = File::create(&log_name).unwrap();
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/linkd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("linkd")
        .output()
        .expect("cargo build linkd failed");

    let child = Command::new(&exec_path)
        .env("RUST_BACKTRACE", "1")
        .arg("--lnk-home")
        .arg(lnk_home)
        .arg("--track")
        .arg("everything")
        .arg("--protocol-listen")
        .arg("127.0.0.1:8799")
        .stdout(Stdio::from(log_file))
        .spawn()
        .expect("linkd failed to start");
    println!("linkd stdout redirected to {}", &log_name);
    thread::sleep(Duration::from_secs(1));
    child
}

fn spawn_lnk_gitd(lnk_home: &str, manifest_path: &str, addr: &str) -> Child {
    let target_dir = bins_target_dir();
    let exec_path = format!("{}/debug/lnk-gitd", &target_dir);

    Command::new("cargo")
        .arg("build")
        .arg("--target-dir")
        .arg(&target_dir)
        .arg("--manifest-path")
        .arg(manifest_path)
        .arg("-p")
        .arg("lnk-gitd")
        .stdout(Stdio::inherit())
        .output()
        .expect("cargo build lnk-gitd failed");

    let child = Command::new(&exec_path)
        .arg(lnk_home)
        .arg("--push-seeds")
        .arg("--fetch-seeds")
        .arg("-a")
        .arg(addr)
        .spawn()
        .expect("lnk-gitd failed to start");
    println!("started lnk-gitd");
    child
}

/// Returns true if runs in the forked child process for git push.
/// returns false if runs in the parent process.
fn run_git_push_in_child_process() -> bool {
    let fork = Fork::from_ptmx().unwrap();
    if let Some(mut parent) = fork.is_parent().ok() {
        let yes = b"yes\n";
        let buf_reader = BufReader::new(parent);
        for line in buf_reader.lines() {
            let line = line.unwrap();
            println!("git-push: {}", line);
            if line.find("key fingerprint").is_some() {
                parent.write_all(yes).unwrap();
            }
        }

        false // This is not the child process.
    } else {
        Command::new("git")
            .arg("push")
            .arg("linkd")
            .status()
            .expect("failed to do git push");
        true // This is the child process.
    }
}

fn git_last_commit() -> String {
    let output = Command::new("git")
        .arg("rev-parse")
        .arg("HEAD")
        .output()
        .expect("failed to run git rev-parse");
    String::from_utf8_lossy(&output.stdout).to_string()
}

/// Returns UNIX_TIME in millis.
fn timestamp() -> u128 {
    let now = SystemTime::now()
        .duration_since(SystemTime::UNIX_EPOCH)
        .unwrap();
    now.as_millis()
}

/// Returns the full path of `bins` manifest file.
fn manifest_path() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/Cargo.toml", package_dir.strip_suffix("/tests").unwrap())
}

/// Returns the full path of `bins/target`.
fn bins_target_dir() -> String {
    let package_dir = env!("CARGO_MANIFEST_DIR");
    format!("{}/target", package_dir.strip_suffix("/tests").unwrap())
}

fn clean_up_known_hosts() {
    // ssh-keygen -f "/home/pi/.ssh/known_hosts" -R "[127.0.0.1]:9987"
    let home_dir = env!("HOME");
    let known_hosts = format!("{}/.ssh/known_hosts", &home_dir);
    let output = Command::new("ssh-keygen")
        .arg("-f")
        .arg(known_hosts)
        .arg("-R")
        .arg("[127.0.0.1]:9987")
        .output()
        .expect("failed to do ssh-keygen");
    println!("ssh-keygen: {:?}", &output);
}
diff --git a/link-git/src/protocol/fetch.rs b/link-git/src/protocol/fetch.rs
index a5a73065..98b92b51 100644
--- a/link-git/src/protocol/fetch.rs
+++ b/link-git/src/protocol/fetch.rs
@@ -38,8 +38,11 @@ use super::{packwriter::PackWriter, remote_git_version, transport};
//
// cf. https://lore.kernel.org/git/CD2XNXHACAXS.13J6JTWZPO1JA@schmidt/
// Fixed in `git.git` 1ab13eb, which should land in 2.34
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace_want_ref(caps: &client::Capabilities) -> bool {
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.33.0").unwrap());
    static FIXED_AFTER: Lazy<Version> = Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version <= *FIXED_AFTER)
diff --git a/link-git/src/protocol/ls.rs b/link-git/src/protocol/ls.rs
index b3516454..de95b4a3 100644
--- a/link-git/src/protocol/ls.rs
+++ b/link-git/src/protocol/ls.rs
@@ -22,9 +22,12 @@ use super::{remote_git_version, transport};
// Work around `git-upload-pack` not handling namespaces properly
//
// cf. https://lore.kernel.org/git/pMV5dJabxOBTD8kJBaPuWK0aS6OJhRQ7YFGwfhPCeSJEbPDrIFBza36nXBCgUCeUJWGmpjPI1rlOGvZJEh71Ruz4SqljndUwOCoBUDRHRDU=@eagain.st/
//
// Based on testing with git 2.25.1 in Ubuntu 20.04, this workaround is
// not needed. Hence the checked version is lowered to 2.25.0.
fn must_namespace(caps: &client::Capabilities) -> bool {
    static MIN_GIT_VERSION_NAMESPACES: Lazy<Version> =
        Lazy::new(|| Version::new("2.31.0").unwrap());
        Lazy::new(|| Version::new("2.25.0").unwrap());

    remote_git_version(caps)
        .map(|version| version < *MIN_GIT_VERSION_NAMESPACES)
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v8 1/3] fix post_receive to send updated refs in request_pull Export this patch

---
 cli/gitd-lib/src/hooks.rs | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/cli/gitd-lib/src/hooks.rs b/cli/gitd-lib/src/hooks.rs
index 5e928ffd..cf37d71f 100644
--- a/cli/gitd-lib/src/hooks.rs
+++ b/cli/gitd-lib/src/hooks.rs
@@ -77,6 +77,16 @@ where
        E: std::error::Error + Send + 'static,
        P: ProgressReporter<Error = E>,
    {
        // update the signed refs before a possible request_pull,
        // so that the peer can receive the latest refs.
        let at = update_signed_refs(
            reporter,
            self.spawner.clone(),
            self.pool.clone(),
            urn.clone(),
        )
        .await?;

        if self.post_receive.request_pull {
            tracing::info!("executing request-pull");
            request_pull(reporter, &self.client, &self.seeds, urn.clone()).await?;
@@ -87,14 +97,7 @@ where
            )
            .await?;
        }
        let at = match update_signed_refs(
            reporter,
            self.spawner.clone(),
            self.pool.clone(),
            urn.clone(),
        )
        .await?
        {
        let at = match at {
            Some(at) => at,
            None => return Ok(()),
        };
-- 
2.32.0 (Apple Git-132)

[PATCH radicle-link v9 1/3] fix post_receive to send updated refs in request_pull Export this patch

Signed-off-by: Han Xu <keepsimple@gmail.com>
---
 cli/gitd-lib/src/hooks.rs | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/cli/gitd-lib/src/hooks.rs b/cli/gitd-lib/src/hooks.rs
index 5e928ffd..cf37d71f 100644
--- a/cli/gitd-lib/src/hooks.rs
+++ b/cli/gitd-lib/src/hooks.rs
@@ -77,6 +77,16 @@ where
        E: std::error::Error + Send + 'static,
        P: ProgressReporter<Error = E>,
    {
        // update the signed refs before a possible request_pull,
        // so that the peer can receive the latest refs.
        let at = update_signed_refs(
            reporter,
            self.spawner.clone(),
            self.pool.clone(),
            urn.clone(),
        )
        .await?;

        if self.post_receive.request_pull {
            tracing::info!("executing request-pull");
            request_pull(reporter, &self.client, &self.seeds, urn.clone()).await?;
@@ -87,14 +97,7 @@ where
            )
            .await?;
        }
        let at = match update_signed_refs(
            reporter,
            self.spawner.clone(),
            self.pool.clone(),
            urn.clone(),
        )
        .await?
        {
        let at = match at {
            Some(at) => at,
            None => return Ok(()),
        };
-- 
2.32.1 (Apple Git-133)

[PATCH radicle-link v4 2/3] check peer1 and peer2 ref/heads/master Export this patch

---
 bins/tests/integration_test.rs | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

diff --git a/bins/tests/integration_test.rs b/bins/tests/integration_test.rs
index 770f1b5e..ea6815ab 100644
--- a/bins/tests/integration_test.rs
@@ -170,11 +170,15 @@ fn two_peers_and_a_seed() {
        return;
    }

    let peer1_last_commit = git_last_commit();
    println!("\n== peer1 project last commit: {} ==\n", &peer1_last_commit);

    println!("\n== Clone to peer2 ==\n");

    env::set_current_dir("..").unwrap(); // out of the peer1 proj directory.
    let peer2_proj = format!("peer2_proj_{}", timestamp());
    let (is_parent, _) = run_lnk(
        LnkCmd::Clone(proj_urn, peer1_peer_id),
        LnkCmd::Clone(proj_urn, peer1_peer_id, peer2_proj.clone()),
        peer2_home,
        passphrase,
    );
@@ -182,9 +186,16 @@ fn two_peers_and_a_seed() {
        return;
    }

    println!("\n== Kill linkd (seed) ==\n");
    env::set_current_dir(peer2_proj).unwrap();
    let peer2_last_commit = git_last_commit();
    println!("\n== peer1 proj last commit: {}", &peer1_last_commit);
    println!("\n== peer2 proj last commit: {}", &peer2_last_commit);

    println!("\n== Cleanup: kill linkd (seed) ==\n");

    linkd.kill().ok();

    assert_eq!(peer1_last_commit, peer2_last_commit);
}

enum LnkCmd {
@@ -195,7 +206,7 @@ enum LnkCmd {
    IdPersonCreate(String),  // the associated string is "the person's name".