Authentication-Results: mail-b.sr.ht; dkim=pass header.d=sous-anneau-org.20210112.gappssmtp.com header.i=@sous-anneau-org.20210112.gappssmtp.com Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by mail-b.sr.ht (Postfix) with ESMTPS id CEBAC11EEDD for <~etalab/codegouvfr-devel@lists.sr.ht>; Mon, 29 Nov 2021 16:28:31 +0000 (UTC) Received: by mail-ed1-f42.google.com with SMTP id t5so74665715edd.0 for <~etalab/codegouvfr-devel@lists.sr.ht>; Mon, 29 Nov 2021 08:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sous-anneau-org.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=r1f86H+1ZLp4hTSEQfhMwmYW8ISsdJIdiWGo9pDF3P0=; b=Sk2NL4OB4gZ30zjBIyBuv7QE513wx/RfhUuhncZOfmbdZOgZJVerykGyZDTA5FYbV6 a5nNIUKIqjmnVm2zHPsRLb+ZbQqJmZkOmYqcbcgtlgDUCbpp6PpY/gRiiaDikGAcMoT5 kRbz0Rl4nCpVNkGBzUp4HU1g4mhSngb70CkETirCUIIwd8pr6NLH/OhclaejycPn8tFK S1Pbu7sWjOeEC9oQWUgTsNCY9+Tjn66WGwLLQiqtCNFyFgRLq2tKXNfbbSu3gl20QUsJ sVx5Ep+SZjt5We5/M/tAL+GOPVs7Adl6w7PaJu6QhjhQD22bzTckpXk/V0NBnRLTBXGi +sEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=r1f86H+1ZLp4hTSEQfhMwmYW8ISsdJIdiWGo9pDF3P0=; b=lrgnDHDlI2cXHlfJOclTZagrpYVwfzR27xiaP6fr9+4lwJn9RChdRTEu7DKxf/tnZ3 e2Q+QQZu64EOs0D1VOA1Mh4I7rh0vVbzKB43+x7iBDb7eo5bodLZytGtyQmzG/Z7KaO5 DvnBI5AU59rftK9LWtWlHDHmloKpvRF729LtYHme9CKGI7nlZtbKuNar9MZjSaTcJvhw LNiiddR1aOn2uBfD/8EiShVqn+19b5WsRInosxlns3EA+4953Li2lRpmPe591ip4PKUU jtttjyT50yswZn44ciat+nuJYUXIHLw2cHu+Rdjvl1Dp6jv10rTzcdD4+bBQCm04d0se 9QCA== X-Gm-Message-State: AOAM533NJJDAexwzZdnDG6NYsMH3/U68F68HPON0teLwRNJYDazGw6q+ nkmoDy7Hi+fB7P1txNTIFZg7dXTiaQWUGIlj X-Google-Smtp-Source: ABdhPJzuRFmO7kmUdl/3ONa9d2W9jZaETVzpN90y5N/Cz7s4ynmyD8P7neAp4cpYCgwh7gLkdXLKEQ== X-Received: by 2002:a17:907:72d2:: with SMTP id du18mr60582559ejc.570.1638203309531; Mon, 29 Nov 2021 08:28:29 -0800 (PST) Received: from Ginkaku-Ji.fritz.box ([2a02:678:5c7:2300:3091:56cc:7b68:c0d6]) by smtp.gmail.com with ESMTPSA id c8sm9171985edu.60.2021.11.29.08.28.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Nov 2021 08:28:29 -0800 (PST) From: Alain Vagner To: ~etalab/codegouvfr-devel@lists.sr.ht Cc: Alain Vagner Subject: [PATCH] added installation instructions Date: Mon, 29 Nov 2021 17:27:39 +0100 Message-Id: <20211129162739.61578-1-alain.vagner@sous-anneau.org> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit --- README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/README.md b/README.md index 09400c0..99a5014 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,20 @@ csv](https://git.sr.ht/~etalab/codegouvfr-fetch-data/tree/master/item/platforms. of supported platforms, we collect the data we need for [code.gouv.fr](https://code.gouv.fr). +# Installation and configuration + +1. Clone this repository: `git clone https://git.sr.ht/~etalab/codegouvfr-fetch-data && cd codegouvfr-fetch-data` +2. Install Python dependencies: `pip install -r requirements.txt` +3. [Create a GitHub Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) +4. Create an account on [libraries.io](https://libraries.io/) and create an API key on your [account page](https://libraries.io/account). +5. Set the following environment variables: GITHUB_TOKEN and LIBRARIESIO_API_KEY. Ex: `export GITHUB_TOKEN="your github token" ; export LIBRARIESIO_API_KEY="your libraries.io api key"` +6. Create the folders that will receive the output data: `mkdir -p data/organizations/csv && mkdir -p data/organizations/json && mkdir -p data/repositories/csv && mkdir -p data/repositories/json && mkdir -p data/libraries/csv && mkdir -p data/libraries/json` +7. Check the content of the `platforms.csv` file and update its content if needed. + +# Generate JSON and CSV files + +Launch the script with `python fetch.py`. The output files will be available in the subfolders of `data`. + ## Todo We aim at collecting data from more forges: -- 2.30.0