Commit graph

91 commits

Author SHA1 Message Date
Atemu 11df41199b nix-prefetch-git: don't fetch everything when given a hash
It's hugely inefficient as we can't use shallow cloning (--depth=1).

This has been tested and adapted for quite a few hosts fetchgit is used on in
Nixpkgs. For those where fetching the hash directly doesn't work (most notably
git.savannah.gnu.org), we simply fall back to the old method.
2021-07-12 20:50:52 +02:00
Bjørn Forsman 2e8f6e0d93 fetchgit: document --fetch-lfs flag
Fixes: 74128561 ("fetchgit: add lfs support")
2021-05-07 15:38:54 -07:00
taku0 a964d7094a
Merge pull request #104714 from codedownio/tree-hashes
fetchgit: support passing tree hashes as "rev"
2021-02-20 15:22:57 +09:00
Michael Weiss 2aadb9a53d
nix-prefetch-git: Run git-init with --initial-branch=master
The reason for this change is simply to avoid the following messages
that are unnecessary and can be confusing (and these messages will be
repeated for each submodule):
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>

With this change the behaviour remains unchanged (apart from the
suppressed "warning" in the console output of course) and therefore this
doesn't cause any hashes to change and by default nix-prefetch-git uses
the "fetchgit" branch anyway (branchName can be set to override the
default):
Switched to a new branch 'fetchgit'

For that reason the initial branch name doesn't matter anyway and since
we're not relying on / hardcoding "master" we could simply switch to
"main" (which seems most common nowadays). See [0] for more details on
why this wouldn't break anything.
However, since the initial branch name doesn't matter and to avoid any
additional risks it was "decided" to keep using "master" (s. #113313).

[0]: https://github.com/NixOS/nixpkgs/pull/113313#issuecomment-780589516
2021-02-18 11:34:34 +01:00
Tom McLaughlin 6779902b32 fetchgit: support passing tree hashes as "rev" 2021-02-13 22:12:08 -08:00
Jonathan Ringer 741285611f fetchgit: add lfs support 2021-01-11 09:41:06 -08:00
Jörg Thalheim 2fb5dac372 nix-prefetch-git: also add store path to json output 2020-05-04 17:43:29 +05:30
Benjamin Hipple 259f3b3b68
Revert "fetchzip, fetchgit: cleanup handling of optional features and whitespace" 2020-03-26 00:35:40 -04:00
Benjamin Hipple 368ea2eb5e
Merge pull request #79581 from bhipple/feature/fetcher-whitespace-clean
fetchzip, fetchgit: cleanup handling of optional features and whitespace
2020-03-26 00:15:24 -04:00
Jan Tojnar 28faf5bc86
nix-prefetch-git: Fix output mangling
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.

Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
2020-02-14 22:15:36 +01:00
Jan Tojnar f19a101a73
nix-prefetch-git: report deepClone & leaveDotGit
Just like in be30ba8e0e
we want `fetchgit (builtins.fromJSON (builtins.readFile ./src.json))` to work.
2020-02-14 06:05:43 +01:00
Benjamin Hipple caff183402 fetchzip, fetchgit: cleanup handling of optional features and whitespace
No intended functional difference, just trying to polish to make this easier to
extend and more aligned with other nix conventions.
2020-02-08 21:51:46 -05:00
Dima df7f3e0f98 nix-prefetch-git: list --branch-name in help
Listing the option and brief explanation.
Addresses: https://github.com/NixOS/nixpkgs/issues/74182
2019-11-26 20:43:23 +01:00
Silvan Mosberger 89636b9a5e
nix-prefetch-git: Remove some pack file non-determinism 2019-08-31 01:16:50 +02:00
Matthew Bauer c08cdfa781
Merge pull request #54543 from thefloweringash/git-prefetch-errors
nix-prefetch-git: propagate errors under --quiet
2019-06-18 11:20:28 -04:00
Andreas Rammhold 46f84adc0c
nix-prefetch-git: add -h as shortcut for --help to improve quality of life
Especially as a new user it is a much better experience to receive a
proper help response to `-h`. Currently passing `-h` will cause some
runtime error with the `git remote` error help being shown. Not very
helpful.

It doesn't hurt to be a bit more user friendly in this case.
2019-05-03 23:56:40 +02:00
Andrew Childs 597d6d2bcd nix-prefetch-git: propagate errors under --quiet 2019-01-25 01:52:26 +09:00
Craig Younkins 8b12b17df3
treewide: Fix broken Gmane URLs 2018-12-25 22:34:55 -05:00
Guillaume Maudoux ea23a1ed4e fetchgit: adapt regex to git 2.18.0 2018-06-28 07:01:14 -05:00
Tim Cuthbertson c5454933a1 nix-prefetch-git: fix output fetchSubmodules property 2018-06-17 21:19:27 +10:00
Lionello Lunesu 9fe26eed9e nix-prefetch-git: fix handling of submodules with spaces
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:

error: key does not contain a section: .url
2018-05-04 23:16:51 +08:00
Will Dietz 0e95bed017 nix-prefetch-git: fix extraction of submodule hashes on latest git
Summary:

According to git-submodule manpage,
"git submodule status" prefixes the hash with a '-' if it is not
initialized, and other chars in other circumstances.
(this is consistent on the various git versions tested)

nix-prefetch-git runs "git submodule init" which does you'd think,
but apparently despite this earlier versions of git before 2.16
would still give the hash the '-' suffix.
In particular this is the behavior when using 2.15 and 2.14.1
from the nixos-17.09 and nixos-17.03 channels respectively.

The script then used awk to drop the first char of the first field
which does the wrong thing when there is no prefix emitted:
while there is a space character before the hash, this is not
part of the field and so we ended up eating the first character
of the hash.

To fix this in a way that also works with the previous behavior,
this commit instead uses awk to grab the hash field
and uses tr to delete any '-' chars should they be present.

This seems to work in my testing, and for example can now
successfully fetch the source for "nginxModules.brotli"
where previously it would generate an error:

fatal: '22564a95d9ab58865a096b8d9f7324ea5f2e03e' is not a commit and a branch 'fetchgit' cannot be created from it

(we dropped a '2' from the beginning of the hash)
2018-01-24 20:18:59 +02:00
Vaibhav Sagar 376f7aa761
Update nix-prefetch-git
s/understand/understood/
2017-10-29 12:02:26 +08:00
Vladimír Čunát 9e1c7ddaae
Merge branch 'master' into staging 2017-07-05 09:53:53 +02:00
Volth 535a6c3520 fetchFromGitHub: fix fetchSubmodules=true when revision is not hexadecimal 2017-07-04 20:22:13 +00:00
Harmen 6dab2c72df nix-prefetch-git: add --quiet to --help 2017-07-03 11:44:33 +02:00
Bjørn Forsman b8658f629b fetchgit: support "git@server:repo" URLs
Update the code that extracts the base name from a git repo URL so that
it can deal with URLs like "git@server:repo".
2017-06-24 14:34:08 +02:00
Connor Clark c7ba4643a9 nix-prefetch-git: remove call to non-existent print_metadata command; improve hash mismatch error message. 2017-01-03 18:57:37 -08:00
Jörg Thalheim 47d0d3d46f
nix-prefetch-git: fix date field
in some cases `git show` include tag information in the output
this is suppressed by the parameter '-1'
2016-12-05 12:32:14 +01:00
Jörg Thalheim 954d995394
nix-prefetch-git: escape string fields properly
json requires certain characters to be escaped in strings.
2016-12-05 12:32:14 +01:00
Bjørn Forsman 68205d0ade nix-prefetch-git: unbreak JSON output
This fixes a regression caused by commit f56ab9e
("nix-prefetch-git: Include the date in the machine-readable [...]")
where a couple of directory paths printed by pushd/popd appeared before
the JSON output on stdout (thus breaking it). Fix it by redirecting the
extraneous output to /dev/null.

Reported by Michael Alan Dorman <mdorman@ironicdesign.com>.
2016-07-04 14:39:47 +02:00
David Grayson f56ab9e5e4 nix-prefetch-git: Include the date in the machine-readable output on
stdout, in strict ISO 8601 format.

This will be helpful for automatically updating fetchgit expressions
and the dates in version numbers associated with them.
2016-07-03 11:19:13 +01:00
Benno Fünfstück 9ff91371d0 nix-prefetch-git: fix bash evaluation order dependency 2016-06-15 22:54:35 +02:00
zimbatm a42b7faaec nix-prefetch-git: shellcheck fixes
Used shellcheck (https://github.com/koalaman/shellcheck) to validate
the script and fixed any resulting escaping and ambiguity issues.
2016-06-12 13:45:20 +01:00
Eelco Dolstra a5fa7c25cb Merge pull request #15469 from NixOS/fetchgit
fetchgit: remove only .git folder
2016-05-16 16:44:55 +02:00
Domen Kožar 64a072e357 fetchgit: remove only .git
Source of this change goes back to 2009 and original version of
fetchgit at 205fb0c87e.

The nondeterminism is really caused by changing .git so leave other
files alone as they might be interesting.

Note: this causes a hash mismatch with Hydra's version of Git Plugin
which we should fix to comply.
2016-05-15 00:24:04 +01:00
Guido Zgraggen 6ea0ae58af nix-prefetch-git: create parent directories 2016-04-22 16:51:49 -07:00
zimbatm 40e9dff04a nix-prefetch-git: fix url_to_name heuristic
The function wasn't checking that *all* of the characters where
[a-z0-9]. Fixes #13921
2016-03-23 11:22:51 +00:00
Ryan Trinkle be30ba8e0e nix-prefetch-scripts: make nix-prefetch-git report fetchSubmodules in its JSON output
Previously, nix-prefetch-git would report the same JSON whether submodules were being fetched or not; with this change, the --fetch-submodules option will cause the JSON output to include "fetchSubmodules": true, so that fetchgit (builtins.fromJSON (builtins.readFile ./path/to/output.json)) will work.
2016-03-21 23:26:18 -04:00
zimbatm 90de261f33 nix-prefetch-git: change the default output to JSON
As discussed on the mailing list. The nix output was short-lived so it's
probably okay to change it.
2016-02-29 22:47:16 +00:00
Tim Cuthbertson 21547a61ba nix-prefetch-git: print out valid nix expression; make --quiet very quiet 2016-02-27 21:26:35 +11:00
Tim Cuthbertson 456cbb29d9 nix-prefetch-git: add --quiet flag and minor cleanup 2016-02-27 16:56:38 +11:00
Graham Christensen 60f354dfb8 nix-prefetch-git: output base32 hash so output matches nix-build errors
It turns out hashFormat has never been set.
2016-02-22 10:50:27 -06:00
zimbatm 02f5a01c19 nix-prefetch-git: use fetchgit's naming heuristic
This commit fixes #6651.

Before this change the `nix-prefetch-git` script would use a different store
name than nix's `fetchgit` function. Because of that it was not possible to
use `nix-prefetch-git` as a way to pre-populate the store (for example when
the user it using private git dependencies that needs access to the ssh agent)
2016-02-13 14:39:44 +00:00
Arnaud Spiwack 07f7ecdcce Fix usage message in nix-prefetch-git
The comment related to the `deepClone` and `no-deepClone` options was
misleading as these options have no relation with submodules, but on the
the depth in `git clone --depth n`.
2016-01-26 16:09:52 +01:00
Domen Kožar 9c571a98ed Merge pull request #10998 from andrewrynhard/master
Add help flag to nix-prefetch-git
2016-01-19 12:01:55 +01:00
Peter Simons ade9f7167d nix-prefetch-git: make sure the script is interpreted by bash
Fixes https://github.com/NixOS/nixpkgs/issues/11284.
2015-11-27 11:17:50 +01:00
Andrew Rynhard 42ff8ad780 Add help flag to nix-prefetch-git 2015-11-12 15:10:56 -08:00
Nikolay Amiantov f5deff5137 nix-prefetch-git: pull all tags from remote when checkouting by revision
Close #9790.
This fixes checkouting for a nasty combination:

1. To be checkouted is a revision which corresponds to tag in a form "<tag>^{}".
2. This revision is not fetched by default.
2015-09-20 18:24:34 +02:00
Georges Dubus 7d67efa3f2 Add support for building cargo'ed Rust programs 2015-04-21 19:46:29 +02:00