Commit graph

140 commits

Author SHA1 Message Date
Atemu 11df41199b nix-prefetch-git: don't fetch everything when given a hash
It's hugely inefficient as we can't use shallow cloning (--depth=1).

This has been tested and adapted for quite a few hosts fetchgit is used on in
Nixpkgs. For those where fetching the hash directly doesn't work (most notably
git.savannah.gnu.org), we simply fall back to the old method.
2021-07-12 20:50:52 +02:00
Bjørn Forsman 2e8f6e0d93 fetchgit: document --fetch-lfs flag
Fixes: 74128561 ("fetchgit: add lfs support")
2021-05-07 15:38:54 -07:00
Sandro d6df2c3fab
Merge pull request #111782 from Luflosi/fetchgit-escape-regex
fetchgit: escape dot in regex
2021-03-21 05:34:53 +01:00
taku0 a964d7094a
Merge pull request #104714 from codedownio/tree-hashes
fetchgit: support passing tree hashes as "rev"
2021-02-20 15:22:57 +09:00
Michael Weiss 2aadb9a53d
nix-prefetch-git: Run git-init with --initial-branch=master
The reason for this change is simply to avoid the following messages
that are unnecessary and can be confusing (and these messages will be
repeated for each submodule):
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>

With this change the behaviour remains unchanged (apart from the
suppressed "warning" in the console output of course) and therefore this
doesn't cause any hashes to change and by default nix-prefetch-git uses
the "fetchgit" branch anyway (branchName can be set to override the
default):
Switched to a new branch 'fetchgit'

For that reason the initial branch name doesn't matter anyway and since
we're not relying on / hardcoding "master" we could simply switch to
"main" (which seems most common nowadays). See [0] for more details on
why this wouldn't break anything.
However, since the initial branch name doesn't matter and to avoid any
additional risks it was "decided" to keep using "master" (s. #113313).

[0]: https://github.com/NixOS/nixpkgs/pull/113313#issuecomment-780589516
2021-02-18 11:34:34 +01:00
Tom McLaughlin 6779902b32 fetchgit: support passing tree hashes as "rev" 2021-02-13 22:12:08 -08:00
Luflosi e238ff073f
fetchgit: escape dot in regex
This regex should match files ending in `.git`, not any character and `git` after that.
2021-02-03 18:41:50 +01:00
Ben Siraphob 02ee14b9d5 treewide: stdenvNoCC.lib -> lib 2021-01-27 13:01:51 +07:00
Jonathan Ringer 741285611f fetchgit: add lfs support 2021-01-11 09:41:06 -08:00
Jörg Thalheim 2fb5dac372 nix-prefetch-git: also add store path to json output 2020-05-04 17:43:29 +05:30
Benjamin Hipple 259f3b3b68
Revert "fetchzip, fetchgit: cleanup handling of optional features and whitespace" 2020-03-26 00:35:40 -04:00
Benjamin Hipple 368ea2eb5e
Merge pull request #79581 from bhipple/feature/fetcher-whitespace-clean
fetchzip, fetchgit: cleanup handling of optional features and whitespace
2020-03-26 00:15:24 -04:00
Jan Tojnar 28faf5bc86
nix-prefetch-git: Fix output mangling
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.

Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
2020-02-14 22:15:36 +01:00
Jan Tojnar f19a101a73
nix-prefetch-git: report deepClone & leaveDotGit
Just like in be30ba8e0e
we want `fetchgit (builtins.fromJSON (builtins.readFile ./src.json))` to work.
2020-02-14 06:05:43 +01:00
Benjamin Hipple caff183402 fetchzip, fetchgit: cleanup handling of optional features and whitespace
No intended functional difference, just trying to polish to make this easier to
extend and more aligned with other nix conventions.
2020-02-08 21:51:46 -05:00
Dima df7f3e0f98 nix-prefetch-git: list --branch-name in help
Listing the option and brief explanation.
Addresses: https://github.com/NixOS/nixpkgs/issues/74182
2019-11-26 20:43:23 +01:00
adisbladis e578b8499f fetchgitPrivate: Remove fetcher
Since Nix 2.0 we have `builtins.fetchGit` which is a much better
option since it runs in the evaluator and has access to the regular
users ssh keys.
2019-11-14 09:34:12 -08:00
volth 7b8fb5c06c treewide: remove redundant quotes 2019-09-08 23:38:31 +00:00
Silvan Mosberger 89636b9a5e
nix-prefetch-git: Remove some pack file non-determinism 2019-08-31 01:16:50 +02:00
Matthew Bauer c08cdfa781
Merge pull request #54543 from thefloweringash/git-prefetch-errors
nix-prefetch-git: propagate errors under --quiet
2019-06-18 11:20:28 -04:00
Andreas Rammhold 46f84adc0c
nix-prefetch-git: add -h as shortcut for --help to improve quality of life
Especially as a new user it is a much better experience to receive a
proper help response to `-h`. Currently passing `-h` will cause some
runtime error with the `git remote` error help being shown. Not very
helpful.

It doesn't hurt to be a bit more user friendly in this case.
2019-05-03 23:56:40 +02:00
Andrew Childs 597d6d2bcd nix-prefetch-git: propagate errors under --quiet 2019-01-25 01:52:26 +09:00
Jörg Thalheim eac6797380
prefer-fetch-remote: an overlay to fetch on remote builders
This is useful when running tools like NixOps or nix-review
on workstations where the upload to the builder is significantly
slower then downloading the source on the builder itself.
2019-01-18 14:41:10 +00:00
Craig Younkins 8b12b17df3
treewide: Fix broken Gmane URLs 2018-12-25 22:34:55 -05:00
volth 52f53c69ce pkgs/*: remove unreferenced function arguments 2018-07-21 02:48:04 +00:00
Guillaume Maudoux ea23a1ed4e fetchgit: adapt regex to git 2.18.0 2018-06-28 07:01:14 -05:00
Tim Cuthbertson c5454933a1 nix-prefetch-git: fix output fetchSubmodules property 2018-06-17 21:19:27 +10:00
Lionello Lunesu 9fe26eed9e nix-prefetch-git: fix handling of submodules with spaces
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:

error: key does not contain a section: .url
2018-05-04 23:16:51 +08:00
Will Dietz 0e95bed017 nix-prefetch-git: fix extraction of submodule hashes on latest git
Summary:

According to git-submodule manpage,
"git submodule status" prefixes the hash with a '-' if it is not
initialized, and other chars in other circumstances.
(this is consistent on the various git versions tested)

nix-prefetch-git runs "git submodule init" which does you'd think,
but apparently despite this earlier versions of git before 2.16
would still give the hash the '-' suffix.
In particular this is the behavior when using 2.15 and 2.14.1
from the nixos-17.09 and nixos-17.03 channels respectively.

The script then used awk to drop the first char of the first field
which does the wrong thing when there is no prefix emitted:
while there is a space character before the hash, this is not
part of the field and so we ended up eating the first character
of the hash.

To fix this in a way that also works with the previous behavior,
this commit instead uses awk to grab the hash field
and uses tr to delete any '-' chars should they be present.

This seems to work in my testing, and for example can now
successfully fetch the source for "nginxModules.brotli"
where previously it would generate an error:

fatal: '22564a95d9ab58865a096b8d9f7324ea5f2e03e' is not a commit and a branch 'fetchgit' cannot be created from it

(we dropped a '2' from the beginning of the hash)
2018-01-24 20:18:59 +02:00
John Ericson 940c4fa3f5 treewide: Fetchers should use stdenvNoCC. 2018-01-10 11:18:44 -05:00
John Ericson 3d59b4d285 treewide: Fixed output fetch* derivations should use nativeBuildInputs 2018-01-09 20:14:46 -05:00
Frederik Rietdijk f8eed5f7a5 fetchgitPrivate: put our custom ssh on PATH
Currently we wrap ssh so it can find the config file passed in by
<ssh-config-file>. If one however uses ProxyCommand ssh, then ssh that
is on PATH is taken (which is also unavailable when using nix-shell
--pure), which is the plain ${openssh}/bin/ssh.

This commit makes sure our wrapped ssh is available on PATH.
2017-11-07 14:07:52 +01:00
Eelco Dolstra c3255fe8ec
fetchzip and friends: Set "name" to "source" by default
This makes them produce the same store paths as builtins.fetchgit,
builtins.fetchTarball etc. See
65b5f177b5.
2017-10-30 17:17:07 +01:00
Vaibhav Sagar 376f7aa761
Update nix-prefetch-git
s/understand/understood/
2017-10-29 12:02:26 +08:00
Vladimír Čunát 986c17727e
Merge: more compatibility for git* fetchers
They're additional commits from #26877.
Changing names of the fetched stuff was changing very many hashes,
and I think it's better to avoid that for the moment to reduce work
needed by nixpkgs users.  The fetchers are expected to be commonly
used even outside nixpkgs, and the current naming wasn't that bad
usually.

(commit analogical to d10c3cc5eedf58e80e2; I haven't noticed the part of
the PR has already got to master)
2017-07-09 10:31:24 +02:00
Volth ab8dd33e5c support old naming, just add escaping of unsafe chars 2017-07-05 16:01:26 +00:00
Vladimír Čunát 9e1c7ddaae
Merge branch 'master' into staging 2017-07-05 09:53:53 +02:00
Volth e7521e289c gitRepoToName: make it compatible with old fetchFromGitHub's "${repo}-${rev}-scr" to avoid mass rebuild 2017-07-04 20:22:13 +00:00
Volth 535a6c3520 fetchFromGitHub: fix fetchSubmodules=true when revision is not hexadecimal 2017-07-04 20:22:13 +00:00
Harmen 6dab2c72df nix-prefetch-git: add --quiet to --help 2017-07-03 11:44:33 +02:00
Bjørn Forsman b8658f629b fetchgit: support "git@server:repo" URLs
Update the code that extracts the base name from a git repo URL so that
it can deal with URLs like "git@server:repo".
2017-06-24 14:34:08 +02:00
Daiderd Jordan 7567c5c1ee
fetchgit: add postFetch argument 2017-06-03 20:51:01 +02:00
Robin Gloster f57185db95
fetch-*: remove md5 support
fixes #4491
2017-03-20 22:23:41 +01:00
Connor Clark c7ba4643a9 nix-prefetch-git: remove call to non-existent print_metadata command; improve hash mismatch error message. 2017-01-03 18:57:37 -08:00
Jörg Thalheim 47d0d3d46f
nix-prefetch-git: fix date field
in some cases `git show` include tag information in the output
this is suppressed by the parameter '-1'
2016-12-05 12:32:14 +01:00
Jörg Thalheim 954d995394
nix-prefetch-git: escape string fields properly
json requires certain characters to be escaped in strings.
2016-12-05 12:32:14 +01:00
Eelco Dolstra e3a873479e
Remove fetchMD5warn
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.

This also reverts commit 2ca8833383.
2016-11-30 15:14:30 +01:00
Alexey Shmalko a80cacd3ec
fetchFromGitHub: add fetchSubmodules option
This commit extends fetchFromGitHub with ability to fetch GitHub
repositories with submodules, so we can use the function consistently
with all GitHub repositories.

Note it doesn't change the previous behavior.
2016-11-18 13:04:26 +02:00
Michael Raskin f603dc11a6 fetch*: print a trace warning about md5 deprecation 2016-10-09 16:19:04 +02:00
Profpatsch 61462c94e6 lib/fetchers.nix: factor out impure proxy vars (#18702)
Apparently everyone just copied those variables, instead of creating a
library constant for them. Some even removed the comment. -.-
2016-09-17 21:50:01 +02:00