Commit graph

54 commits

Author SHA1 Message Date
Jörg Thalheim eac6797380
prefer-fetch-remote: an overlay to fetch on remote builders
This is useful when running tools like NixOps or nix-review
on workstations where the upload to the builder is significantly
slower then downloading the source on the builder itself.
2019-01-18 14:41:10 +00:00
Yegor Timoshenko f44012ba10
lib.release -> lib.trivial.release 2018-06-11 02:14:18 +00:00
Yegor Timoshenko 5a232b0f5b
Leverage lib.release 2018-06-11 01:50:06 +00:00
Yegor Timoshenko 16ed09a10e
Merge branch 'master' into fetchurl-user-agent 2018-05-18 10:39:44 +03:00
Jan Malakhovski a89899ce4e fetchurl: cleanup, better errors
Also fix what seems like bugs in uncommon `stdenv`s.
2018-02-18 14:24:53 +00:00
John Ericson 940c4fa3f5 treewide: Fetchers should use stdenvNoCC. 2018-01-10 11:18:44 -05:00
John Ericson 3d59b4d285 treewide: Fixed output fetch* derivations should use nativeBuildInputs 2018-01-09 20:14:46 -05:00
Dan Peebles 0cb623c3d9
fetchurl: add user agent
It would be nice to be able to track Nix requests. It's not trustworthy,
but can be helpful for stats and routing in HTTP logs.

Since `fetchurl` is used so widely, we should "magically" get a UA on
`fetchzip`, `fetchFromGitHub`, and other related fetchers.

Since `fetchurl` is only used for fixed-output derivations, this should
cause no mass rebuild.

User-Agent example: curl/7.57.0 Nixpkgs/18.03
2017-12-23 22:20:56 +00:00
Frederik Rietdijk ef21b240c8 fetchurl: add passthru
so that we could add e.g. the `version` attribute.
2017-11-05 10:12:19 +01:00
Shea Levy b71b7ee622 fetchFromGitHub: Allow private repos, hosted githubs 2017-05-19 15:38:01 -04:00
Sven Slootweg fa8adf2c47 fetchurl: fixed typo in error message
This typo was likely introduced by copy-pasting the error message from elsewhere and forgetting to change the text, during the MD5 deprecation process (#4491).
2017-04-21 16:41:09 +02:00
Robin Gloster f57185db95
fetch-*: remove md5 support
fixes #4491
2017-03-20 22:23:41 +01:00
Eelco Dolstra e3a873479e
Remove fetchMD5warn
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.

This also reverts commit 2ca8833383.
2016-11-30 15:14:30 +01:00
Michael Raskin f603dc11a6 fetch*: print a trace warning about md5 deprecation 2016-10-09 16:19:04 +02:00
Profpatsch 61462c94e6 lib/fetchers.nix: factor out impure proxy vars (#18702)
Apparently everyone just copied those variables, instead of creating a
library constant for them. Some even removed the comment. -.-
2016-09-17 21:50:01 +02:00
Domen Kožar 0f9268e52c fetchurl: assert required Nix version for sha512 2016-04-14 12:50:21 +01:00
Eelco Dolstra 3ecbe604ef fetchurl: Support SHA-512 hashes 2016-04-13 14:11:14 +02:00
Scott Olson 43a523526d Require at least one of url or urls in fetchurl. 2016-02-19 03:18:21 -06:00
Scott Olson 9cf93ba135 Simplify fetchurl assertion logic.
The two lines I removed technically assert the exact same thing, since `!a -> b`
is equivalent to `a || b`. So, I replaced the two lines with the more symmetric
form to make it clearer.
2016-02-18 22:39:43 -06:00
Domen Kožar c7383cb34b fetchurl: support executables 2016-01-19 11:17:49 +01:00
Jan Malakhovski bdf32ed2ab fetchurl: allow adding meta info; fetchFrom*: add meta.homepage
The point of this is to be able to do `meta.homepage = src.meta.homepage;`
instead of the usual copy-paste for the packages that are hosted
on these hosting services.
2015-05-24 14:26:39 +00:00
Vladimír Čunát c00d36f6c1 fetchurl: fix typo from 105154afed pushed today 2015-02-22 20:27:29 +01:00
Vladimír Čunát 105154afed fetchurl: build the mirror list locally 2015-02-22 19:26:22 +01:00
Domen Kožar 909b62fa6e fetchurl: friendly message if hash is not given 2014-11-23 17:49:32 +01:00
Linquize f40db99c7b fetchurl: Fix typo in comment (close #2591) 2014-05-13 11:23:50 +02:00
Eelco Dolstra ea36f3b868 fetchFromGitHub: Use .tar.gz instead of .zip
Also clean up the name attribute of fetchzip derivations a bit.
2014-05-09 15:53:44 +02:00
Eelco Dolstra c8df888858 Add a function "fetchzip"
This function downloads and unpacks a file in one fixed-output
derivation. This is primarily useful for dynamically generated zip
files, such as GitHub's /archive URLs, where the unpacked content of
the zip file doesn't change, but the zip file itself may (e.g. due to
minor changes in the compression algorithm, or changes in timestamps).

Fetchzip is implemented by extending fetchurl with a "postFetch" hook
that is executed after the file has been downloaded. This hook can
thus perform arbitrary checks or transformations on the downloaded
file.
2014-05-08 15:30:17 +02:00
Eelco Dolstra 83ca2c272a fetchurl: Ensure that ‘urls’ is a list 2014-02-19 13:58:42 +01:00
Eelco Dolstra acba9240cd nixos.org/tarballs -> tarballs.nixos.org
It's currently the same machine, but tarballs.nixos.org should become
an S3/CloudFront site eventually.
2013-06-25 14:12:16 +02:00
Gergely Risko 07b26ce493 fetchUrl: add curlOpts parameter (merge #535) 2013-05-16 10:45:16 +02:00
Eelco Dolstra 04765da09a fetchurl: Don't wait too long for the hashed mirrors
If the hashed mirror (nixos.org/tarballs) doesn't respond in 15
seconds, give up and try the original URL.
2013-02-06 15:15:28 +01:00
Eelco Dolstra f337ab4430 fetchurl: Remove some compatibility hacks 2013-02-06 15:00:33 +01:00
Eelco Dolstra 1baf48f087 Do some let-floating to prevent repeated evaluation of impureEnvVars 2012-08-13 15:15:16 -04:00
Eelco Dolstra aded38809b * Turn on `preferLocalBuild' for fetchurl.
svn path=/nixpkgs/trunk/; revision=22948
2010-08-04 12:37:03 +00:00
Yury G. Kudryashov 952bc80b4d Let fetchurl retry failed downloads. Add NIX_CURL_FLAGS impure var
svn path=/nixpkgs/trunk/; revision=22006
2010-05-27 18:59:19 +00:00
Yury G. Kudryashov 796ffd9f71 Move "fetchurl restricted" code to requireFile.
This reverts r21474 and r21467.
Also migrate wtk (r21472) to new API.

svn path=/nixpkgs/trunk/; revision=21575
2010-05-03 09:13:17 +00:00
Yury G. Kudryashov 3e1e56e10c fetchurl restricted: list nix-prefetch-url as well
svn path=/nixpkgs/trunk/; revision=21474
2010-05-01 07:07:36 +00:00
Yury G. Kudryashov 404c314412 Add support for restricted downloads
svn path=/nixpkgs/trunk/; revision=21467
2010-04-30 20:40:42 +00:00
Eelco Dolstra 18a08aa008 * Add an openSUSE mirror that contains discontinued releases (such as
10.3).

svn path=/nixpkgs/trunk/; revision=18385
2009-11-17 14:26:42 +00:00
Marc Weber ec0e356b9a Revert "adding NIX_CONTINUE_DOWNLOADS feature, see mkdir comment"
This reverts commit -r16401

svn path=/nixpkgs/trunk/; revision=16405
2009-07-16 17:17:23 +00:00
Marc Weber 10f6c2592c adding NIX_CONTINUE_DOWNLOADS feature, see mkdir comment
svn path=/nixpkgs/trunk/; revision=16401
2009-07-16 15:18:26 +00:00
Eelco Dolstra aa84bce64d * Get rid of all references to nix.cs.uu.nl.
svn path=/nixpkgs/trunk/; revision=13299
2008-11-14 16:57:19 +00:00
Eelco Dolstra eebb241519 * Write the list of mirrors to a file that we can reuse between
fetchurl instantiations, instead of passing the mirrors to fetchurl
  instantiations via environment variables.  This makes the resulting
  store derivations (.drv files) much smaller, which in turn makes
  nix-env/nix-instantiate faster (4.8 -> 4.2 seconds on nix-env -qa
  --out-path).

svn path=/nixpkgs/trunk/; revision=12695
2008-08-22 15:53:21 +00:00
Eelco Dolstra a7c209e474 * Interface to support mirror:// in nix-prefetch-url. The function
resolveMirrorURLs produces a file containing the URLs to which a
  mirror:// URL resolves.

svn path=/nixpkgs/trunk/; revision=12410
2008-07-23 16:04:10 +00:00
Eelco Dolstra 76a8d120a4 * Allow the user to override the list of mirrors for specific
mirror:// sites through environment variables, e.g.

  NIX_MIRRORS_gnu="ftp://ftp.nluug.nl/pub/gnu/ ftp://ftp.gnu.org/pub/gnu/"

  or

  NIX_MIRRORS_sourceforge="http://surfnet.dl.sourceforge.net/sourceforge/"

svn path=/nixpkgs/trunk/; revision=9302
2007-09-11 15:00:49 +00:00
Eelco Dolstra 6c4fd2e3df * Allow the user to override the list of content-addressable mirrors
from the command-line by setting the NIX_HASHED_MIRRORS environment
  variable.

svn path=/nixpkgs/trunk/; revision=9301
2007-09-11 13:48:53 +00:00
Eelco Dolstra 45a2c87402 * Support for mirror:// URLs a la Gentoo (NIXPKGS-70). Instead of
fetchurl {
      url = http://heanet.dl.sourceforge.net/sourceforge/zapping/zapping-0.9.6.tar.bz2;
      md5 = "8306775c6a11de4d72345b5eee970ea6";
    };

  you can write

    fetchurl {
      url = mirror://sourceforge/zapping/zapping-0.9.6.tar.bz2;
      md5 = "8306775c6a11de4d72345b5eee970ea6";
    };

  which causes fetchurl to try the SourceForge mirrors listed in the
  `sourceforge' attribute in build-support/fetchurl/mirrors.nix.
  (They're currently tried in sequence, and the lists of mirrors are
  not configurable yet.)

  The syntax for mirror URLs is mirror://site/path/to/file, where
  `site' is currently one of `sourceforge', `gnu' (mirrors of
  ftp://ftp.gnu.org/pub/gnu) and `kernel' (mirrors of
  http://www.all.kernel.org/pub/).

svn path=/nixpkgs/trunk/; revision=9197
2007-08-27 12:44:01 +00:00
Eelco Dolstra 50321b735b * Basic transparent mirror support in fetchurl (NIXPKGS-70). When
fetching a file with hash HASH of type TYPE, we first try to
  download <base-url>/<type>/<hash>, where <base-url> is one of a list
  of mirrors.  For instance, given

    src = fetchurl {
      url = http://releases.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.6/source/firefox-2.0.0.6-source.tar.bz2;
      sha1 = "eb72f55e4a8bf08e8c6ef227c0ade3d068ba1082";
    };

  and the mirror list [http://nix.cs.uu.nl/dist/tarballs], we first
  try to download

    eb72f55e4a

  and if that fails, we use the original URL.

  The list of mirrors is not yet user-configurable.

* `fetchurl' now also accepts an argument `urls' instead of `url' for
  a list of alternative download locations, which fetchurl will try in
  sequence.

svn path=/nixpkgs/trunk/; revision=9190
2007-08-23 15:22:30 +00:00
Eelco Dolstra 2d33b25877 * fetchurl: accept an optional name. Useful for file names that contain
unacceptable characters (such as ? on Windows).

svn path=/nixpkgs/trunk/; revision=8149
2007-03-02 19:07:36 +00:00
Eelco Dolstra bb93fa69fa * fetchurl: use standard proxy-related variables from the caller such
as `http_proxy'.

svn path=/nixpkgs/trunk/; revision=5347
2006-05-31 09:52:39 +00:00