Commit graph

303 commits

Author SHA1 Message Date
Andreas Rammhold be5597fc9d
buildRustCrate: remove superfluous dependency overrides
By overriding each dependency on every level of the dependency tree we
are creating a lot of unnecessary instances of the same derivation

Looking at the output size of `nix-instantiate --trace-function-calls
-vvvv …` and the execution time I got about a 10x improvement after
applying this change.

It was probably good intentions that lead to these overrides but in
practice no tooling (that I know of) really needs this. `carnix` and
`crate2nix` are fine without those overrides. Furthermore I believe that
it is the job of the tooling around `buildRustCrate` to provide a
coherent set of overrides. By not enforcing all of the overrides, debug
flags, verbosity, … to be the same throughout the closure we also allow
consumers to override specific aspects of the crates. Some (older?)
crates might need different `crateOverrides` then newer crates with the
same name. Currently such situations can not (easily) be implemented
with the override in-place.
2020-02-11 11:48:45 +01:00
Benjamin Hipple 2115a2037c fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir
This has several advantages:

1. It takes up less space on disk in-between builds in the nix store.
2. It uses less space in the binary cache for vendor derivation packages.
3. It uses less network traffic downloading from the binary cache.
4. It plays nicely with hashed mirrors like tarballs.nixos.org, which only
   substitute --flat hashes on single files (not recursive directory hashes).
5. It's consistent with how simple `fetchurl` src derivations work.
6. It provides a stronger abstraction between input src-package and output
   package, e.g., it's harder to accidentally depend on the src derivation at
   runtime by referencing something like `${src}/etc/index.html`. Likewise, in
   the store it's harder to get confused with something that is just there as a
   build-time dependency vs. a runtime dependency, since the build-time
   src dependencies are tarred up.

Disadvantages are:
1. It takes slightly longer to untar at the start of a build.

As currently implemented, this attaches the compacted vendor.tar.gz feature as a
rider on `verifyCargoDeps`, since both of them are relatively newly implemented
behavior that change the `cargoSha256`.

If this PR is accepted, I will push forward the remaining rust packages with a
series of treewide PRs to update the `cargoSha256`s.
2020-02-10 10:17:29 -05:00
Andreas Rammhold 56e11bc8df
buildRustCrate: remap the current build dir to / for (more) reproducible builds 2020-02-06 01:18:59 +01:00
Andreas Rammhold a57d0fe0bb
buildRustCrate: fix #78412
`build.rs` files might create files. Those files are supposed to go into
`OUT_DIR` (envirionment variable) and not be overlayed onto the source
tree.
2020-01-28 14:07:58 +01:00
Andreas Rammhold 19698d15ce
buildRustCrateTests: add regression test for #74071 2020-01-28 14:07:58 +01:00
Andreas Rammhold 78faab1be0 buildRustCrateTests: add test case for rlib linking 2020-01-21 17:46:32 +01:00
Andreas Rammhold 406e0c9d51
buildRustCrateTests: fix some formatting issues 2020-01-21 17:32:48 +01:00
Andreas Rammhold d6a8b55fb0
buildRustCrate: treat rlib crates just like lib crates
Both version provide `rlib` files to link against. Previously we would
try to find a matching shared library in the `lib` output.
2020-01-21 17:22:59 +01:00
Andreas Rammhold 69c96adc53
buildRustCrateTests: use releaseTools.aggregate
Previously I did use `runCommand` to do the same. Using
releaseTools.aggregate seems a lot saner and we might get nicer hydra
output of the tests that are failing.
2020-01-16 13:24:15 +01:00
Andreas Rammhold 29a8575e3d
buildRustCrate: remove one of the odd library filename cases
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:

some_crate/
 `- src
   `- main.rs
   `- some_crate.rs

We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.

I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
2020-01-16 13:24:13 +01:00
Benjamin Hipple 6e8c377562 rustPlatform.buildRustPackage: cleaner output on verifyCargoDeps (#77567)
When this fails, the user may want to copy-paste the path to the "bad"
Cargo.lock file to inspect. The trailing `.` on `$cargoDeps.` gets caught in
most terminal copy-pastes. Since half the lines already don't have it, this
removes it from all of them for consistent output.
2020-01-12 17:19:17 +00:00
Andreas Rammhold 3e61906e1c
buildRustCrate: slight "rewording" and reformatting
There is no point in reinventinb builtins through `filterAttrs` or the
like. Lets just stick to what we already have in our toolbelt.
2020-01-07 11:57:34 +01:00
Andreas Rammhold a3a51763f9
buildRustCrate: add buildTests flag to tell rustc to build tests instead of binaries
This helps us instruct rustc to build tests instead of binaries. The
actual build will then ONLY produce test executables. This is a first
step towards having rust crate tests within nixpkgs.

We default back to only a single output in test cases since that is the
only reasonable thing to do here.

Producing libraries or binaries in addition to tests would theoretically
be feasible but usually generates different dependency trees. It is very
common to have some libraries in `[dev-depdendencies]` within Cargo.toml
just for your tests. To not start mixing things up going with a
dedicated derivation for the test build sounds like the best choice for
now.

To use this you must provide a proper test dependency chain to
`buildRustCrate` (as you would usually do with your non-test inputs).
And then set the `buildTests` attribute to `true`. The derivation will
then contain all tests that were built in `$out/tests`. All common test
patterns and directories should be supported and tested by this change.

Below is an example how you would run a single test from the derivation.
This commit contains some more examples in the `buildRustCrateTests`
attribute set that might be helpful.

```
let
  drv = buildRustCrate {
     …
     buildTests true;
  };
in runCommand "test-my-crate" {} ''
  touch $out
  exec ${drv}/tests/my-test
''
```
2020-01-07 11:57:34 +01:00
Andreas Rammhold 6383b42dae
buildRustCrate: fixup usage of builtins.filterSource
While unifying most of the lib function calls I accidentially changed
the filterSource functions as well. Since there were no tests I ended
up forgetting about this case (even thought I ran into it…).
2020-01-07 00:49:48 +01:00
Andreas Rammhold 9f03cb8562
Merge pull request #75563 from andir/cleanup-buildRustCrate
Cleanup buildRustCrate expression
2020-01-02 13:42:33 +01:00
Alyssa Ross b9d274b89d rustPlatform.fetchcargo: expose 2019-12-23 18:27:56 +00:00
Alyssa Ross fdfbb4671e rustPlatform: forward unpackPhase to fetchcargo
If a custom unpackPhase is used for the package, it needs to also be
used for fetchcargo so the same source is available for vendoring.
2019-12-23 18:27:56 +00:00
Alyssa Ross 839c9e9344 rustPlatform: forward fetchcargo args to stdenv
Most stdenv wrappers already work like this -- it allows greater
customisation.  We just have to be careful to remove arguments we're
using that shouldn't be passed to stdenv.  I've been conservative
here, because fetchcargo checksums shouldn't change lightly.
2019-12-23 18:27:56 +00:00
Andreas Rammhold 2eaaf7aafd
buildRustCrate: move common build functions to a dedicated file
This means we aren't rebuilding hat file for each crate we are building
and the buildPhase expression is a lot easier to comprehent.
2019-12-12 13:55:04 +01:00
Andreas Rammhold 3f49d7a3ea
buildRustCrate: deduplicate dependency override code
The previous lines were only different in the kind of dependencies but
otherwise exactly the same. It makes the entire thing a bit more
readable by moving this into a function that takes care of this.
2019-12-12 01:03:41 +01:00
Andreas Rammhold 6ad22f5b4d
buildRustCrate: use less bash for the build script
We can get rid of a bunch of workarounds that were in the build script
before by just passing on the `crateBin` attribute.

Before we converted the list of attributes to a string only to convert
it back in bash during the build phase. We can do the entire looping
through builds in Nix and thus need no conversion and parsing of
attributes over and over again.

The big part that still remains bash is the heuristic that cargo
introduced and that we can't do at eval time.
2019-12-12 01:03:11 +01:00
Andreas Rammhold 5ad83267ed
buildRustCrate: reflow the way extraRustcOpts is constructed
This should make it more obvious that we have three parts to it and not
just one long gibberish string that makes up all of it.
2019-12-11 23:27:58 +01:00
Andreas Rammhold d37f001164
buildRustCrate: rename makeDeps function to mkRustcDepArgs
This should carry the function better then `makeDeps` as it isn't
producing deps but the rustc arguments required to link against those.
2019-12-11 23:23:55 +01:00
Andreas Rammhold f4aeabd04a
buildRustCrate: document and cleanup the symbol seeding
That code had been in the derivation for a while but no explanation was
given why that is needed. It might be helpful to our future selfs to
document why things are done the way they are.
2019-12-11 23:23:55 +01:00
Andreas Rammhold db55d1f89d
buildRustCrate: use tr instead of sed (it reads a bit nicer)
I already have a few changes in here that will trigger rebuilds so I
might as well do that substitution now.
2019-12-11 22:40:19 +01:00
Andreas Rammhold 50b2ef28f7
buildRustCrate: move the color loggign & remove some runtime checks
The expression is already long and confusing enough without the color
stuff sprinkled in. Moving it to a dedicated file makes sense.

I switched a bit of the color support code to pure Nix since there
wasn't much point in doing that in bash while we can just do it in Nix.
2019-12-11 22:35:44 +01:00
Andreas Rammhold 0aac0e8d2c
buildRustCrate: builtins -> lib where possible
We can just use `lib` instead of `builtins` in all cases but the
`hashString` case. Also changed a few lines to make use of some optional
helpers from lib.
2019-12-11 22:01:36 +01:00
Jörg Thalheim 435c3ecde7
rust: add support for armv6l-linux and armv7l-linux (#73472)
rust: add support for armv6l-linux and armv7l-linux
2019-11-29 12:47:15 +00:00
Andreas Rammhold 1b748554d5
buildRustCrate: add lib output
This cuts down the dependency tree on some rust builds where a crate not
just exposes a binary but also a library. `$out/lib` contained a bunch
of extra support files that among other information carry linker flags
(including the full path to link-time dependencies). Worst case this led
to some binary outputs depending on the full build closure of rust
crates.

Moving all the `$out/lib` files to `$lib/lib` solves this nicely.

`lib` might be a bit weird here as they are most of the time just rlib
files (rust libraries). Those are essential only required during
compilation but they can also be shared objects (like with traditional
C-style packages). Which is why I went with `lib` for the new output.

One of the caveats we are running into here is that we do not (always)
know ahead of time of a crate produces just a library or just a binary.
Cargo allows for some ambiguity regarding whether or not a crate
provides one, two, … binaries and libraries as it's outputs. Ideally we
would be able to rely on the `crateType` entirely but so far that isn't
the case. More work on that area might show how difficult that actually
is.
2019-11-26 15:05:01 +01:00
Ben Wolsieffer 83ac9c07e4 rust: add support for armv6l-linux and armv7l-linux 2019-11-23 19:19:31 -05:00
Jörg Thalheim 56240d7f20
Merge pull request #71899 from decentriq/aslemmer/build-rust-package-add-target
build-support/rust: Add target option
2019-11-01 15:46:47 +00:00
exfalso a588b1dfbc build-support/rust: Add target option 2019-10-24 11:39:08 +01:00
Andreas Rammhold d13022417f buildRustPackage: support checkFlags and checkFlagsArray 2019-10-20 02:31:13 +02:00
zimbatm f8d67ec135
buildRustPackage: add verifyCargoDeps option
One issue with cargoSha256 is that it's hard to detect when it needs to
be updated or not. It's possible to upgrade a package and forget to
update cargoSha256 and run with old versions of the program or
libraries.

This commit introduces `verifyCargoDeps` which, when enabled, will check
that the Cargo.lock is not out of date in the cargoDeps by comparing it
with the package source.
2019-09-30 17:09:52 +00:00
Joachim F ad773d31e2
Merge pull request #69345 from joachifm/feat/split-version
Replace uses of splitString for splitting version strings
2019-09-27 06:19:18 +00:00
Joachim Fasting bad07dfac5
tree-wide: replace uses of splitString "." with lib.versions
Quoting from the splitString docstring:

   NOTE: this function is not performant and should never be used.

This replaces trivial uses of splitString for splitting version
strings with the (potentially builtin) splitVersion.
2019-09-26 17:42:49 +02:00
Mario Rodas aed74e8284
cargo-vendor: drop
Cargo 1.37 imported `cargo-vendor` as built-in command [1]

[1] https://github.com/rust-lang/cargo/pull/6869
2019-09-22 20:20:20 -05:00
Yegor Timoshenko 17a6ae03cb
Merge pull request #66617 from transumption-unstable/201908/rust-aarch64-linux-musl
buildRustPackage: fix cross-compilation to aarch64-unknown-linux-musl
2019-09-19 01:21:55 +00:00
Yegor Timoshenko 08b73c1d58
buildRustPackage: fix cross-compilation to aarch64-unknown-linux-musl 2019-09-12 09:00:17 +00:00
Andreas Rammhold a69a6c1117
Merge pull request #68296 from danieldk/crateRenames
buildRustCrate: add support for renaming crates
2019-09-10 10:57:54 +02:00
Daniël de Kok 85c6d72011 buildRustCrate: add support for renaming crates
Before this change, buildRustCrate always called rustc with

--extern libName=[...]libName[...]

However, Cargo permits using a different name under which a dependency
is known to a crate. For example, rand 0.7.0 uses:

[dependencies]
getrandom_package = { version = "0.1.1", package = "getrandom", optional = true }

Which introduces the getrandom dependency such that it is known as
getrandom_package to the rand crate. In this case, the correct extern
flag is of the form

--extern getrandom_package=[...]getrandom[...]

which is currently not supported. In order to support such cases, this
change introduces a crateRenames argument to buildRustCrate. This
argument is an attribute set of dependencies that should be renamed. In
this case, crateRenames would be:

{
  "getrandom" = "getrandom_package";
}

The extern options are then built such that if the libName occurs as
an attribute in this set, it value will be used as the local
name. Otherwise libName will be used as before.
2019-09-08 19:17:52 +02:00
volth 08f68313a4 treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
volth 35d68ef143 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
Frederik Rietdijk c68f58d95c Merge master into staging-next 2019-08-17 09:30:16 +02:00
Yegor Timoshenko da3da08cdf
buildRustPackage: link against pthreads-w32 on x86_64-pc-mingw32 2019-08-15 08:44:54 +00:00
Yegor Timoshenko 0fcffe88db
buildRustPackage: support cross-compilation to x86_64-pc-mingw32 2019-08-15 08:44:54 +00:00
Frederik Rietdijk 4ca8e53e1d Merge staging-next into staging 2019-08-01 09:44:06 +02:00
Frederik Rietdijk 55e4555b77 Merge master into staging-next 2019-08-01 09:42:54 +02:00
Adelbert Chang 4403d44763 buildRustPackage: fix cargo flag for release/debug build (#61521)
Previous behavior did not actually allow for "debug" build as it would
try to pass --debug to cargo, which is not a valid flag.
2019-07-31 13:19:01 +01:00
Atkins fb961d1cdb
buildRustPackage: Remove unneeded cat 2019-07-30 12:27:16 +01:00
Atkins 626ccd7121
buildRustPackage: Avoid altering .cargo/config in source 2019-07-30 12:26:27 +01:00
worldofpeace b1bc0645ea gdk-pixbuf: rename from gdk_pixbuf 2019-07-22 18:50:57 -04:00
Mario Rodas da492d230b
buildRustPackage: by default use Rust's platforms 2019-07-21 00:00:00 -05:00
worldofpeace 3f4a353737 treewide: use dontUnpack 2019-07-01 04:23:51 -04:00
volth f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Peter Kolloch 61ac550082 Fix #60125 - buildRustCrate: Always set CARGO_PKG_VERSION_PRE and CARGO_PKG_HOMEPAGE
(as cargo does)
2019-04-23 23:41:57 +02:00
Andreas Rammhold 1bb989ca70 cargo-vendor: fix build on Darwin
This is supposedly fixing the build of the cargo crate on Drawin [1].

[1] https://github.com/NixOS/nixpkgs/pull/57017#pullrequestreview-228868016
2019-04-21 12:11:54 +02:00
Andreas Rammhold 2e2f7cba90 cargo: fix build on Darwin
This is supposedly fixing the build of the cargo crate on Drawin [1].

[1] https://github.com/NixOS/nixpkgs/pull/57017#pullrequestreview-228868016
2019-04-21 12:11:54 +02:00
Alexander Krupenkin a4902a33bf cargo-vendor: 0.1.13 -> 0.1.23 2019-04-21 12:11:54 +02:00
Frederik Rietdijk d108b49168 Merge master into staging-next 2019-04-09 16:38:35 +02:00
Andreas Rammhold 1462ef27bb
Merge pull request #58394 from P-E-Meunier/carnix-0.10
Carnix: 0.9 -> 0.10
2019-04-08 20:46:14 +00:00
Jan Tojnar cb1a20499a
Merge branch 'master' into staging 2019-04-05 11:37:15 +02:00
Michael Eden f93470d7a3 rust: set PKG_CONFIG_ALLOW_CROSS conditionally in buildRustPackage 2019-03-29 11:11:00 -04:00
Michael Eden bdd3c3fdcb rust: allow building in debug or release modes 2019-03-29 11:11:00 -04:00
Michael Eden 60761e65ba rust: move releaseDir to target/release in the buildPhase 2019-03-29 11:11:00 -04:00
Jörg Thalheim 912dca193a rust: fix cross-compilation 2019-03-29 11:11:00 -04:00
Andreas Rammhold e0b4356c0d
Merge pull request #57936 from andir/build-rust-crate-nix-build-cores
buildRustCrate: use $NIX_BUILD_CORES for each of the crates
2019-03-29 12:29:43 +01:00
Pierre-Étienne Meunier 81d9ddab1a Carnix: 0.9 -> 0.10 2019-03-26 18:54:28 +00:00
Teo Klestrup Röijezon 5eec83eb83 buildRustCrate: Fix include filter
buildRustCrate has a handy `include` helper, that only imports those whitelisted
files and folders to the store.

However, the function's matching logic is broken and includes all files,
regardless of whether or not they're whitelisted, as long as the whitelist
contains at least one name (regardless of whether that name exists). This is
because it doesn't take into account that
`lib.strings.removePrefix "foo" "bar" == "bar"` (that is, paths that don't match
the prefix are passed straight through).
2019-03-25 15:24:42 +01:00
Andreas Rammhold 4c89619152
buildRustCrate: use $NIX_BUILD_CORES for each of the crates 2019-03-20 02:19:50 +01:00
Jörg Thalheim cd498f7c80
Merge pull request #56634 from marsam/buildrustcreate-pname-attribute
buildRustPackage: Allow pname attribute
2019-03-11 10:08:00 +00:00
Peter Kolloch baa3d6f43b
buildRustCrate: Make CARGO_MANIFEST_DIR absolute
While it is not obvious from the source, cargo sets CARGO_MANIFEST_DIR to an absolute directory. This let to a build problem with the popular "tera" crate using the "pest" crate.

## Cargo details

The variable is set here:

f7c91ba622/src/cargo/core/compiler/compilation.rs (L229)

and computed from the `manifest_path`:

f7c91ba622/src/cargo/core/package.rs (L163)

The manifest path is also exported via `cargo metadata` where you can see that it is absolute.
2019-03-03 12:02:26 +01:00
Mario Rodas 435b326722
buildRustPackage: Allow pname attribute 2019-03-01 21:48:44 -05:00
Pierre-Étienne Meunier 32e94c2674 buildRustCrate: adding the description field 2019-02-25 15:22:03 +00:00
Ryan Mulligan d461adff17
Merge pull request #56314 from P-E-Meunier/carnix-0.9.8
Carnix: 0.9.7 -> 0.9.8
2019-02-24 15:10:31 -08:00
Pierre-Étienne Meunier 7a1853ef35 Carnix: 0.9.7 -> 0.9.8 2019-02-24 19:29:13 +00:00
Andreas Rammhold 6a3b144d02
buildRustCrate: support rust editions
In combination with carnix we can now build crates that require a
specific edition of rust features. A few crates started requiring that
already and having this in nixpkgs is just logical.
2019-02-18 01:09:54 +01:00
Andreas Rammhold 044a8a24ed
buildRustCrate: pass extraRustcOpts to configure crate
Previously build flags would not be available during the configure phase
while they might be required to build the `build.rs` file.
2019-02-18 00:10:36 +01:00
Pierre-Etienne Meunier 121318a9b4 toml2nix: init at 0.1.1 (#53883) 2019-01-15 23:36:36 +02:00
Alexander Krupenkin 451b181586
fetchcargo: fixed config installation path 2018-12-14 16:23:33 +03:00
Darius Jahandarie da84602f61 buildRustCrate: fix extraRustcOpts so it is not discarded 2018-12-12 01:21:46 -06:00
Frederik Rietdijk 5f554279ec Merge master into staging-next 2018-12-07 15:22:35 +01:00
Austin Seipp 3aa9091162 buildRustCreate: export RUSTDOC during cargo config
See https://github.com/NixOS/nixpkgs/pull/50452#issuecomment-443455411

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2018-12-04 19:51:51 -06:00
Graham Christensen fc459de60e
Merge pull request #50452 from P-E-Meunier/carnix-fix
Rust build-support: fixing a compilation error in some crates
2018-12-03 10:53:25 -05:00
Pierre-Étienne Meunier 3083fa2aa1 Carnix 0.9.2 2018-11-27 16:08:11 +00:00
Jörg Thalheim dc8aca448d
Merge pull request #51028 from clefru/tmp-cargo-config
buildRustPackage: write cargo config to temporary file instead of source dir
2018-11-26 15:00:16 +00:00
Clemens Fruhwirth 8d4fbc55d8 Write cargo config to temporary file instead of source dir.
... as this fails if the source dir contains a "config" directory.
2018-11-25 15:26:31 +01:00
Pierre-Étienne Meunier 0e8332ca2b Fixing "include" 2018-11-22 11:40:03 +00:00
Jörg Thalheim 952f4fda86
makeRustPlatform: refactor to make it easier to understand
It is now clearer what is supposed to be in the rust attribute set
without having studied type theory. The amount of code is identically.
2018-11-21 12:44:58 +00:00
Austin Seipp 04a543b3a0 defaultCrateOverrides: foundationdb native dependencies
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2018-11-17 19:28:48 -06:00
Pierre-Étienne Meunier f1de24feb8 Rust build-support: fixing a compilation error in some crates (such as proc-macro2) 2018-11-16 12:12:59 +00:00
Jörg Thalheim 96c627b3f6
defaultCrateOverrides: add serde_derive 2018-10-28 21:59:19 +00:00
Jörg Thalheim e0a5689528
defaultCrateOverrides: order alphabetically 2018-10-28 21:55:26 +00:00
Pierre-Etienne Meunier ae3b4655a4 Carnix: 0.7.2 -> 0.8.10 (#40587)
Carnix: splits input into two parts: creates from creates.io and local ones
2018-10-28 00:06:29 +01:00
Justin Humm 64d0676fe1
buildRustPackage: fix regex for separating lib and bin
E.g. exa was wrongly put into /lib, as it matches

  .*.a

but not

  .*\.a
2018-10-07 22:14:19 +02:00
Edward Tate 6ad43a0bce
buildRustPackage now correctly installs binaries to bin and libraries to lib. 2018-10-03 16:27:10 +02:00
John Ericson 7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
Silvan Mosberger 50578abfc5
fetchcargo: Fix cargo-vendor-normalise for darwin 2018-09-17 20:23:50 +02:00
Andreas Rammhold fc5e595003
buildRustCrate: added some edge cases with binaries
This commit adds test based on real-world crates (brotli).
There were a few more edge cases that were missing beforehand. Also it
turned out that we can get rid of the `finalBins` list since that will
now be handled during runtime.
2018-09-13 22:00:29 +02:00
Andreas Rammhold 0f95d05548 buildRustCrate: add test cases 2018-09-13 20:28:39 +02:00
Andreas Rammhold fdc2017f1c buildRustCrate: binary heuristic should be able to treat spaces 2018-09-13 20:28:39 +02:00
Andreas Rammhold 1371815060 buildRustCrate: extracted builder scripts into dedicated files
The build expression got quiet large over time and to make it a bit
easier to grasp the different scripts involved in the build are now
separated from the nix file.
2018-09-13 20:28:39 +02:00
Andreas Rammhold 0c50140da5 buildRustCrate: add heuristic to picking the right source files
Cargo has a few odd (old) ways of picking source files if the `bin.path`
attribute isn't given in the Cargo.toml. This commit adds support for
some of those. The previous behaviour always defaulted to `src/main.rs`
which was not always the right choice.

Since there is  look-ahead into the unpacked sources before running the
actual builder the path selection logic has to be embedded within the
build script.

`buildRustCrate` currently supports two ways of running building
binaries when processing a crate:

- Explicit definition of all the binaries (& optionally the paths to
their respective `main.rs`) and,
- if not binary was explictly configured all files matching the patterns
  `src/main.rs`, `src/bin/*.rs`.

When the explicit list is given without path information paths are now
being picked from a list of candidates. The first match wins. The order
is the same as within the cargo compatibility code.

If the crate does not provide any libraries the path `src/{bin_name}.rs`
is also considered.

All underscores within the binary names are translated into dashes (`-`)
before the lookups are made. This seems to be a common convention.
2018-09-13 20:28:39 +02:00
Symphorien Gibol a3e1da17cb cargo-vendor-normalise: add a small install check 2018-09-11 23:44:14 +02:00
Jörg Thalheim 7bfa20198a fetchcargo: add type checking to cargo-vendor-normalise.py 2018-09-11 23:44:14 +02:00
Symphorien Gibol f20b229aa1 fectchcargo: don't break old sha256 2018-09-11 23:44:14 +02:00
Symphorien Gibol ccf72b8537 fetchcargo: normalise cargo config to ensure determinism 2018-09-11 23:44:14 +02:00
Justin Humm b66ef28841 buildRustPackage, fetchcargo: optionally use vendor config from cargo-vendor
By setting useRealVendorConfig explicitly to true, the actual (slightly
modified) config generated by cargo-vendor is used.

This solves a problem, where the static vendor config in
pkgs/build-support/rust/default.nix would not sufficiently replace all
crates Cargo is looking for.

As useRealVendorConfig (and writeVendorConfig in fetchcargo) default to
false, there should be no breakage in existing cargoSha256 hashes.

Nethertheless, imho using this new feature should become standard. A
possible deprecation path could be:

- introduce this patch
- set useRealVendorConfig explicitly to false whereever cargoSha256 is
  set but migration is not wanted yet.
- after some time, let writeVendorConfig default to true
- when useRealVendorConfig is true everywhere cargoSha256 is set and
  enough time is passed, `assert cargoVendorDir == null ->
  useRealVendorConfig;`, remove old behaviour
- after some time, remove all appearences of useRealVendorConfig and the
  parameter itself
2018-09-11 23:44:14 +02:00
John Ericson 0828e2d8c3 treewide: Remove usage of remaining redundant platform compatability stuff
Want to get this out of here for 18.09, so it can be deprecated
thereafter.
2018-08-30 17:20:32 -04:00
Stewart Mackenzie efac36aa88 carnix overrides: add gmp to rink-rs buildInputs & correct crateBin 2018-08-15 13:20:04 +08:00
Léo Gaspard 48e5fbe8ee
buildRustPackage: allow patches to fix Cargo.lock 2018-08-13 22:07:58 +09:00
volth 52f53c69ce pkgs/*: remove unreferenced function arguments 2018-07-21 02:48:04 +00:00
Frederik Rietdijk 1a6af9f88e
Merge pull request #43857 from volth/unused
[bot] treewide: remove unreferenced code
2018-07-20 21:06:32 +02:00
volth 87f5930c3f [bot]: remove unreferenced code 2018-07-20 18:48:37 +00:00
Matthew Bauer 76999cc40e treewide: remove aliases in nixpkgs
This makes the command ‘nix-env -qa -f. --arg config '{skipAliases =
true;}'’ work in Nixpkgs.

Misc...

- qtikz: use libsForQt5.callPackage

  This ensures we get the right poppler.

- rewrites:

  docbook5_xsl -> docbook_xsl_ns
  docbook_xml_xslt -> docbook_xsl

diffpdf: fixup
2018-07-18 23:25:20 -04:00
Shea Levy 98ddba156c
buildRustCrate: Add some commentary about target_os. 2018-07-02 11:22:47 -04:00
Shea Levy cb692ff813
Merge branch 'feature/fix-build-rust-create-darwin' of git://github.com/marsam/nixpkgs
Set target_os properly on darwin.
2018-07-02 11:20:25 -04:00
Vladimír Čunát 392e6de7d0
Merge branch 'master' into staging 2018-05-20 13:20:53 +02:00
P-E-Meunier aa1d7961e7 curl-sys: fix linking against zlib 2018-05-20 11:30:06 +01:00
P-E-Meunier c0e2f7bbbe buildRustCrate: add extraLinkFlags parameter
This is useful when build scripts do not apply linking flags
2018-05-20 11:29:34 +01:00
Tuomas Tynkkynen 003473613a Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/top-level/all-packages.nix
2018-05-18 03:54:38 +03:00
Matthew Bauer 768bd58a48 crate-overrides: curl-sys needs zlib 2018-05-17 14:20:29 -05:00
Matthew Bauer e0fccdcc8d rust: add more sys overrides 2018-05-17 14:20:29 -05:00
Daiderd Jordan 65e92d19d2
Merge pull request #34968 from timokau/rust-find
buildRustPackage: Restrict `find` to files
2018-04-23 21:29:08 +02:00
pe@pijul.org ec40f193ac disable parallel rustc (-C codegen-units=1) 2018-04-16 16:16:28 +02:00
pe@pijul.org 8e87f73e36 Update to 0.7.2 2018-04-16 16:07:47 +02:00
pe@pijul.org 29a3059746 Carnix 0.7 2018-04-16 14:11:25 +02:00
Kevin Cox 4499513e54
rust: Allow setting cargoSha256 to null.
Setting the hash to null is a convenient way to bypass the hash check
while developing. It looks like the ability to do this was inadvertently
removed while adding vendor directory support.

This still checks that the user is explicitly setting the value but
allows null as a valid option.
2018-04-07 22:48:55 +01:00
Jörg Thalheim d12cab3bb1 buildRustCrate: remove ancient test guards
Let's leave x"" to the 1990s, where they belong
2018-03-28 09:24:22 +01:00
Shell Turner 8cc6897ae9
buildRustCrate: fix equality testing
Use string equality instead of integer equality.
2018-03-27 20:08:48 +01:00
Jan Tojnar a31d98f312
tree-wide: autorename gnome packages to use dashes 2018-02-25 17:41:16 +01:00
Mario Rodas bdf031dd4f buildRustCrate: Set target_os to "macos" on darwin
The rust compiler uses "macos" as "target_os" conditional on Mac OS[1]

[1] 8e7a609e63/src/librustc_back/target/x86_64_apple_darwin.rs (L29)
2018-02-23 18:00:39 -05:00
Yurii Rashkovskii e1aecec4cd
build-support/rust: make use of abandoned cargoUpdateHook
Previously, cargoUpdateHook was meaningful as it was used
in
[`cargo-fetch-deps`](19d3cf81d3/pkgs/build-support/rust/fetch-cargo-deps (L71)).

However, this entire file was removed in
5f8cf0048e. As far as I can
tell, nothing in the code is using it, but it is still
being passed around:
https://github.com/NixOS/nixpkgs/search?q=cargoUpdateHook&type=Code&utf8=%E2%9C%93

There are, however, legitimate use cases for it. For example,
in some software, some dependencies are not locked in Cargo.toml
and this causes Cargo to try fetching another version of them.
This doesn't work well with vendoring crates.

This hook allows to inject patching or whatever necessary workarounds
in the crate vendoring process. I suppose that's what it was for
in there in the first place.

This patch restores this hook and makes it usable again.
2018-02-23 11:17:03 +07:00
Jörg Thalheim f61e8d98ff
rust: 1.22.1 -> 1.24.0 2018-02-20 09:59:26 +00:00
Pierre-Etienne Meunier 8e5ab6e7ac BuildRustCrate: more general overrides, and handling the "dylib" crate type (#35171)
* buildRustCrate: adding a symlink from libblah-xxxxx.so to libblah.so
* BuildRustCrate: overriding phases
* Carnix: 0.6.5 -> 0.6.6
* Fixing symlink_dependencies --buildDep
* Shorter symlink_dependencies
* running `runHook postBuild` *after* the build
2018-02-20 08:55:04 +01:00
Timo Kaufmann dc53518dc3 buildRustPackage: Restrict find to files
`find -executable` finds everything with the executable bit set,
including directories. Thats not harmful in this scenario as `cp` won't
copy those directories, but it does result in a few warning messages.
2018-02-14 17:27:03 +01:00
Stewart Mackenzie a5cabdb6b1 buildRustCrate: add a postInstall phase (#34906) 2018-02-13 17:28:32 +01:00
pe@pijul.org 113591c803 defaultCrateOverrides: add pq-sys
fixes #34228
2018-02-10 06:59:56 -06:00
pe@pijul.org 508bf1b318 defaultCrateOverrides: add thrussh-libsodium 2018-02-10 06:59:56 -06:00
gnidorah 810a19bab3 way-cooler: 0.6.2 -> 0.8.0 2018-02-04 05:17:53 +03:00
Jörg Thalheim 2a2c8eab26 rust: fix evaluation 2018-02-04 00:09:00 +00:00
Jörg Thalheim 6580b18d3f cargo-vendor: move to all-packages 2018-02-03 22:35:27 +00:00
pe@pijul.org 8f20e7ce3a carnix: 0.6.0 -> 0.6.5 2018-02-03 22:31:54 +00:00
Jörg Thalheim 8ee54334e9
Merge pull request #33980 from thefloweringash/cargo-vendor-carnix
cargo-vendor: Build from source using carnix
2018-02-03 10:28:57 +00:00
Pierre-Etienne Meunier 6fbaa05dd1 Carnix 0.6 (#34238) 2018-01-26 10:53:18 +00:00
Andrew Childs be797f7e1c cargo-vendor: Build from source using carnix
Removes a binary bootstrap, and enables cargo-vendor on aarch64.
2018-01-18 20:44:42 +09:00
Andrew Childs 62dcb3d5d0 buildRustCrate: Allow arbitrary attributes in crateOverrides 2018-01-18 20:42:00 +09:00
Tuomas Tynkkynen 6ed0fe7e45 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/build-support/fetchbower/default.nix
	pkgs/build-support/fetchdarcs/default.nix
	pkgs/build-support/fetchgx/default.nix
	pkgs/development/python-modules/botocore/default.nix
	pkgs/os-specific/linux/firmware/firmware-linux-nonfree/default.nix
	pkgs/tools/admin/awscli/default.nix
2018-01-14 21:18:27 +02:00
John Ericson e017a027d5
Merge pull request #33681 from obsidiansystems/fixed-output-deps
Fixed output deps
2018-01-10 14:28:10 -05:00
John Ericson 888404f11b treewide: Fix deps in a few other fixed output derivations 2018-01-10 11:18:44 -05:00