doc/stdenv: more changes

Lots of reworking here. May need to be split up.
This commit is contained in:
Matthew Bauer 2018-11-18 23:20:12 -06:00
parent 49e5bd0dbe
commit 4a49921a69

View file

@ -228,18 +228,19 @@ genericBuild
</para> </para>
<para> <para>
The extension of <envar>PATH</envar> with dependencies, alluded to above, The extension of <envar>PATH</envar> with dependencies, alluded to
proceeds according to the relative platforms alone. The process is carried above, proceeds according to the relative platforms alone. The
out only for dependencies whose host platform matches the new derivation's process is carried out only for dependencies whose host platform
build platformi.e. which run on the platform where the new derivation matches the new derivation's build platform i.e. dependencies which
will be built. run on the platform where the new derivation will be built.
<footnote xml:id="footnote-stdenv-native-dependencies-in-path"> <footnote xml:id="footnote-stdenv-native-dependencies-in-path">
<para> <para>
Currently, that means for native builds all dependencies are put on the Currently, this means for native builds all dependencies are put
<envar>PATH</envar>. But in the future that may not be the case for sake on the <envar>PATH</envar>. But in the future that may not be the
of matching cross: the platforms would be assumed to be unique for native case for sake of matching cross: the platforms would be assumed
and cross builds alike, so only the <varname>depsBuild*</varname> and to be unique for native and cross builds alike, so only the
<varname>nativeBuildDependencies</varname> dependencies would affect the <varname>depsBuild*</varname> and
<varname>nativeBuildInputs</varname> would be added to the
<envar>PATH</envar>. <envar>PATH</envar>.
</para> </para>
</footnote> </footnote>
@ -251,28 +252,27 @@ genericBuild
<para> <para>
The dependency is propagated when it forces some of its other-transitive The dependency is propagated when it forces some of its other-transitive
(non-immediate) downstream dependencies to also take it on as an immediate (non-immediate) downstream dependencies to also take it on as an immediate
dependency. Nix itself already takes a package's transitive dependencies dependency. Nix itself already takes a package's transitive dependencies into
into account, but this propagation ensures nixpkgs-specific infrastructure account, but this propagation ensures nixpkgs-specific infrastructure like
like setup hooks (mentioned above) also are run as if the propagated setup hooks (mentioned above) also are run as if the propagated dependency.
dependency.
</para> </para>
<para> <para>
It is important to note dependencies are not necessary propagated as the It is important to note dependencies are not necessarily propagated as the
same sort of dependency that they were before, but rather as the same sort of dependency that they were before, but rather as the
corresponding sort so that the platform rules still line up. The exact rules corresponding sort so that the platform rules still line up. The exact rules
for dependency propagation can be given by assigning each sort of dependency for dependency propagation can be given by assigning to each dependency two
two integers based one how it's host and target platforms are offset from integers based one how its host and target platforms are offset from the
the depending derivation's platforms. Those offsets are given depending derivation's platforms. Those offsets are given below in the
below in the descriptions of each dependency list attribute. descriptions of each dependency list attribute. Algorithmically, we traverse
Algorithmically, we traverse propagated inputs, accumulating every propagated inputs, accumulating every propagated dependency's propagated
propagated dep's propagated deps and adjusting them to account for the dependenciess and adjusting them to account for the "shift in perspective"
"shift in perspective" described by the current dep's platform offsets. This described by the current dependency's platform offsets. This results in sort
results in sort a transitive closure of the dependency relation, with the a transitive closure of the dependency relation, with the offsets being
offsets being approximately summed when two dependency links are combined. approximately summed when two dependency links are combined. We also prune
We also prune transitive deps whose combined offsets go out-of-bounds, which transitive dependencies whose combined offsets go out-of-bounds, which can be
can be viewed as a filter over that transitive closure removing dependencies viewed as a filter over that transitive closure removing dependencies that
that are blatantly absurd. are blatantly absurd.
</para> </para>
<para> <para>
@ -287,8 +287,8 @@ genericBuild
propagation logic. propagation logic.
</para> </para>
</footnote> </footnote>
They're confusing in very different ways so...hopefully if something doesn't They're confusing in very different ways so... hopefully if something doesn't
make sense in one presentation, it does in the other! make sense in one presentation, it will in the other!
<programlisting> <programlisting>
let mapOffset(h, t, i) = i + (if i &lt;= 0 then h else t - 1) let mapOffset(h, t, i) = i + (if i &lt;= 0 then h else t - 1)
@ -307,13 +307,13 @@ dep(h0, _, A, B)
propagated-dep(h1, t1, B, C) propagated-dep(h1, t1, B, C)
h0 + h1 in {-1, 0, 1} h0 + h1 in {-1, 0, 1}
h0 + t1 in {-1, 0, -1} h0 + t1 in {-1, 0, -1}
-------------------------------------- Take immediate deps' propagated deps ----------------------------- Take immediate dependencies' propagated dependencies
propagated-dep(mapOffset(h0, t0, h1), propagated-dep(mapOffset(h0, t0, h1),
mapOffset(h0, t0, t1), mapOffset(h0, t0, t1),
A, C)</programlisting> A, C)</programlisting>
<programlisting> <programlisting>
propagated-dep(h, t, A, B) propagated-dep(h, t, A, B)
-------------------------------------- Propagated deps count as deps ----------------------------- Propagated dependencies count as dependencies
dep(h, t, A, B)</programlisting> dep(h, t, A, B)</programlisting>
Some explanation of this monstrosity is in order. In the common case, the Some explanation of this monstrosity is in order. In the common case, the
target offset of a dependency is the successor to the target offset: target offset of a dependency is the successor to the target offset:
@ -324,31 +324,31 @@ let f(h, h + 1, i) = i + (if i &lt;= 0 then h else (h + 1) - 1)
let f(h, h + 1, i) = i + (if i &lt;= 0 then h else h) let f(h, h + 1, i) = i + (if i &lt;= 0 then h else h)
let f(h, h + 1, i) = i + h let f(h, h + 1, i) = i + h
</programlisting> </programlisting>
This is where the "sum-like" comes from above: We can just sum all the host This is where "sum-like" comes in from above: We can just sum all of the host
offset to get the host offset of the transitive dependency. The target offsets to get the host offset of the transitive dependency. The target
offset is the transitive dep is simply the host offset + 1, just as it was offset is the transitive dependency is simply the host offset + 1, just as it
with the dependencies composed to make this transitive one; it can be was with the dependencies composed to make this transitive one; it can be
ignored as it doesn't add any new information. ignored as it doesn't add any new information.
</para> </para>
<para> <para>
Because of the bounds checks, the uncommon cases are <literal>h = Because of the bounds checks, the uncommon cases are <literal>h = t</literal>
t</literal> and <literal>h + 2 = t</literal>. In the former case, the and <literal>h + 2 = t</literal>. In the former case, the motivation for
motivation for <function>mapOffset</function> is that since its host and <function>mapOffset</function> is that since its host and target platforms
target platforms are the same, no transitive dep of it should be able to are the same, no transitive dependency of it should be able to "discover" an
"discover" an offset greater than its reduced target offsets. offset greater than its reduced target offsets.
<function>mapOffset</function> effectively "squashes" all its transitive <function>mapOffset</function> effectively "squashes" all its transitive
dependencies' offsets so that none will ever be greater than the target dependencies' offsets so that none will ever be greater than the target
offset of the original <literal>h = t</literal> package. In the other case, offset of the original <literal>h = t</literal> package. In the other case,
<literal>h + 1</literal> is skipped over between the host and target <literal>h + 1</literal> is skipped over between the host and target offsets.
offsets. Instead of squashing the offsets, we need to "rip" them apart so no Instead of squashing the offsets, we need to "rip" them apart so no
transitive dependencies' offset is that one. transitive dependencies' offset is that one.
</para> </para>
<para> <para>
Overall, the unifying theme here is that propagation shouldn't be Overall, the unifying theme here is that propagation shouldn't be introducing
introducing transitive dependencies involving platforms the needing package transitive dependencies involving platforms the depending package is unaware
is unaware of. The offset bounds checking and definition of of. The offset bounds checking and definition of
<function>mapOffset</function> together ensure that this is the case. <function>mapOffset</function> together ensure that this is the case.
Discovering a new offset is discovering a new platform, and since those Discovering a new offset is discovering a new platform, and since those
platforms weren't in the derivation "spec" of the needing package, they platforms weren't in the derivation "spec" of the needing package, they
@ -369,20 +369,20 @@ let f(h, h + 1, i) = i + h
A list of dependencies whose host and target platforms are the new A list of dependencies whose host and target platforms are the new
derivation's build platform. This means a <literal>-1</literal> host and derivation's build platform. This means a <literal>-1</literal> host and
<literal>-1</literal> target offset from the new derivation's platforms. <literal>-1</literal> target offset from the new derivation's platforms.
They are programs/libraries used at build time that furthermore produce These are programs and libraries used at build time that produce programs
programs/libraries also used at build time. If the dependency doesn't and libraries also used at build time. If the dependency doesn't care
care about the target platform (i.e. isn't a compiler or similar tool), about the target platform (i.e. isn't a compiler or similar tool), put it
put it in <varname>nativeBuildInputs</varname> instead. The most common in <varname>nativeBuildInputs</varname> instead. The most common use of
use for this <literal>buildPackages.stdenv.cc</literal>, the default C this <literal>buildPackages.stdenv.cc</literal>, the default C compiler
compiler for this role. That example crops up more than one might think for this role. That example crops up more than one might think in old
in old commonly used C libraries. commonly used C libraries.
</para> </para>
<para> <para>
Since these packages are able to be run at build time, that are always Since these packages are able to be run at build-time, they are always
added to the <envar>PATH</envar>, as described above. But since these added to the <envar>PATH</envar>, as described above. But since these
packages are only guaranteed to be able to run then, they shouldn't packages are only guaranteed to be able to run then, they shouldn't
persist as run-time dependencies. This isn't currently enforced, but persist as run-time dependencies. This isn't currently enforced, but could
could be in the future. be in the future.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -395,21 +395,20 @@ let f(h, h + 1, i) = i + h
A list of dependencies whose host platform is the new derivation's build A list of dependencies whose host platform is the new derivation's build
platform, and target platform is the new derivation's host platform. This platform, and target platform is the new derivation's host platform. This
means a <literal>-1</literal> host offset and <literal>0</literal> target means a <literal>-1</literal> host offset and <literal>0</literal> target
offset from the new derivation's platforms. They are programs/libraries offset from the new derivation's platforms. These are programs and
used at build time that, if they are a compiler or similar tool, produce libraries used at build-time that, if they are a compiler or similar tool,
code to run at run time—i.e. tools used to build the new derivation. If produce code to run at run-time—i.e. tools used to build the new
the dependency doesn't care about the target platform (i.e. isn't a derivation. If the dependency doesn't care about the target platform (i.e.
compiler or similar tool), put it here, rather than in isn't a compiler or similar tool), put it here, rather than in
<varname>depsBuildBuild</varname> or <varname>depsBuildTarget</varname>. <varname>depsBuildBuild</varname> or <varname>depsBuildTarget</varname>.
This would be called <varname>depsBuildHost</varname> but for historical This could be called <varname>depsBuildHost</varname> but
continuity. <varname>nativeBuildInputs</varname> is used for historical continuity.
</para> </para>
<para> <para>
Since these packages are able to be run at build time, that are added to Since these packages are able to be run at build-time, they are added to
the <envar>PATH</envar>, as described above. But since these packages the <envar>PATH</envar>, as described above. But since these packages are
only are guaranteed to be able to run then, they shouldn't persist as only guaranteed to be able to run then, they shouldn't persist as run-time
run-time dependencies. This isn't currently enforced, but could be in the dependencies. This isn't currently enforced, but could be in the future.
future.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -422,34 +421,33 @@ let f(h, h + 1, i) = i + h
A list of dependencies whose host platform is the new derivation's build A list of dependencies whose host platform is the new derivation's build
platform, and target platform is the new derivation's target platform. platform, and target platform is the new derivation's target platform.
This means a <literal>-1</literal> host offset and <literal>1</literal> This means a <literal>-1</literal> host offset and <literal>1</literal>
target offset from the new derivation's platforms. They are programs used target offset from the new derivation's platforms. These are programs used
at build time that produce code to run at run with code produced by the at build time that produce code to run with code produced by the depending
depending package. Most commonly, these would tools used to build the package. Most commonly, these are tools used to build the runtime or
runtime or standard library the currently-being-built compiler will standard library taht the currently-being-built compiler will inject into
inject into any code it compiles. In many cases, the currently-being any code it compiles. In many cases, the currently-being-built-compiler is
built compiler is itself employed for that task, but when that compiler itself employed for that task, but when that compiler won't run (i.e. its
won't run (i.e. its build and host platform differ) this is not possible. build and host platform differ) this is not possible. Other times, the
Other times, the compiler relies on some other tool, like binutils, that compiler relies on some other tool, like binutils, that is always built
is always built separately so the dependency is unconditional. separately so that the dependency is unconditional.
</para> </para>
<para> <para>
This is a somewhat confusing dependency to wrap ones head around, and for This is a somewhat confusing concept to wrap ones head around, and for
good reason. As the only one where the platform offsets are not adjacent good reason. As the only dependency type where the platform offsets are
integers, it requires thinking of a bootstrapping stage not adjacent integers, it requires thinking of a bootstrapping stage
<emphasis>two</emphasis> away from the current one. It and it's use-case <emphasis>two</emphasis> away from the current one. It and its use-case go
go hand in hand and are both considered poor form: try not to need this hand in hand and are both considered poor form: try to not need this sort
sort dependency, and try not avoid building standard libraries / runtimes of dependency, and try to avoid building standard libraries and runtimes
in the same derivation as the compiler produces code using them. Instead in the same derivation as the compiler produces code using them. Instead
strive to build those like a normal library, using the newly-built strive to build those like a normal library, using the newly-built
compiler just as a normal library would. In short, do not use this compiler just as a normal library would. In short, do not use this
attribute unless you are packaging a compiler and are sure it is needed. attribute unless you are packaging a compiler and are sure it is needed.
</para> </para>
<para> <para>
Since these packages are able to be run at build time, that are added to Since these packages are able to run at build time, they are added to the
the <envar>PATH</envar>, as described above. But since these packages <envar>PATH</envar>, as described above. But since these packages are only
only are guaranteed to be able to run then, they shouldn't persist as guaranteed to be able to run then, they shouldn't persist as run-time
run-time dependencies. This isn't currently enforced, but could be in the dependencies. This isn't currently enforced, but could be in the future.
future.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -460,15 +458,15 @@ let f(h, h + 1, i) = i + h
<listitem> <listitem>
<para> <para>
A list of dependencies whose host and target platforms match the new A list of dependencies whose host and target platforms match the new
derivation's host platform. This means a both <literal>0</literal> host derivation's host platform. This means a <literal>0</literal> host offset
offset and <literal>0</literal> target offset from the new derivation's and <literal>0</literal> target offset from the new derivation's host
host platform. These are packages used at run-time to generate code also platform. These are packages used at run-time to generate code also used
used at run-time. In practice, that would usually be tools used by at run-time. In practice, this would usually be tools used by compilers
compilers for metaprogramming/macro systems, or libraries used by the for macros or a metaprogramming system, or libraries used by the macros or
macros/metaprogramming code itself. It's always preferable to use a metaprogramming code itself. It's always preferable to use a
<varname>depsBuildBuild</varname> dependency in the derivation being <varname>depsBuildBuild</varname> dependency in the derivation being built
built than a <varname>depsHostHost</varname> on the tool doing the over a <varname>depsHostHost</varname> on the tool doing the building for
building for this purpose. this purpose.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -479,20 +477,20 @@ let f(h, h + 1, i) = i + h
<listitem> <listitem>
<para> <para>
A list of dependencies whose host platform and target platform match the A list of dependencies whose host platform and target platform match the
new derivation's. This means a <literal>0</literal> host offset and new derivation's. This means a <literal>0</literal> host offset and a
<literal>1</literal> target offset from the new derivation's host <literal>1</literal> target offset from the new derivation's host
platform. This would be called <varname>depsHostTarget</varname> but for platform. This would be called <varname>depsHostTarget</varname> but for
historical continuity. If the dependency doesn't care about the target historical continuity. If the dependency doesn't care about the target
platform (i.e. isn't a compiler or similar tool), put it here, rather platform (i.e. isn't a compiler or similar tool), put it here, rather than
than in <varname>depsBuildBuild</varname>. in <varname>depsBuildBuild</varname>.
</para> </para>
<para> <para>
These often are programs/libraries used by the new derivation at These are often programs and libraries used by the new derivation at
<emphasis>run</emphasis>-time, but that isn't always the case. For <emphasis>run</emphasis>-time, but that isn't always the case. For
example, the machine code in a statically linked library is only used at example, the machine code in a statically-linked library is only used at
run time, but the derivation containing the library is only needed at run-time, but the derivation containing the library is only needed at
build time. Even in the dynamic case, the library may also be needed at build-time. Even in the dynamic case, the library may also be needed at
build time to appease the linker. build-time to appease the linker.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -604,10 +602,10 @@ let f(h, h + 1, i) = i + h
<listitem> <listitem>
<para> <para>
A natural number indicating how much information to log. If set to 1 or A natural number indicating how much information to log. If set to 1 or
higher, <literal>stdenv</literal> will print moderate debug information higher, <literal>stdenv</literal> will print moderate debugging
during the build. In particular, the <command>gcc</command> and information during the build. In particular, the <command>gcc</command>
<command>ld</command> wrapper scripts will print out the complete command and <command>ld</command> wrapper scripts will print out the complete
line passed to the wrapped tools. If set to 6 or higher, the command line passed to the wrapped tools. If set to 6 or higher, the
<literal>stdenv</literal> setup script will be run with <literal>set <literal>stdenv</literal> setup script will be run with <literal>set
-x</literal> tracing. If set to 7 or higher, the <command>gcc</command> -x</literal> tracing. If set to 7 or higher, the <command>gcc</command>
and <command>ld</command> wrapper scripts will also be run with and <command>ld</command> wrapper scripts will also be run with
@ -666,11 +664,10 @@ passthru = {
<literal>hello.baz.value1</literal>. We don't specify any usage or schema <literal>hello.baz.value1</literal>. We don't specify any usage or schema
of <literal>passthru</literal> - it is meant for values that would be of <literal>passthru</literal> - it is meant for values that would be
useful outside the derivation in other parts of a Nix expression (e.g. in useful outside the derivation in other parts of a Nix expression (e.g. in
other derivations). An example would be to convey some specific other derivations). An example would be to convey some specific dependency
dependency of your derivation which contains a program with plugins of your derivation which contains a program with plugins support. Later,
support. Later, others who make derivations with plugins can use others who make derivations with plugins can use passed-through dependency
passed-through dependency to ensure that their plugin would be to ensure that their plugin would be binary-compatible with built program.
binary-compatible with built program.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -1144,12 +1141,11 @@ passthru = {
By default, when cross compiling, the configure script has By default, when cross compiling, the configure script has
<option>--build=...</option> and <option>--host=...</option> passed. <option>--build=...</option> and <option>--host=...</option> passed.
Packages can instead pass <literal>[ "build" "host" "target" ]</literal> Packages can instead pass <literal>[ "build" "host" "target" ]</literal>
or a subset to control exactly which platform flags are passed. or a subset to control exactly which platform flags are passed. Compilers
Compilers and other tools should use this to also pass the target and other tools can use this to also pass the target platform.
platform, for example.
<footnote xml:id="footnote-stdenv-build-time-guessing-impurity"> <footnote xml:id="footnote-stdenv-build-time-guessing-impurity">
<para> <para>
Eventually these will be passed when in native builds too, to improve Eventually these will be passed building natively as well, to improve
determinism: build-time guessing, as is done today, is a risk of determinism: build-time guessing, as is done today, is a risk of
impurity. impurity.
</para> </para>
@ -1647,13 +1643,11 @@ installTargets = "install-bin install-doc";</programlisting>
</term> </term>
<listitem> <listitem>
<para> <para>
A package can export a <link A package can export a <link linkend="ssec-setup-hooks">setup hook</link>
linkend="ssec-setup-hooks">setup by setting this variable. The setup hook, if defined, is copied to
hook</link> by setting this variable. The setup hook, if defined, is <filename>$out/nix-support/setup-hook</filename>. Environment variables
copied to <filename>$out/nix-support/setup-hook</filename>. Environment are then substituted in it using <function
variables are then substituted in it using linkend="fun-substituteAll">substituteAll</function>.
<function
linkend="fun-substituteAll">substituteAll</function>.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -2086,12 +2080,12 @@ someVar=$(stripHash $name)
<title>Package setup hooks</title> <title>Package setup hooks</title>
<para> <para>
Nix itself considers a build-time dependency merely something that should Nix itself considers a build-time dependency as merely something that should
previously be built and accessible at build time—packages themselves are previously be built and accessible at build time—packages themselves are
on their own to perform any additional setup. In most cases, that is fine, on their own to perform any additional setup. In most cases, that is fine,
and the downstream derivation can deal with its own dependencies. But for a and the downstream derivation can deal with its own dependencies. But for a
few common tasks, that would result in almost every package doing the same few common tasks, that would result in almost every package doing the same
sort of setup work---depending not on the package itself, but entirely on sort of setup workdepending not on the package itself, but entirely on
which dependencies were used. which dependencies were used.
</para> </para>
@ -2106,20 +2100,19 @@ someVar=$(stripHash $name)
</para> </para>
<para> <para>
The Setup hook mechanism is a bit of a sledgehammer though: a powerful The setup hook mechanism is a bit of a sledgehammer though: a powerful
feature with a broad and indiscriminate area of effect. The combination of feature with a broad and indiscriminate area of effect. The combination of
its power and implicit use may be expedient, but isn't without costs. Nix its power and implicit use may be expedient, but isn't without costs. Nix
itself is unchanged, but the spirit of adding dependencies being effect-free itself is unchanged, but the spirit of added dependencies being effect-free
is violated even if the letter isn't. For example, if a derivation path is is violated even if the letter isn't. For example, if a derivation path is
mentioned more than once, Nix itself doesn't care and simply makes sure the mentioned more than once, Nix itself doesn't care and simply makes sure the
dependency derivation is already built just the same—depending is just dependency derivation is already built just the same—depending is just
needing something to exist, and needing is idempotent. However, a dependency needing something to exist, and needing is idempotent. However, a dependency
specified twice will have its setup hook run twice, and that could easily specified twice will have its setup hook run twice, and that could easily
change the build environment (though a well-written setup hook will change the build environment (though a well-written setup hook will therefore
therefore strive to be idempotent so this is in fact not observable). More strive to be idempotent so this is in fact not observable). More broadly,
broadly, setup hooks are anti-modular in that multiple dependencies, whether setup hooks are anti-modular in that multiple dependencies, whether the same
the same or different, should not interfere and yet their setup hooks may or different, should not interfere and yet their setup hooks may well do so.
well do so.
</para> </para>
<para> <para>
@ -2138,15 +2131,14 @@ someVar=$(stripHash $name)
<para> <para>
Packages adding a hook should not hard code a specific hook, but rather Packages adding a hook should not hard code a specific hook, but rather
choose a variable <emphasis>relative</emphasis> to how they are included. choose a variable <emphasis>relative</emphasis> to how they are included.
Returning to the C compiler wrapper example, if it itself is an Returning to the C compiler wrapper example, if the wrapper itself is an
<literal>n</literal> dependency, then it only wants to accumulate flags from <literal>n</literal> dependency, then it only wants to accumulate flags from
<literal>n + 1</literal> dependencies, as only those ones match the <literal>n + 1</literal> dependencies, as only those ones match the
compiler's target platform. The <envar>hostOffset</envar> variable is compiler's target platform. The <envar>hostOffset</envar> variable is defined
defined with the current dependency's host offset with the current dependency's host offset <envar>targetOffset</envar> with
<envar>targetOffset</envar> with its target offset, before its setup hook is its target offset, before its setup hook is sourced. Additionally, since most
sourced. Additionally, since most environment hooks don't care about the environment hooks don't care about the target platform, that means the setup
target platform, That means the setup hook can append to the right bash array hook can append to the right bash array by doing something like
by doing something like
<programlisting language="bash"> <programlisting language="bash">
addEnvHooks "$hostOffset" myBashFunction addEnvHooks "$hostOffset" myBashFunction
</programlisting> </programlisting>
@ -2171,19 +2163,19 @@ addEnvHooks "$hostOffset" myBashFunction
</term> </term>
<listitem> <listitem>
<para> <para>
Bintools Wrapper wraps the binary utilities for a bunch of miscellaneous The Bintools Wrapper wraps the binary utilities for a bunch of
purposes. These are GNU Binutils when targetting Linux, and a mix of miscellaneous purposes. These are GNU Binutils when targetting Linux, and
cctools and GNU binutils for Darwin. [The "Bintools" name is supposed to a mix of cctools and GNU binutils for Darwin. [The "Bintools" name is
be a compromise between "Binutils" and "cctools" not denoting any supposed to be a compromise between "Binutils" and "cctools" not denoting
specific implementation.] Specifically, the underlying bintools package, any specific implementation.] Specifically, the underlying bintools
and a C standard library (glibc or Darwin's libSystem, just for the package, and a C standard library (glibc or Darwin's libSystem, just for
dynamic loader) are all fed in, and dependency finding, hardening (see the dynamic loader) are all fed in, and dependency finding, hardening
below), and purity checks for each are handled by Bintools Wrapper. (see below), and purity checks for each are handled by the Bintools
Packages typically depend on CC Wrapper, which in turn (at run time) Wrapper. Packages typically depend on CC Wrapper, which in turn (at run
depends on Bintools Wrapper. time) depends on the Bintools Wrapper.
</para> </para>
<para> <para>
Bintools Wrapper was only just recently split off from CC Wrapper, so The Bintools Wrapper was only just recently split off from CC Wrapper, so
the division of labor is still being worked out. For example, it the division of labor is still being worked out. For example, it
shouldn't care about about the C standard library, but just take a shouldn't care about about the C standard library, but just take a
derivation with the dynamic loader (which happens to be the glibc on derivation with the dynamic loader (which happens to be the glibc on
@ -2191,24 +2183,24 @@ addEnvHooks "$hostOffset" myBashFunction
to need to share, and probably the most important to understand. It is to need to share, and probably the most important to understand. It is
currently accomplished by collecting directories of host-platform currently accomplished by collecting directories of host-platform
dependencies (i.e. <varname>buildInputs</varname> and dependencies (i.e. <varname>buildInputs</varname> and
<varname>nativeBuildInputs</varname>) in environment variables. Bintools <varname>nativeBuildInputs</varname>) in environment variables. The
Wrapper's setup hook causes any <filename>lib</filename> and Bintools Wrapper's setup hook causes any <filename>lib</filename> and
<filename>lib64</filename> subdirectories to be added to <filename>lib64</filename> subdirectories to be added to
<envar>NIX_LDFLAGS</envar>. Since CC Wrapper and Bintools Wrapper use <envar>NIX_LDFLAGS</envar>. Since the CC Wrapper and the Bintools Wrapper
the same strategy, most of the Bintools Wrapper code is sparsely use the same strategy, most of the Bintools Wrapper code is sparsely
commented and refers to CC Wrapper. But CC Wrapper's code, by contrast, commented and refers to the CC Wrapper. But the CC Wrapper's code, by
has quite lengthy comments. Bintools Wrapper merely cites those, rather contrast, has quite lengthy comments. The Bintools Wrapper merely cites
than repeating them, to avoid falling out of sync. those, rather than repeating them, to avoid falling out of sync.
</para> </para>
<para> <para>
A final task of the setup hook is defining a number of standard A final task of the setup hook is defining a number of standard
environment variables to tell build systems which executables full-fill environment variables to tell build systems which executables fulfill
which purpose. They are defined to just be the base name of the tools, which purpose. They are defined to just be the base name of the tools,
under the assumption that Bintools Wrapper's binaries will be on the under the assumption that the Bintools Wrapper's binaries will be on the
path. Firstly, this helps poorly-written packages, e.g. ones that look path. Firstly, this helps poorly-written packages, e.g. ones that look
for just <command>gcc</command> when <envar>CC</envar> isn't defined yet for just <command>gcc</command> when <envar>CC</envar> isn't defined yet
<command>clang</command> is to be used. Secondly, this helps packages <command>clang</command> is to be used. Secondly, this helps packages not
not get confused when cross-compiling, in which case multiple Bintools get confused when cross-compiling, in which case multiple Bintools
Wrappers may simultaneously be in use. Wrappers may simultaneously be in use.
<footnote xml:id="footnote-stdenv-per-platform-wrapper"> <footnote xml:id="footnote-stdenv-per-platform-wrapper">
<para> <para>
@ -2220,20 +2212,20 @@ addEnvHooks "$hostOffset" myBashFunction
</para> </para>
</footnote> </footnote>
<envar>BUILD_</envar>- and <envar>TARGET_</envar>-prefixed versions of <envar>BUILD_</envar>- and <envar>TARGET_</envar>-prefixed versions of
the normal environment variable are defined for the additional Bintools the normal environment variable are defined for additional Bintools
Wrappers, properly disambiguating them. Wrappers, properly disambiguating them.
</para> </para>
<para> <para>
A problem with this final task is that Bintools Wrapper is honest and A problem with this final task is that the Bintools Wrapper is honest and
defines <envar>LD</envar> as <command>ld</command>. Most packages, defines <envar>LD</envar> as <command>ld</command>. Most packages,
however, firstly use the C compiler for linking, secondly use however, firstly use the C compiler for linking, secondly use
<envar>LD</envar> anyways, defining it as the C compiler, and thirdly, <envar>LD</envar> anyways, defining it as the C compiler, and thirdly,
only so define <envar>LD</envar> when it is undefined as a fallback. only so define <envar>LD</envar> when it is undefined as a fallback. This
This triple-threat means Bintools Wrapper will break those packages, as triple-threat means Bintools Wrapper will break those packages, as LD is
LD is already defined as the actual linker which the package won't already defined as the actual linker which the package won't override yet
override yet doesn't want to use. The workaround is to define, just for doesn't want to use. The workaround is to define, just for the
the problematic package, <envar>LD</envar> as the C compiler. A good way problematic package, <envar>LD</envar> as the C compiler. A good way to
to do this would be <command>preConfigure = "LD=$CC"</command>. do this would be <command>preConfigure = "LD=$CC"</command>.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -2243,30 +2235,31 @@ addEnvHooks "$hostOffset" myBashFunction
</term> </term>
<listitem> <listitem>
<para> <para>
CC Wrapper wraps a C toolchain for a bunch of miscellaneous purposes. The CC Wrapper wraps a C toolchain for a bunch of miscellaneous purposes.
Specifically, a C compiler (GCC or Clang), wrapped binary tools, and a C Specifically, a C compiler (GCC or Clang), wrapped binary tools, and a C
standard library (glibc or Darwin's libSystem, just for the dynamic standard library (glibc or Darwin's libSystem, just for the dynamic
loader) are all fed in, and dependency finding, hardening (see below), loader) are all fed in, and dependency finding, hardening (see below),
and purity checks for each are handled by CC Wrapper. Packages typically and purity checks for each are handled by the CC Wrapper. Packages
depend on CC Wrapper, which in turn (at run time) depends on Bintools typically depend on the CC Wrapper, which in turn (at run-time) depends
Wrapper. on the Bintools Wrapper.
</para> </para>
<para> <para>
Dependency finding is undoubtedly the main task of CC Wrapper. This Dependency finding is undoubtedly the main task of the CC Wrapper. This
works just like Bintools Wrapper, except that any works just like the Bintools Wrapper, except that any
<filename>include</filename> subdirectory of any relevant dependency is <filename>include</filename> subdirectory of any relevant dependency is
added to <envar>NIX_CFLAGS_COMPILE</envar>. The setup hook itself added to <envar>NIX_CFLAGS_COMPILE</envar>. The setup hook itself
contains some lengthy comments describing the exact convoluted mechanism contains some lengthy comments describing the exact convoluted mechanism
by which this is accomplished. by which this is accomplished.
</para> </para>
<para> <para>
CC Wrapper also like Bintools Wrapper defines standard environment Similarly, the CC Wrapper follows the Bintools Wrapper in defining
variables with the names of the tools it wraps, for the same reasons standard environment variables with the names of the tools it wraps, for
described above. Importantly, while it includes a <command>cc</command> the same reasons described above. Importantly, while it includes a
symlink to the c compiler for portability, the <envar>CC</envar> will be <command>cc</command> symlink to the c compiler for portability, the
defined using the compiler's "real name" (i.e. <command>gcc</command> or <envar>CC</envar> will be defined using the compiler's "real name" (i.e.
<command>clang</command>). This helps lousy build systems that inspect <command>gcc</command> or <command>clang</command>). This helps lousy
on the name of the compiler rather than run it. build systems that inspect on the name of the compiler rather than run
it.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -2326,9 +2319,11 @@ addEnvHooks "$hostOffset" myBashFunction
<listitem> <listitem>
<para> <para>
The <varname>autoreconfHook</varname> derivation adds The <varname>autoreconfHook</varname> derivation adds
<varname>autoreconfPhase</varname>, which runs autoreconf, libtoolize <varname>autoreconfPhase</varname>, which runs autoreconf, libtoolize and
and automake, essentially preparing the configure script in automake, essentially preparing the configure script in autotools-based
autotools-based builds. builds. Most autotools-based packages come with the configure script
pre-generated, but this hook is necessary for a few packages and when you
need to patch the packages configure scripts.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -2372,9 +2367,9 @@ addEnvHooks "$hostOffset" myBashFunction
</term> </term>
<listitem> <listitem>
<para> <para>
Exports <envar>GDK_PIXBUF_MODULE_FILE</envar> environment variable the Exports <envar>GDK_PIXBUF_MODULE_FILE</envar> environment variable to the
the builder. Add librsvg package to <varname>buildInputs</varname> to builder. Add librsvg package to <varname>buildInputs</varname> to get svg
get svg support. support.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -2411,7 +2406,7 @@ addEnvHooks "$hostOffset" myBashFunction
PaX flags on Linux (where it is available by default; on all other PaX flags on Linux (where it is available by default; on all other
platforms, <varname>paxmark</varname> is a no-op). For example, to platforms, <varname>paxmark</varname> is a no-op). For example, to
disable secure memory protections on the executable disable secure memory protections on the executable
<replaceable>foo</replaceable>: <replaceable>foo</replaceable>
<programlisting> <programlisting>
postFixup = '' postFixup = ''
paxmark m $out/bin/<replaceable>foo</replaceable> paxmark m $out/bin/<replaceable>foo</replaceable>