Commit graph

135 commits

Author SHA1 Message Date
Robert Hensing afa6c51f27 lib: Use Nix's static scope checking, fix error message, optimize
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.

Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.

If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.

Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
2020-10-22 13:46:47 +02:00
zimbatm d8e4c8e612
Merge pull request #96641 from zimbatm/data-module-imports
nixos: Data module imports
2020-10-09 17:07:51 +00:00
zimbatm 035627dff2
lib: allow to import JSON and TOML files
The vision here is that configuration tools can generate .json or .toml
files, which can be plugged into an existing configuration.

Eg:

    { lib, ... }:
    {
      imports = [
        (lib.modules.importJSON ./hardware-configuration.json)
      ];
    }
2020-09-12 16:37:50 +02:00
Silvan Mosberger 1d4656225d
lib/types: Allow types to emit a deprecation warning
Previously the only way to deprecate a type was using

  theType = lib.warn "deprecated" (mkOptionType ...)

This caused the warning to be emitted when the type was evaluated, but
the error didn't include which option actually used that type.

With this commit, types can specify a deprecationMessage, which when
non-null, is printed along with the option that uses the type
2020-09-07 13:17:14 +02:00
rnhmjoj 20d491a317
treewide: completely remove types.loaOf 2020-09-02 00:42:50 +02:00
Maximilian Bosch fa30c9abed
lib/modules: improve error-message for undeclared options if prefix contains no options
An easy-to-make mistake when declaring e.g. a submodule is the accidental
confusion of `options` and `config`:

    types.submodule {
      config = {
        foo = mkOption { /* ... */ };
      };
    }

However the error-message

  The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.

is fairly unhelpful because it seems as the options are declared at the
first sight. In fact, it took a colleague and me a while to track down such
a mistake a few days ago and we both agreed that this should be somehow caught
to save the time we spent debugging the module in question.

At first I decided to catch this error in the `submodules`-type directly
by checking whether `options` is undeclared, however this becomes fairly
complicated as soon as a submodule-declaration e.g. depends on existing
`config`-values which would've lead to some ugly `builtins.tryExec`-heuristic.

This patch now simply checks if the option's prefix has any options
defined if a point in evaluation is reached where it's clear that the
option in question doesn't exist. This means that this patch doesn't
change the logic of the module system, it only provides a more detailed
error in certain cases:

  The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.

  However it seems as there are no options defined in [definition 1-entry 1]. Are you sure you've
  declared your options properly? This happens if you e.g. declared your options in `types.submodule'
  under `config' rather than `options'.
2020-08-18 15:25:26 +02:00
Silvan Mosberger d5700d626c
lib/modules: Fix nonexistant option error
The refactoring in fd75dc8765
introduced a mistake in the error message that doesn't show the full
context anymore. E.g. with this module:

  options.foo.bar = lib.mkOption {
    type = lib.types.submodule {
      baz = 10;
    };
    default = {};
  };

You'd get the error

  The option `baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.

instead of the previous

  The option `foo.bar.baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.

This commit undoes this regression
2020-08-18 00:12:36 +02:00
Silvan Mosberger 42cf8130d7
lib/modules: Add syntactic sugar for config._module.freeformType
This introduces `freeformType` as a top-level module attribute, allowing
definitions like

  {
    freeformType = ...;
    options = ...;
    config = ...;
  }
2020-08-14 22:49:04 +02:00
Silvan Mosberger e0ded8f4ba
lib/modules: Fix freeform modules when there's no definitions 2020-08-10 17:27:33 +02:00
Silvan Mosberger 65e25deb06
lib/modules: Implement freeform modules
For programs that have a lot of (Nix-representable) configuration options,
a simple way to represent this in a NixOS module is to declare an
option of a type like `attrsOf str`, representing a key-value mapping
which then gets generated into a config file. However with such a type,
there's no way to add type checking for only some key values.

On the other end of the spectrum, one can declare a single separate
option for every key value that the program supports, ending up with a module
with potentially 100s of options. This has the benefit that every value
gets type checked, catching mistakes at evaluation time already. However
the disadvantage is that the module becomes big, becomes coupled to the
program version and takes a lot of effort to write and maintain.

Previously there was a middle ground between these two
extremes: Declare an option of a type like `attrsOf str`, but declare
additional separate options for the values you wish to have type
checked, and assign their values to the `attrsOf str` option. While this
works decently, it has the problem of duplicated options, since now both
the additional options and the `attrsOf str` option can be used to set a
key value. This leads to confusion about what should happen if both are
set, which defaults should apply, and more.

Now with this change, a middle ground becomes available that solves above
problems: The module system now supports setting a freeform type, which
gets used for all definitions that don't have an associated option. This
means that you can now declare all options you wish to have type
checked, while for the rest a freeform type like `attrsOf str` can be
used.
2020-08-03 22:37:00 +02:00
Silvan Mosberger fd75dc8765
lib/modules: Internally collect all unmatched definitions
This fundamentally changes how the module evaluation internally
handles definitions without an associated option.

Previously the values of these definitions were discarded and only
the names were propagated. This was fine because that's all that's
needed for optionally checking whether all definitions have an
associated option with _module.check.

However with the upcoming change of supporting freeform modules,
we *do* need the values of these.

With this change, the module evaluation cleanly separates definitions
that match an option, and ones that do not.
2020-08-03 22:37:00 +02:00
Silvan Mosberger b02a3d7b08
lib/modules: Scope module evaluation variables more tightly
This is a purely cosmetic change so it's easier to see dependencies
between variables.
2020-08-03 22:37:00 +02:00
Silvan Mosberger 910dfdc41e
lib/modules: Evaluate single defs for readOnly error
If multiple definitions are passed, this evaluates them all as if they
were the only one, for a better error message. In particular this won't
show module-internal properties like `_type = "override"` and co.
2020-09-21 18:24:52 +02:00
Silvan Mosberger bdfcee2590
lib/modules: Improve error messages using showDefs 2020-09-21 18:24:52 +02:00
Silvan Mosberger 9eecf2d057
Revert "lib/modules: Throw better error when definitions assign to an option set"
This reverts commit 15c873b486.

This was causing infinite recursion when depending on nested options
2020-03-19 03:50:15 +01:00
Silvan Mosberger e931de58a2
lib/modules: Fix type checks not being done before merging
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
2020-03-18 04:38:53 +01:00
Silvan Mosberger 15c873b486
lib/modules: Throw better error when definitions assign to an option set 2020-03-18 04:38:50 +01:00
Silvan Mosberger dcdd232939
lib/modules: Remove internal _module attribute from config
The _module option is added as an internal option set, and it messes up
the results of module evaluations, requiring people to manually filter
_modules out.

If people depend on this, they can still use config._module from inside
the modules, exposing _module as an explicitly declared user option. Or
alternatively with the _module attribute now returned by evalModules.
2020-03-17 19:19:39 +01:00
Robert Hensing 9c0ab2f26d lib/modules.nix: Add file context to unmerged values in mergeDefinitions
This helps with troubleshooting exceptions in config values, which were hard
to track down for options with many definitions.
The trace will look like:

    error: while evaluating the attribute 'config.foo' at undefined position:
    [...]
    while evaluating the option `foo':
    [...]
    while evaluating definitions from `/home/user/mymod.nix':
    while evaluating 'dischargeProperties' at /home/user/nixpkgs/lib/modules.nix:464:25, called from /home/user/nixpkgs/lib/modules.nix:392:137:
    while evaluating the attribute 'value' at /home/user/nixpkgs/lib/modules.nix:277:44:
    Value error!

where the `/home/user/mymod.nix` module is

    { lib, ... }: {
      options.foo = lib.mkOption {
        type = lib.types.lines;
      };
      config.foo = builtins.throw "Value error!";
    }
2020-02-24 00:15:36 +01:00
Janne Heß 790cd012d0
nixos/lib: Inherit type for doRename options
Co-authored-by: Silvan Mosberger <contact@infinisil.com>
2020-01-20 22:06:20 +01:00
Silvan Mosberger 9e97e64847
lib/modules: Switch _module.args from attrsOf to lazyAttrsOf 2020-01-10 16:20:31 +01:00
Silvan Mosberger 130a0c9878
lib/modules: Move the isDefined check into mergedValue
Without this change, accessing `mergedValue` from `mergeDefinitions` in
case there are no definitions will throw an error like

  error: evaluation aborted with the following error message: 'This case should never happen.'

This change makes it throw the appropriate error

  error: The option `foo' is used but not defined.

This is fully backwards compatible.
2020-01-10 16:19:54 +01:00
Silvan Mosberger e0ea5f4d9b
lib/modules: Fix store imports
This fixes imports from the store not being possible, which was caused by
https://github.com/NixOS/nixpkgs/pull/76857

E.g. such a case:

  imports = [ "${home-manager}/nixos" ];
2020-01-10 04:13:28 +01:00
Silvan Mosberger e9c16ec186
Merge pull request #76857 from Infinisil/recursive-disableModules
Apply `disabledModules` recursively
2020-01-09 18:20:12 +01:00
Silvan Mosberger de5f73d434
lib/modules: Recursive disabledModules
With this change, disabledModules applies recursively, meaning if you
have a module "foo.nix" with

    imports = [ ./bar.nix ];

then setting

  disabledModules = [ "foo.nix" ];

will disable both "foo.nix" and "bar.nix", whereas previously only
"foo.nix" would be disabled.

This change along with https://github.com/NixOS/nixpkgs/pull/61570 allows
modules to be fully disabled even when they have some `mkRenamedOption`
imports.
2020-01-09 17:26:05 +01:00
Silvan Mosberger b46776d14e
Clarify error message of 'assigning to top-level attribute' (#76702)
Clarify error message of 'assigning to top-level attribute'
2020-01-08 17:36:43 +01:00
Arnout Engelen 43ef3a8d00 lib/modules: clarify error message of 'assigning to top-level attribute'
If I understand correctly, the problem isn't so much that you're assigning to
that top-level attribute, but that the assignment to the attribute (or any
child of the attribute) introduces the 'config' object and prevents 'lifting'
all settings to a generated 'config' object.
2020-01-05 10:22:32 +01:00
Silvan Mosberger 5414b4018b
lib/modules: Don't pack submodules specially
This has the beneficial side effect of allowing paths to be used as modules in
types.{submodule,submoduleWith}
2020-01-01 01:13:03 +01:00
Silvan Mosberger 3cc77ce756
lib/modules: Make unifyModuleSyntax fully idempotent
Because why not
2019-12-05 05:51:44 +01:00
Silvan Mosberger aa613427b7
lib/modules: file -> _file for a more idempotent unifyModuleSyntax
This will be useful for doing more complicated module evaluations
2019-12-05 05:51:44 +01:00
Chuck 4ded9beea2 Add note: Keep error message in sync with nixos-option 2019-11-04 15:11:45 +01:00
Silvan Mosberger 2b1e2f2e97
Merge pull request #69746 from Infinisil/rem-opt-usage-message
lib.mkRemovedOptionModule: Show replacement for option usage too
2019-10-02 23:11:41 +02:00
Robin Gloster b08b0bcbbe mkRemovedOptionModule: assert on removed options
We don't want to ignore config that can mess up machines. In general
this should always fail evaluation, as you think you are changing
behaviour and don't, which can easily create run-time errors we can
catch early.
2019-09-30 12:07:13 +02:00
Silvan Mosberger ebb136da9f
lib.mkRemovedOptionModule: Show replacement for option usage too
Previously mkRemovedOptionModule would only show the replacement
instructions when the removed option was *defined*. With this change, it
also does so when an option is *used*.

This is essential for options that are only intended to be used such as
`security.acme.directory`, whose replacement instructions would never
trigger without this change because almost everybody only uses the
option and isn't defining it.
2019-09-28 04:10:22 +02:00
Silvan Mosberger de9cb24938
lib/modules: Use options apply function even if no values are defined
This allows `apply` functions to return a valid value if they completely
ignore their argument, which is the case for the option renaming
functions like `mkAliasOptionModule`. Therefore this solves issue #63693
2019-08-10 00:56:56 +02:00
Danylo Hlynskyi 60e8fcf0e5
module system: revert "remove types.optionSet", just deprecate (#56857)
The explicit remove helped to uncover some hidden uses of `optionSet`
in NixOps. However it makes life harder for end-users of NixOps - it will
be impossible to deploy 19.03 systems with old NixOps, but there is no
new release of NixOps with `optionSet` fixes.

Also, "deprecation" process isn't well defined. Even that `optionSet` was
declared "deprecated" for many years, it was never announced. Hence, I
leave "deprecation" announce. Then, 3 releases after announce,
we can announce removal of this feature.

This type has to be removed, not `throw`-ed in runtime, because it makes
some perfectly fine code to fail. For example:
```
$ nix-instantiate --eval -E '(import <nixpkgs/lib>).types' --strict
trace: `types.list` is deprecated; use `types.listOf` instead
error: types.optionSet is deprecated; use types.submodule instead
(use '--show-trace' to show detailed location information)
```
2019-03-07 21:28:09 +02:00
Silvan Mosberger d3216be6d9
Merge pull request #54528 from cdepillabout/module-alias-uses-priority
lib/modules: Change mkAliasOptionModule to use the priority for the alias
2019-02-23 16:43:05 +01:00
danbst 27982b408e types.optionSet: deprecate and remove last usages 2019-01-31 00:41:10 +02:00
danbst aa2e63ce5e lib/modules.nix: small eval optimization (foldl' + foldl' + attrNames -> foldl' + mapAttrs) 2019-01-30 15:26:44 +02:00
(cdep)illabout 81fa1e392b lib/modules: Change mkAliasOptionModule to use the priority for the alias.
This commit changes the `mkAliasOptionModule` function to make sure that
the priority for the aliased option is propagated to the non-aliased
option.

This also affects the `mkRenamedOptionModule` function in a similar
fashion.

This also removes the `mkAliasOptionModuleWithPriority` function, since
its functionality is now subsumed by `mkAliasOptionModule`.

This change was recommended by @nbp:
https://github.com/NixOS/nixpkgs/pull/53397#discussion_r245487432
2019-01-24 13:02:16 +09:00
(cdep)illabout f24e2d0721
Pull out defaultPriority to a top-level definition. 2019-01-06 17:48:37 +09:00
(cdep)illabout b81b3ad1b0
lib/modules: Add a function to create an option alias that respects the priority
This commit adds a function `mkAliasOptionModuleWithPriority`.  This
function will make an alias to an existing option and copy over the
priority.

This functionality is needed for PRs like #53041.  In that case
`nixos-generate-config` added an option to `hardware-configuration.nix`
with `mkDefault`.  That option was then changed and an alias created for
the old name.

The end user should be able to set the non-alias option in their
`configuration.nix` and have everything work correctly.  Without this
function, the priority for the option won't be copied over correctly
and the end-user will get a message saying they have the same option
set to two different values.
2019-01-04 18:35:10 +09:00
Symphorien Gibol 526d604670 module system: rework module merging
The asymptotic complexity is now much lower.
2018-08-27 17:11:58 +02:00
volth 87f5930c3f [bot]: remove unreferenced code 2018-07-20 18:48:37 +00:00
xeji 249be1c560
Merge pull request #42138 from NixOS/yegortimoshenko-patch-6
lib/modules: decrease mkOptionDefault priority to 1500
2018-06-27 20:29:39 +02:00
Yegor Timoshenko 441796cb16
lib/modules: bump mkOptionDefault priority to 1500 2018-06-17 23:29:16 +03:00
Jan Malakhovski 449d43fe01 lib: fix and simplify doRename
Before this change `mkRenamedOptionModule` would override option defaults
even when the old option name is left unused. For instance

```nix
{
  optios = {
    services.name.new = mkOption {
      default = { one = {}; };
    };
  };
  imports = [
    (mkRenamedOptionModule [ "services" "name" "old" ] [ "services" "name" "new" "two" ])
  ];
  config = {};
}
```

would evaluate to

`{ config.services.name.new = { two = {}; }; }`

when you'd expect it to evaluate to

`{ config.services.name.new = { one = {}; }; }`.
2018-06-11 15:06:27 +00:00
Jan Malakhovski c0c43e9c07 lib: simplify mkAliasAndWrapDefinitions 2018-06-11 15:06:27 +00:00
Matthew Justin Bauer 165c151f7a
Merge pull request #34805 from rycee/fix/dorename
lib: make use of visible variable in doRename
2018-05-14 18:08:26 -05:00
Jan Malakhovski 4017fdcafd lib: modules: propagate highestPrio
Yeah, it's ugly. But it's the minimal change that doesn't break anything
else.
2018-05-12 19:27:09 +00:00