oddlama's blog

Tracking NixOS option values and dependencies

There are thousands of options in NixOS, but as users, we usually only interact with a select few of them. Despite that, a huge amount of those options does influence the final result in some way. Have you ever wondered which of them were actually relevant for your specific system?

If you are now asking yourself why you would need this information, think about what happens when you enable a random service, say services.immich.enable = true. Do you get new systemd services? A database maybe? Some nginx reverse proxy configuration? It's not super easy to tell without looking at the NixOS module definition. But even then, imagine the module is updated at some point. How do you know what will change when you update? Okay, you can dig through the git diff in nixpkgs and find out, too. But this is getting harder. Finally, think about what might happen when you decide to bump system.stateVersion. This requires careful consideration, as it may break your system. How do you know which of your services would be affected?

Now, if we just had more information about options and their values, we could immediately answer all of these questions. So this is exactly what I built.


TL;DR: By adding dependency tracking primitives to the Nix evaluator and hooking them into the NixOS module system, we can track which options depend on which other options during a real NixOS evaluation. This reveals the full dependency graph of your configuration, allows diffing option values between different system versions, and answers questions like "what would changing system.stateVersion actually affect on my system?"

Link to the GitHub repository: nixos-config-tui

A minimal example

Have a look at this minimal NixOS configuration.nix. It only sets options that are absolutely necessary so we can evaluate the result:

{
  boot.loader.grub.device = "nodev";
  fileSystems."/" = {
    device = "/dev/sda1";
    fsType = "ext4";
  };
  system.stateVersion = "25.11";
}

Just four options were used, right? Well, at the time of writing this post, a NixOS system built from this configuration evaluates a whopping 6293 option values that all could have an immediate impact on the resulting system toplevel if they were changed. Most of these options are either the default defined in one of the ~1100 NixOS modules or depend on the value of other options (e.g., enabling nextcloud adds a systemd service, user, group, ...).

Don't take that number of options too seriously though, the exact value depends a lot on what you count as an option. The current tracking implementation doesn't know anything about "options" in the sense of what is created by calling mkOption. Instead it just traces accesses to anything and everything that is part of the fixpoint evaluation, no matter whether it is an attribute from options, config, pkgs or something else. The value 6293 comes from heavily filtering the list of all accesses to roughly match option boundaries introduced by mkOption, but my filter is certainly not perfect. Counting all attribute accesses we would arrive at roughly 33.1k values that play a role somehow, with over 150k dependencies between them. But that number includes a lot of things internal to the NixOS module evaluator.

If we were generally able to track which options influence which other options, we could gather all values that contribute to the toplevel and build a dependency graph of the entire evaluation. Rendering that graph for the above NixOS configuration (and adding some colors for the top-level option paths), looks like this:

A graph showing the dependencies between NixOS option values

There's a lot to unpack here. Since this was originally graphed in 3D, the screenshot doesn't really do it justice. If you want to explore this yourself, you can download the corresponding .dot file and import it into graphia. We can observe several bigger clusters that get influenced by a lot of smaller nodes:

  • environment.etc (the top lobe)
  • systemd.services on the left hand side pulling in many of the service enable options in blue
  • The toplevel iself (bottom lobe) pulling in some services a lot from users.users and users.groups
  • Accesses to pkgs (light blue block on right)
  • suid wrappers (the pink-ish block on the bottom right)
  • warnings (on the back side, not visible in this picture)
  • and assertions (green blob in the middle)

In any case, this is still only counting values that are actually used during evaluation! As you might know, most options are gated behind an .enable flag of some service or more generally some kind of mkIf in a module. Those will not be shown as they are not contributing to the result (changing them would have no effect).

We already have nix-diff

and that's great! nix-diff is an awesome tool which I use a lot to see what has changed in terms of derivations or file content. This usually gives enough information to reason about whether changing an option had a relevant effect on the resulting system. But the diff can become unwieldy, especially when doing a larger system update with many package upgrades. Then, even if the NixOS options didn't change at all, the new package versions will cause thousands of changes to the derivation chain due to new store hashes.

To gain a better understanding of what really changed semantically, we need to be able to diff option values directly on the config level. I want to know whether the definition of my services.nginx.virtualHosts has changed and what caused the change, not just the effect on the resulting config file.

Tracking values at the configuration-level and tracking changes at the derivation level with nix-diff are fundamentally different tasks, and which one is better depends a lot on what you want to analyze.

Exploring and diffing configurations

Without going too much into detail, I want to give you a rough overview of what is happening. The big challenges here are that this tracking must work without adjusting any code in the NixOS modules, otherwise tracking support would need to be added to each module individually. And secondly performance... My prior attempts at implementing this were so atrocious, I aborted evaluation after half an hour of waiting. So now it works like this:

  1. A patch for nix adds a few new primitives and code to the evaluator which allows us to remember why and in which context a thunk was evaluated (in functional languages, a thunk is a value that is yet to be evaluated). We don't interfere with how this happens, we only observe when it happens. So later we still know what caused a value to be evaluated.
  2. A slight change to the NixOS module system will enable tracking for values accessed when the configuration is evaluated. The main part of the module system is a kind of fixpoint iteration where the final configuration value is calculated and can depend on itself. This is why you can access the final config values in your configuration while you are technically still writing the configuration. Nix automatically evaluates this self-recursive construct until the result is stable.
  3. The resulting list of what got accessed by what can be queried via another added builtin and will by default be written to a JSON file tracking-deps.json in the toplevel, next to the activation script. This ensures we don't lose the information after the evaluation has finished and the system is deployed.
  4. Now, we also know all configuration values that were relevant for our system, so we serialize that into a secondary JSON file tracking.json. And finally to make our lives a bit easier later on, we add a preprocessed variant of this JSON tracking-explicit.json which only includes values that were set explicitly, i.e. are not the default value implied by the mkOption definition.

To enable all of this in a NixOS configuration, all that needs to be done is to set trackDependencies = true when calling lib.nixosSystem (and use the patched nix version of course):

{
  nixosConfigurations.host1 = nixpkgs.lib.nixosSystem {
    trackDependencies = true; # ← enable tracking
    system = "x86_64-linux";
    modules = [ ./configuration.nix ];
  };
}

To make exploration and diffing these JSON files simple, I've included a utility called nixos-config which can be used to interpret the tracking information and option values. If you remember the minimal configuratoin from the beginning, we can now explore it by calling it on the toplevel path using

nixos-config show /nix/store/iq7v04w69321k9pgp3zysszwgyaz9s4m-nixos-toplevel-tracked

The toplevel can also be given using a flake reference like nixos-config show .#host1 which will have the advantage that the resulting system doesn't need to be built, but it requires the configuration source. In both cases, this will open a TUI interface:

A TUI showing an explorable and structured view of a NixOS configuration, including option dependencies and dependents

In this TUI above we can see that system.stateVersion was only accessed by system.build.toplevel and nothing else, so we immediately know that no services depend on that value in our configuration.

So let's modify our configuration and add a service which does depend on system.stateVersion. I've chosen immich, a popular self-hosted photo library which requires postgresql. And postgresql depends on system.stateVersion to avoid system breakage when major versions are bumped.

{
  boot.loader.grub.device = "nodev";
  fileSystems."/" = {
    device = "/dev/sda1";
    fsType = "ext4";
  };
  system.stateVersion = "25.11";

  services.immich.enable = true;
}

Now let's look at the same option in the TUI of the new configuration:

A TUI showing system.stateVersion of the same NixOS configuration but with immich enabled

By looking at the reverse dependencies, it immediately becomes clear that postgresql's package version depends on system.stateVersion! The TUI also features an integrated diff, so if we want to explore exactly what has changed between the two configurations, we can invoke nixos-config diff OLD NEW and will see something like this for our example:

A TUI showing the diff between two configurations

Unless the filter mode is changed, the TUI will show only options that changed their value (or were newly evaluated or removed) in comparison with the original configuration. I've focused services.postgresql here, which itself is not actually a value that is directly accessed, so it shows no dependencies. But you can clearly see that three services got added. We can now dig in and see that services.postgresql.enable newly depends on the value of services.immich.database.enable, which is the reason why it got enabled.

A TUI showing the diff between two configurations, focusing on services.postgresql.enable

If that is too detailed and you just want to see the diff between two configurations as pseduo configuration.nix files, there's nixos-config text-diff. In explicit mode, it will only show values that are not set by mkOption defaults, so a large diff becomes digestible:

A TUI showing a textual diff between two pseudo nix configurations, derived from their effective option values

Try it yourself

If you want to try this on your own NixOS configuration, you'll need the patched Nix evaluator and a patched nixpkgs which integrates this into the module system. You do not need to upgrade or change your system's nix daemon. Since all changes are in expression evaluation, it suffices to run the patched nix CLI. I've already prepared everything in my forks so you can just follow a few simple steps:

  1. Enter a shell with the the patched nix binary and the nixos-config utility:

    nix shell github:oddlama/nix/thunk-origins-v1 github:oddlama/nixpkgs/thunk-origins-v1#nixos-config
  2. Define a host using the patched nixpkgs repo and set trackDependencies = true. An example flake is attached below, but of course you can also temporarily switch to my nixpkgs fork in your actual configuration. Just beware that I have to manually rebase the branch sometimes, so it will probably be tracking something in-between master and nixos-unstable.

    # flake.nix
    {
      inputs.nixpkgs.url = "github:oddlama/nixpkgs/thunk-origins-v1";
      outputs = { self, nixpkgs }: {
        nixosConfigurations.host1 = nixpkgs.lib.nixosSystem {
          system = "x86_64-linux";
          trackDependencies = true;
          modules = [{
            boot.loader.grub.device = "nodev";
            fileSystems."/" = {
              device = "/dev/sda1";
              fsType = "ext4";
            };
            system.stateVersion = "25.11";
          }];
        };
      };
    }

    If you don't want to use flakes, you can also clone https://github.com/oddlama/nixpkgs to ./nixpkgs, enter the thunk-origins-v1 branch and call eval-modules.nix directly:

    direct evaluation (click to expand)
    # toplevel.nix
    let
      nixpkgs = import ./nixpkgs { };
      lib = nixpkgs.lib;
    in import ./nixpkgs/nixos/lib/eval-config.nix {
      inherit lib;
      trackDependencies = true;
      modules = [{
        boot.loader.grub.device = "nodev";
        fileSystems."/" = {
          device = "/dev/sda1";
          fsType = "ext4";
        };
        system.stateVersion = "25.11";
      }];
    }
  3. Build the tracked configuration to store the tracking information permanently. Beware that this evaluation will take about 10-20 seconds longer than what you are used to.

    nix build --print-out-paths .#nixosConfigurations.host1.config.system.build.toplevel
    # remember the output path
  4. Change the configuration slightly and rebuild the tracked configuration. to store the tracking information permanently

    # edit the configuration and rebuild:
    nix build --print-out-paths .#nixosConfigurations.host1.config.system.build.toplevel
    # remember the output path
  5. Explore a single configuration or diff the configs:

    # explore one of the built toplevels from before
    nixos-config show /nix/store/xyz-toplevel-tracked
    # show from ad-hoc evaluation (flakes only)
    nixos-config show .#host1
    # diff two toplevels
    nixos-config diff /nix/store/abc-toplevel-tracked /nix/store/def-toplevel-tracked
    # diff two toplevels, hide options values that are defaults
    nixos-config diff --explicit /nix/store/abc-toplevel-tracked /nix/store/def-toplevel-tracked
    # diff two toplevels textually, hide options values that are defaults
    nixos-config text-diff --explicit /nix/store/abc-toplevel-tracked /nix/store/def-toplevel-tracked

The raw information from the tracking (values, dependencies, graphviz, ...) is also available in .#nixosConfigurations.host1.dependencyTracking, after forcing the evaluation of the toplevel with buildins.seq.

The rest of the blog post is about implementation details, so you can skip ahead to the Conclusion if you are not interested in that.

Previous attempts

I've had three attempts at implementing option tracking, and I failed twice before arriving at something that worked. In case you are interested in how that went, I've summarized them here briefly.

Attempt 1: Full evaluation graph. The simplest idea was to track every single primitive operation during evaluation. Every attribute access, every function application, every + or //. This would give a complete evaluation trace from which dependencies could be extracted. Simple to implement but horrid performance. I aborted evaluation for a basic NixOS config after 30 minutes and it was nowhere near done. The sheer number of operations in a full NixOS evaluation makes this approach completely impractical.

Attempt 2: Global access-tracking attrsets. The next idea was more targeted: only track attribute accesses on specific attrsets (like config). Basically, when you call foo.bar, foo would record that bar was accessed in a global map. A second builtin would set an "accessor ID", a global variable indicating who is currently being evaluated.

Performance was ok-ish and I didn't spend time optimizing it at all. But I was already unsatisfied with the approach because of two problems. First, the global map is impure. Once evaluation is done, any further access to config, even just printing the result, would record additional spurious dependencies. Second, and more fundamentally: because Nix evaluates thunks lazily, the "current accessor" at the time a thunk is actually forced might be completely wrong. If thunk A triggers the evaluation of thunk B (which accesses config.x), the access gets attributed to A, not to B where the code actually lives. As soon as more complex expressions were involved, this approach would fail to attribute accesses to the correct scope.

Attempt 3: Thunk-embedded origins. So third time's the charm apparently, haha. I wanted a generic and pure implementation this time, so it can reliably track things regardless of thunk evaluation order or access order. This is the approach that worked out in the end. I'll glance go over the implementation below.

Thunk-embedded origins

Instead of using a global variable to track who is being evaluated, we embed the origin information directly in each thunk.

When a thunk is created, we tag it with its identity which is the option path it represents. When that thunk is later forced by Nix, it pushes its own identity as the "current accessor" onto a context stack. Any attribute accesses on tracked attrsets during that evaluation are then correctly attributed to this thunk. When the evaluation finishes, the context is popped from the stack.

This has two important properties:

  • It respects laziness. Dependencies are only recorded when values are actually forced. Unforced thunks contribute nothing.
  • It correctly attributes accesses to where they are written, not where they are triggered. If thunk A causes thunk B to be evaluated, accesses during B's evaluation are attributed to B, because B pushes its own origin onto the stack. This matches how you would think about the code when reading it - the accessor is determined at definition time, not at force time.

To see where this matters we need to look at an non-trivial example. Basically imagine there is an option that is a function. I'll omit option definitions here for brevity.

{config, ...}: {
  nproc = 10;
  add = a: b: a + b;
  addNproc = config.add config.nproc;
  nprocPlus2 = config.addNproc 2;
}

Here we have a mix of values and accesses between them. There are constants (nproc), a function add which evaluates two parameters and uses their value, addNproc which creates a new function with only one parameter by accessing the add function and applying our constant and finally a result nprocPlus2 which calls this second function.

The NixOS module evaluator would make sure each of these is tagged with its ["nproc"], ["add"], ["addNproc"], and ["nprocPlus2"] respectively. They are lists so we can represent nested attribute paths. Now, when something forces nprocPlus2, we need to make sure that accesses are attributed at the location of definition not the location where the value is forced. So what happens is:

  1. The thunk for nprocPlus2 is forced. It pushes ["nprocPlus2"] as the accessor context.
  2. It accesses config.addNproc → dependency recorded: nprocPlus2 → addNproc
  3. Accessing addNproc forces its thunk, which pushes ["addNproc"] as the new accessor context.
  4. Inside addNproc, config.add is accessed → dependency: addNproc → add
  5. Now config.nproc is accessed inside the add function. Notice that nprocPlus2 does not directly depend on nproc. That access happened inside addNproc's evaluation, so it was correctly attributed to addNproc.
  6. addNproc finishes, pops its context.
  7. Back in nprocPlus2, the function application returns 12.
  8. nprocPlus2 finishes, pops its context.

So the recorded dependencies are:

nprocPlus2 → addNproc
addNproc   → add
addNproc   → nproc

New builtins

Disclaimer: I am not very familiar with the nix codebase and the patch was partly generated by Claude. So this should really be inspected by someone with a deeper knowledge of nix to verify that I didn't make a grave mistake somewhere. Do NOT use this in production, while all internal tests do pass, I can't guarantee that I didn't break some hidden invariant.

The implementation adds several new builtins to Nix, designed for a two-phase setup that works at least with the NixOS module system's evaluation order. I tried make the design as generic as possible, so it should work with all kinds of fixpoint evaluations.

  1. builtins.createTrackingScope creates a new, independent tracking scope and returns its ID. The scope exists independently from any attrset, allowing it to be created before the tracked config is built. This is important for breaking the circular dependency in the module system, where config depends on option values that need to be tagged with the scope ID.
  2. builtins.registerTrackedAttrset scopeId attrset associates an attrset with an existing scope. Any attribute accesses on this attrset (or sub-attrsets encountered during evaluation) will be recorded as dependencies.
  3. builtins.tagThunkOrigin scopeId path thunk tags a thunk with an origin path. When that thunk is later forced, it pushes the path as the accessor context. The thunk is returned unchanged (it is not forced).
  4. builtins.tagAttrThunkOrigin scopeId path attrName attrset is similar, but tags a specific attribute's thunk directly within the attrset's internal Bindings structure. This avoids creating a wrapper thunk, which is important to remove internal .value intermediaries added by the NixOS module evaluator.
  5. builtins.getDependencies scopeId returns all recorded dependencies for a scope as a list of { accessor = [...]; accessed = [...]; } records.
  6. builtins.tryCatchAll expr is a helper similar to tryEval, but catches all evaluation errors including missing attributes, type errors, and abort. It also deeply evaluates its argument before returning.

Using these primitives, a simple tracking example looks like this:

let
  fix = f: let x = f x; in x;
  scopeId = builtins.createTrackingScope;
  config = fix (self: {
    base = 10;
    helper = x: x + self.base;
    impl = self.helper 42;
  });
  _ = builtins.registerTrackedAttrset scopeId config;
  # Force impl to trigger evaluation
  _ = builtins.seq config.impl true;
in builtins.getDependencies scopeId
# Result:
# [ { accessor = ["impl"];   accessed = ["helper"]; }
#   { accessor = ["helper"]; accessed = ["base"];   } ]

Edge cases

There are surprisingly many edge cases that need to be taken care of to ensure we get a properly connected dependency graph (and I'm certain that I still missed a ton more). Let me go over some of them.

Auto-registration of sub-attrsets

We need tracking to automatically propagate into nested attrsets. When a tagged thunk is forced and produces an attrset, the evaluator should automatically register that attrset and tag its member thunks (without overwriting any existing explicit tags). This allows accessing config.services.nginx.enable to work correctly even though only the top-level config was explicitly registered. The intermediate attrsets services and nginx are registered on-the-fly as they are encountered during evaluation.

Additionally, ExprSelect::eval (the code path for a.b.c attribute selections) records dependencies at each intermediate step, not just for the final path. Without this, sub-attrset accessor nodes would become disconnected in the dependency graph. For example, boot.kernelPackages.kernel.features would not show edges for the intermediate boot.kernelPackages and boot.kernelPackages.kernel nodes.

Issues with wrapper thunks

One subtle issue I ran into during the nixpkgs integration is that the NixOS options tree contains option records which are attrsets with attributes like .value, .isDefined, .type, etc. We want to tag each option's .value thunk with the option path (e.g., ["users" "users"]), so that when the value is forced, accesses are attributed to that option.

My first approach was something like:

opt // { value = builtins.tagThunkOrigin scopeId loc opt.value; }

But this creates a new attrset with a wrapper thunk for the value attribute. The wrapper thunk is the expression { value = tagThunkOrigin ... } itself. When the auto-registration later encounters this attrset and tags its member thunks, the wrapper gets tagged with ["opt" "path" "value"], but we wanted ["opt" "path"] without the trailing "value". Otherwise, dependency paths look like users.users.value.nixbld11 instead of the correct users.users.nixbld11.

The fix was to add builtins.tagAttrThunkOrigin, which tags the actual Value* stored in the attrset's internal Bindings structure. Since auto-registration preserves existing tags (getThunkOrigin check), the correct origin is set before auto-registration ever sees it.

Feels a bit hacky though.

Integration with the NixOS module system

The nixpkgs changes are pretty minimal, most of my changes are related to graphviz generation. Only lib/modules.nix and the eval-config files have relevant changes. And importantly, none of the actual NixOS module had to be modified.

When evalModules is called with trackDependencies = true:

  1. A tracking scope is created before config is built (two-phase setup to break the circular reference between scope ID and config).
  2. tagOptionsRecursive walks the merged options tree. At each option node, tagAttrThunkOrigin tags the .value attribute's thunk with the option's path.
  3. Both config and options attrsets are registered as tracked.
  4. When any option value is forced during evaluation, its origin path becomes the accessor context, and any accesses to other tracked attributes are recorded.
  5. The result includes _dependencyTracking.getDependencies to retrieve all recorded edges.

Here's how the tagOptionsRecursive function looks:

tagOptionsRecursive = pfx: opts:
  mapAttrs (name: opt:
    let loc = pfx ++ [ name ];
    in if isOption opt
       then builtins.tagAttrThunkOrigin trackingScopeId loc "value" opt
       else tagOptionsRecursive loc opt
  ) opts;

It walks the options tree and whenever it finds an option node (recognized by _type = "option"), it tags the .value attribute's thunk in-place with the option path. Non-option attrsets are recursed into.

Additionally, eval-config.nix wraps the output so that config.system.build.toplevel transparently includes tracking data as JSON files. The inner toplevel is evaluated first via builtins.seq (which records all dependencies), and then the tracking data is serialized alongside the system closure.

Filtering out some of the noise

The raw dependency data contains over 150k edges for the basic configuration from the beginnging, and includes access to a lot of the internal attributes created by the NixOS module system, or come from accesses to _module.args.pkgs which are not actually options but appear because the pkgs specialArg refers to config._module.args.pkgs and config is tracked wholistically.

Other noise comes from type checking, value merging, resolving priorities, and so on. A post-processing step in dependency-tracking.nix filters this down to something more meaningful, but I know there's still some unnecessary stuff left over.

Config value serialization

Beyond the dependency graph, the system also serializes the actual config values for all "leaf" options. Before we can save that as JSON though, we need to filter out some of the stuff that cannot be serialized into JSON like derivations, functions and paths which are replaced with <derivation:name>, <function> and <path:...> placeholders.

Each value is wrapped in builtins.tryCatchAll to safely handle evaluation errors. Quite often there is stuff that cannot be evaluated in a representable form, for example assertions or warnings. The messages in there often contain string interpolations like "${fileSystems'.cycle}" that throws missing-attribute errors when the assertion passes, as the message can only be safely evaluated when the assertion fails. Unfortunately, tryEval cannot catch these, but tryCatchAll can.

Limitations

There's definitely stuff I missed. If you are trying this out yourself and happen find a bug or something interesting, please open an issue, or just send it to me directly on Matrix. I'll probably gather a bunch of issues for a follow-up at some point.

List accesses

Accesses inside list literals are not fully tracked. When you write [ config.x ], the list elements become thunks that are forced after the tracking context for the surrounding expression has been popped. This doesn't occur super often in NixOS though, so I didn't implement this properly yet.

No full provenance tracking

Originally I planned to include full provenance information, because ideally I'd like to know the actual file, line and column where the value for a specific option came from. Sounds reasonably simple at first to just attach that information, but when the thunk origin information is added by our new builtins, I didn't find a way to access this provenance information. For example when a literal like 42 is parsed, it quickly becomes a plain integer without any way to attach additional information. So the source location is already lost.

Adding this would be another bigger change in nix, and I didn't want to spend more time on this for now.

Conclusion

I found it quite interesting to see just how many options are actually involved in building a NixOS configuration. Diffing NixOS configurations on the configuration level itself is something I wanted to have for a long time, and I hope this serves well as a Proof-of-concept, showing that it is both possible and feasible! It does come with a noticeable performance impact, but only if you actually enable trackDependencies = true.

I'm sure there are many ways in which this information could be used that I haven't thought of yet, so please share your ideas!

But there is more work to be done. As outlined before, I would love to have proper provenance that tells me exactly where values were defined, and apart from the known limitations there are certainly still bugs and edge-cases that need to be found. But even in its current state, I've found the tracking data to be genuinely useful for understanding how my NixOS configurations change over time.


If you'd like to send me feedback or just reach out, feel free to contact me on Matrix!

Discussion threads for this post: