We’re going a little overboard here again. I’ve spent the last six months reorganizing my entire infrastructure, or rather, organizing it properly for the first time, because until now it was just a loose collection of servers and devices.

0x01 MOTIVATION

As I’ve mentioned several times here, and as I regularly bring up elsewhere, I run a number of devices and servers on NixOS. Unfortunately, my previous setup was rather pieced together, with every machine having its own configuration and no consistent structure. The downside was that changes, fixes, updates, etc. all had to be carried out manually on each individual machine, making the whole setup difficult to maintain, let alone keep organized. Over time, I’ve accumulated various ThinkPads, Raspberry Pis, ThinkCentres, and VPS instances, all running NixOS and somehow fulfilling their respective tasks. However, NixOS is perfectly suited for describing infrastructure as code, making maintenance and management significantly easier. So my goal over the past six months has been to build my own independent infrastructure that is easy to maintain and can scale easily. Once that is in place, hosting your own services and taking the path towards digital independence becomes considerably easier. There are other benefits, too. I get to learn about integrating a wide variety of hardware architectures and services I’ve never encountered before, as well as flakes, Home Manager, and so on. With centralized infrastructure management, everything can be version-controlled using Git. And thanks to NixOS, the entire setup is reproducible anyway. But now let’s take a look at the tools and mechanisms holding this infrastructure together.

0x02 TECHNOLOGIES USED

NixOS is used as the base operating system. For those unfamiliar with NixOS, it follows a purely declarative and reproducible approach and uses a rollback system that forgives errors and allows you to return to a previous working generation at any time to fix configuration issues. In this case, reproducibility allows us to build the entire infrastructure using centralized packages that can be shared across different nodes. To keep the configuration from becoming unnecessarily complex and difficult to manage, two additional tools are being introduced into the infrastructure. I had deliberately avoided them for quite some time, simply because I hadn’t had the time to properly explore them: Flakes and Home Manager. NixOS Flakes provides the central configuration file, flake.nix, and pins the exact versions of all dependencies. The second tool is Home Manager. With Home Manager, the entire user environment can be configured declaratively via Nix. This allows us to create and deploy all necessary user-specific settings directly across all nodes. That way, we can provide a consistent configuration for ZSH, XMonad, Git, browsers, Helix, and so on. The actual centralized deployment is handled by Colmena. Colmena takes on the central role of managing multiple nodes and devices. This allows us to update, repair, or further expand any reachable target directly from the central deployment machine, without having to manually connect to each computer via SSH and perform everything by hand. Later on, we’ll have multiple nodes authorized to deploy the infrastructure via Colmena. More on that further down. The other tools being used are familiar ones that are already in use anyway. Git for version control, SSH for connecting to individual nodes, and BorgBackup to back up all nodes and prevent data loss. Last but not least, we keep an eye on everything using Grafana, Prometheus, and a few Alloy instances, allowing us to monitor everything that needs to be monitored and should always be running. The basic tools are now clear. Next up is planning the infrastructure architecture itself.

0x03 ARCHITECTURE

This architecture has evolved over an extended period and has been changed several times along the way. Fundamentally, the architecture is divided into two different areas: hardware and software. We’ll start with the software side. The hardware will follow in the next section. The basic infrastructure is structured as follows:

assets
colmena
flake.nix / flake.lock
images
modules
nodes
secrets
users

Within assets, you’ll find files that exist outside the actual infrastructure code but are still part of it. These include wallpapers and fonts that are supposed to be deployed across all systems within the structure. Inside colmena, we have our hive.nix. This file defines our entire Colmena infrastructure, as well as all the components important to our setup. This is where we define things such as Home Manager, sops, snm, stylix, disko, and similar tools. It is also where each individual node receives its exact definition:

1CA-EX-00 = {
2    deployment.targetHost = "0.0.0.0";
3    imports = [
4        ../nodes/CA-EX-00
5        sopsNix.nixosModules.sops
6        snm.nixosModules.mailserver
7    ];
8};

Here, for example, the node CA-EX-00 is configured as a VPS and receives two additional modules that we define via flake.nix. In flake.nix, we take care of the inputs we want to include in our infrastructure, such as the SNM mail server on CA-EX-00 from the previous example. Since we also want to be able to add new nodes directly from our Colmena hive, this is where the integration of Disko and nixos-anywhere is handled, as well as the creation of Raspberry Pi images. As mentioned earlier, everything is pinned via flake.lock. To initially deploy different platforms, we have images. This directory contains the Nix files for the initial deployment on platforms such as x86, ARM, and so on. My infrastructure now includes various ARM devices, x86 systems, RPis, uConsoles, and similar devices, all running different systems. These modules are kept relatively small and are only intended to provide a minimal initial base system on each individual node. Once a node has been added to the infrastructure, Colmena takes care of everything else. The next major component of our infrastructure is modules. The modules directory contains all the services, programs, and other mechanisms we want to use in our infrastructure. Modules is divided into base, hardware, networking, roles, services, and shared. Let’s start at the beginning. base contains the foundation that should be identical across all nodes. It provides all the necessary basic configurations that are essential for the nodes. These include SSH, users, BorgBackup, and a small visual gimmick called motd.nix, which displays a small information window in the terminal showing which node you’ve connected to via SSH. In addition, base handles the build of Home Manager modules for each user, the time zone, and whether we want to use sops on the node. It also explicitly defines which sops secrets file belongs to each respective node. With base, we already have a system that boots and can be accessed via SSH.

Under hardware, we add a few exceptions to the base configuration. For example, a NixOS deployment on a Raspberry Pi 4 requires some additional options to make the system run properly on the Raspberry Pi 4.

networking currently contains configurations for my DIY Raspberry Pi 4 Wi-Fi access points. Here, we need to precisely control when the USB mode switch for the Wi-Fi adapters starts, what hostapd needs to do on the Pis, and so on. In other words, everything required for the access points.

Another major component is roles. Colmena allows us to assign individual roles to nodes. Let’s take Nextcloud as an example. The role is defined in nextcloud.nix. Within this role, we define all the components required for a Nextcloud instance: nextcloud, nginx, postgresql, and the corresponding sops secrets from the previously mentioned sops secrets file assigned to the respective node. This allows us to build rules and assign them to nodes later, rather than putting individual large, difficult-to-monitor configuration blocks on the nodes. We don’t just do this for individual services, but also for window managers, development hosts, Fail2Ban, firewalls, Grafana, keyboard layouts, and so on. Essentially, this is the heart of the infrastructure.

Next come the services. In services, we define individual tools. The easiest example here is Karakeep, a wonderful RSS feed collector. We create the service karakeep.nix, which contains everything Karakeep needs to run. Since Karakeep provides its own web interface, we also create a website_karakeep.nix under websites. This contains all the NGINX parameters and virtual hosts required by Karakeep. We then combine these two modules into a role under /role/karakeep.nix. This allows us to assign the role to a node later, giving that node everything it needs to host Karakeep. If something ever breaks in Karakeep, we can use this separation to specifically identify where the problem lies and fix it.

Finally, we have shared. This is where the connections between assets (which include fonts and wallpapers) and our infrastructure are collected. These are individual Nix files defining where our fonts and wallpapers should be placed on the new systems. Additionally, I’ve included themes in shared, ensuring that Dracula is available across all systems.

Next on the list is nodes. Nodes is the part of our infrastructure that connects the individual nodes with their services, tools, window managers, users, and everything else. Within nodes, each individual machine receives its own specific configuration.

This definition is implemented through the provided default.nix. In default.nix, we import all the roles and services we want on the node. We also specify which sops file belongs to the node, which key can be used to decrypt it, and which individual parts of the sops files should be used.

Last but not least, we define which network name the machine receives and make any necessary bootloader adjustments. This allows us to assemble our nodes from individual modules and roles.

The final two components are secrets and users. secrets contains .yaml files holding the passwords and keys for services and users in the corresponding format. The only thing to pay attention to here is how a service expects its key to be provided: encrypted or in cleartext.

users gets a little more complex again. This is where we assemble our user-specific configurations and define our default editor, among other things.

0x04 HARDWARE

Let’s take a look at the hardware distribution within my infrastructure. I’ll keep this section short, since none of the nodes contain particularly impressive hardware and are mostly standard devices that are readily available and affordable almost everywhere. Nevertheless, they belong here, because the actual combination of individual modules and roles depends on the individual nodes. The naming convention is fairly simple. Since we’re operating within the Cortex Hive, all nodes are Cortex Agents, or CA for short, followed by an identifier indicating their purpose and a sequential number. This gives us the following node groups within my infrastructure:

CA-IF-0x - IF stands for Interfaces, covering laptops, desktop computers, and the uConsole. In other words, everything used actively while working in front of a screen. In my home, these are ThinkPads and ThinkCentres. CA-RI-0x - RI stands for Rack Interfaces. These are also Tiny ThinkCentres, but they run headless and are located here locally in a small DIY mini rack. CA-EX-0x - EX stands for external machines. In my case, these are Manitu VPS instances and a storage box. CA-AC-0x - AC describes Wi-Fi access points running NixOS and based on the Raspberry Pi 4. They use an external Wi-Fi module and are monitored directly through Grafana.

Whether, how, and when additional devices will be added remains to be seen. For now, however, these categories are more than sufficient for me.

0x05 SELF-HOSTING / SERVICES

To minimize our dependence on large service providers and reduce the risk of our data being used for purposes we haven’t agreed to, we naturally prefer to host our own stuff. There are already a few things running in the hive that make everyday life easier. For quickly transferring data or simply making it available online, I use Nextcloud. It’s easy to integrate, and clients are available for almost every end device. RSS feeds make up a large part of my information consumption. Since TTRSS is morally impossible for me to support, I’m still running FreshRSS as a feed aggregator. This keeps read entries and inboxes synchronized across devices. You often come across things on the internet that you want to read again later (or at least that’s what you tell yourself). That’s what Karakeep is for, serving as a read-it-later service. My old VPS used to run Synapse as a Matrix server. Even though Matrix often does Matrix things, it still needs to be integrated into the infrastructure again. This time, however, I’m using Continuwuity. The great thing about it is that it runs without complaining. And if you don’t feel like having to check first whether the software you’re using is operated by some fascist tech company, you can be sure that this isn’t the case here. Of course, BorgBackup runs on all nodes and backs up everything important that really shouldn’t be lost. Databases, configurations, data, and so on. The usual stuff. SNM, or simple nixos mailserver, is also a necessity, because every project naturally needs its own email address and because family emails run through it as well. Often overlooked because it simply runs in the background, we also have NGINX, which takes care of serving the websites.

0x06 MONITORING / BACKUPS

Once all services and mechanisms have been implemented and everything is running, things can quickly get boring. Most of the time, you only need to touch the infrastructure when something new is added or something breaks. One thing that helps fight the boredom is building an extensive monitoring setup, so you can at least watch your machines doing their thing. Initially, I used Beszel as a simple monitoring system. Beszel runs centrally as a hub (including the web interface, etc.), while a corresponding agent runs on every device that needs to be monitored. Nevertheless, I decided against Beszel. For one thing, it wasn’t 100% reliable in my case. Sometimes connected agents were shown as unreachable even though a manual check confirmed that they should have been accessible. On top of that, the dashboard design doesn’t make optimal use of the available space, with scrollbars appearing where there should actually be enough room. Apart from that, Beszel is a great tool for simple monitoring. However, the design requirements eventually brought me back to Grafana. Grafana can be integrated into the infrastructure just as easily as any other module. We need a server running Grafana, including the dashboards, and a Prometheus exporter on each node to provide us with the metrics. The external VPS instances are no problem. Prometheus can simply be installed on them, and they can then be integrated into Grafana as data sources. The locally hosted nodes are a little more complicated, since they don’t have fixed static IP addresses and are constantly changing. However, we still want to include them in our monitoring setup. That’s why we’re building a small collector that sends the Prometheus metrics directly to Grafana instead of relying on Grafana to retrieve them. This also allows us to monitor and keep an eye on machines such as laptops and desktop computers. Another important topic is backups. Thanks to Borg, however, this is nowhere near as complicated as you might think. Each node gets its own Borg instance. Currently, Borg sends the backup for each node via SSH to the Hetzner Storage Box. In the future, however, this will also be done locally.

0x07 DEPLOYMENT AND MAINTENANCE WITH COLMENA

Now we get to the actual core of the entire infrastructure: Colmena. Colmena takes care of defining and distributing the individual nodes. To provide Colmena with the necessary infrastructure definitions, we create a file called hive.nix. In this file, we define all hosts that belong to the infrastructure and should be deployable via Colmena. We provide each host with its IP address, the path to its node configuration (../nodes/CA-IF-00), and the modules we want on that node. These are usually ‘sopsNix.nixosModules.sops’, along with additional modules such as ‘snm.nixosModules.mailserver’ if a mail server is supposed to be configured on that node. The advantage is that we no longer have to build the configuration on the actual node itself. Instead, we can do everything on the node that has been designated as authorized for deployment. Once we’ve finished the module, node, or whatever else we want to add to the infrastructure, we commit everything to the Git repository:

1git add . && git commit -m "change"

After that, we have two options for deploying with Colmena. We can either deploy an individual node using:

1colmena apply --on CA-IF-00

Or, for general changes, we can deploy to all nodes using:

1colmena apply

After that, all packages are built and the configuration is deployed to the individual node or to all nodes. If you have errors in your code, Colmena will show them to you before deploying anything to the target system. Deployment only takes place once validation has completed without errors. That’s all there is to it. Everything is nicely organized and easy to monitor.

0x08 DESIGN AND USER ENVIRONMENT

Of course, looks matter too. As always, I want DRACULA as the global color scheme across all machines so I don’t have to sit in front of colorless terminals. NixOS offers a tool called stylix for global color scheme management. What used to be handled through .Xresources on other distributions can now be taken care of by stylix. However, Stylix goes a few steps further and also passes the colors on to the bootloader and GRUB, for example. The only small exception here is XMonad. I’ve defined its colors directly in xmonad.hs. There is an option to organize everything using variables, but I haven’t reached that point in my configuration yet. Colors are one thing. Dotfiles are another. Naturally, I want my helix to have the same keybindings and behave the same way on every machine. The same applies to tmux, alacritty, zsh, and so on. As described earlier, this is where Home Manager comes into play and takes care of everything. The advantage is that I can define window managers and desktop environments on a truly user-specific basis. If someone else logs into the infrastructure, they can immediately use their own familiar window manager (XFCE in this case).

0x09 CONCLUSION AND FUTURE PLANS

And with that, we’ve reached the provisional end of my infrastructure. But that’s rarely the actual end. There are still quite a few services running at large tech corporations where you don’t really want your stuff to be. The best example of this is local storage. At current prices (as of 2026), buying several terabytes of hard drives is simply not justifiable. Unfortunately, a lot of other things depend on local storage as well, such as a local media library, NAS, or local backups of all machines. As long as storage prices don’t come down, it’ll probably be a while before these modules find their way into my infrastructure. But there is other interesting software, too. My own Mastodon instance, so that the next migration will only be necessary when I no longer feel like hosting it myself. Pi-hole as a black hole for everything that makes the internet unreadable and serves nothing but consumerism, aka keeping advertising out of the home network. A self-hosted password manager solution is also still on the to-do list. I also wanted to try running a local NixOS caching server, such as Attic or Harmonia. That way, all binaries would be available directly from the local network, and we would no longer depend on the NixOS caching server (even though it is very reliably accessible). Another interesting project would be setting up my own (hardware) firewall for the local network based on NixOS, simply for the learning experience. As you can see, there are still plenty of individual building blocks that can be integrated here, ensuring that the fun never ends. I deliberately haven’t gone into individual code examples, as this article is initially intended to provide a general overview of the possibilities and mechanisms behind a NixOS infrastructure. There will certainly be more posts covering individual modules with concrete code examples to recreate. If you’d already like to see some practical examples, Daniel is currently putting out some good NixOS content over at zenzes.me. Go check it out.

FORM ALLIANCES - SHARE SERVICES - HACK THE PLANET