From a lab unit to a real production Nerves system

The first series of posts on this was all about one disposable lab unit, kept deliberately wide open, with nothing depending on it, so I could make every possible mistake without any real consequence. That was the right way to actually learn this, but it's not the end goal.

The real production devices I work with don't run a generic Linux distribution — they run a purpose-built embedded Linux system, built specifically for this hardware and this application, with its own kernel configuration and its own bootloader build already tuned for the job. That system already works well today, and it's running in the field right now. So the second half of this project is about carrying everything I learned in the first series over into that real system — without ever touching the currently-working version while I do it.

The rule I'm setting for myself before this even starts

I'm not going to touch the existing, working build at all. All of this work happens in a separate copy — a fork, essentially — of that system, so that if anything goes sideways during this experiment, the thing that's actually running today is completely unaffected. Only once the new, secured version genuinely proves itself does it become a candidate to actually replace anything.

And just like the first series, I'm doing all of this on the very same lab unit the first series used — not the units that are actually deployed and doing real work. Once this second series proves the whole chain end to end, on hardware that's explicitly fine to experiment on, converting anything that's actually in the field becomes its own separate decision, made deliberately, later, not a side effect of this project.

Why this genuinely needed its own series

The production system does things noticeably differently from the simple setup in series one — most importantly, where it keeps its startup configuration. That system needs its startup settings to live somewhere it can safely update over the network as part of normal operation, which is a very reasonable thing for a fleet of deployed devices to need. But it means those settings live in a different, more exposed place than in the simple lab setup from series one — which turns out to matter quite a bit for how a properly locked-down bootloader configuration needs to work. I ran into this exact detail while researching the first series and it changed a design decision for this second one before I'd even started writing this post — which I think says something about why it's worth its own dedicated writeup, rather than a quick footnote at the end of series one.

Where things actually stand right now

Update: this series has started for real. Series one's actual secure boot flow got proven, cleanly, on real hardware, tamper tests included — including one honest limit I found along the way and explain properly in that series' own closing post, about exactly what does and doesn't get proven without permanently locking anything down. Once that was settled, this series picked up right where this post left off: forking the production system, never touching the working original, on the same disposable lab unit. The next post covers the fork itself, and a first real milestone — a signed bootloader that I could actually put on the board and watch pass a real hardware check.