Authoring a Nerves system by hand

I wanted to run Elixir on a CompuLab IOT-DIN-IMX8PLUS — an industrial gateway that lives on a DIN rail in a cabinet, takes 24 V, and is expected to sit there for years without anyone looking at it. There is no published Nerves system for this board. There is one for a sibling board from the same vendor, which is close but not the same hardware.

So the first real decision was what to do about that, and I want to explain it before any of the technical detail, because everything afterwards follows from it.

Fork it, or write it#

The fast option is to fork the sibling system and change what looks wrong. The slow option is to author every file myself — nerves_defconfig, the kernel config, the U-Boot config, fwup.conf, the rootfs overlay — using the sibling system as read-only reference and the actual hardware documentation as the source of truth.

I chose the slow option. Not out of principle: a fork hides which values are deliberate and which are inherited. When a board does not boot, that distinction is the whole debugging problem. If I have written each line, then every value is one I can defend or change. If I have forked, then half the file is somebody else's answer to a slightly different question, and I cannot tell which half.

That decision cost me a week and it was correct. Parts two through four of this series are almost entirely me discovering things about my own files. Every one of those discoveries would have been harder to reason about had I inherited the file.

The board#

Built around NXP's i.MX8M Plus: four Cortex-A53 cores, a Cortex-M7 for real-time work, and an NPU nobody is using yet. 4 GB of LPDDR4 on the system-on-module. That last number matters much more than it looks like it does — see part four.

Console is /dev/ttymxc1 at 115200 8N1, on the micro-USB labelled DBG. Storage is a 32 GB eMMC at /dev/mmcblk2, with the two 4 MB hardware boot partitions that turn out to matter a great deal.

What a Nerves system actually consists of#

This is the part that is hard to picture until you have written one. A Nerves system is not a program — it is a Buildroot configuration plus the handful of files that tell Buildroot what hardware it is targeting:

nerves_system_iotdin_imx8p/
├── nerves_defconfig              # the Buildroot config: what goes in the image
├── mix.exs                       # platform (nerves_system_br), arch (aarch64/cortex_a53)
├── linux/
│   ├── linux-6.6.defconfig       # the kernel config — 1463 lines
│   └── 0001-squashfs-*.patch     # required for a read-only rootfs
├── uboot/
│   ├── uboot.defconfig           # the bootloader config
│   └── uboot.env                 # the default U-Boot environment
├── fwup.conf                     # how a firmware image is laid out and written
├── fwup_include/
│   ├── fwup-common.conf          # partition layout, A/B slots
│   ├── provisioning.conf
│   └── fwup-ops.conf             # factory reset, revert, validate, status
└── rootfs_overlay/etc/
    ├── erlinit.config            # which console the Erlang VM talks on
    ├── fw_env.config             # where U-Boot's environment lives
    └── boardid.config            # serial-number source for the hostname

Every one of those files is a decision about this board. That is why forking hides so much: a fork gives you twenty files full of someone else's answers, and no way to tell which were deliberate.

The kernel config, and one that would have bitten me#

The kernel defconfig is 1463 lines, and I did not write it from nothing. I based it on CompuLab's own in-tree compulab_v8_defconfig at the pinned 6.6.23 commit, plus a documented 19-line Nerves addendum.

The interesting part is what I didn't use. There is also a kernel config fragment in the vendor's Yocto meta-layer, at 6.6.3, and it looks like the obvious starting point. I diffed both before choosing, and the fragment was missing several drivers this board needs:

CONFIG_USB_NET_QMI_WWAN     # the LTE modem's QMI interface
CONFIG_TI_ADS1015           # the ADC expansion module
CONFIG_RTC_DRV_ABX80X       # the on-board AB1805 real-time clock

Starting from the fragment would have produced a system that booted, looked fine, and quietly had no modem and no ADC. The in-tree v8 config already enables all of those, along with USB_SERIAL_FTDI_SIO, USB_SERIAL_OPTION, GPIO_PCA953X with IRQ support, CAN_FLEXCAN, MWIFIEX_SDIO, and TCG_TIS_I2C_INFINEON for the TPM.

My 19-line addendum adds only what Nerves itself needs and the base was missing:

CONFIG_F2FS_FS=y            # the expanding application-data partition
CONFIG_USB_WDM=y            # /dev/cdc-wdm0, the QMI control channel
CONFIG_SQUASHFS_ZSTD=y      # the read-only rootfs

The lesson: diff the candidate configs before you pick one. Two plausible starting points differed by exactly the drivers that would have been hardest to debug as absent, because a missing driver does not error — the device simply never appears.

The U-Boot config, and a placeholder that was quietly wrong#

uboot/uboot.defconfig is based on CompuLab's authoritative configs/iotdin-imx8p_defconfig at the pinned commit. Taking it verbatim and adding on top caught a real defect: the file I had started from carried a sibling board's target:

- CONFIG_TARGET_IOT_GATE_IMX8PLUS=y
+ CONFIG_TARGET_IOTDIN_IMX8P=y

Along with the right device tree:

CONFIG_DEFAULT_DEVICE_TREE="imx8mp-iotdin"
CONFIG_OF_LIST="imx8mp-iotdin"

A wrong target symbol does not fail to build. It builds a bootloader for a board you do not have.

The divergence I chose deliberately#

The factory keeps the U-Boot environment in the eMMC hardware boot partition mmcblk2boot0, at offset 0x3f0000. I moved it to the user area, at the same byte offset:

# rootfs_overlay/etc/fw_env.config
/dev/mmcblk2 0x3F0000 0x20000 0x200 256

Not because the factory is wrong, but because Nerves' whole OTA model assumes fwup manages the user area. Keeping the env there means fwup and fw_setenv agree about where it lives, and A/B updates work the standard way. The cost is a documented divergence from factory that I have to remember — which is exactly why it is written into the file itself and not just into my head.

The thing I had wrong, and had to correct#

This one is worth stating plainly because I built a whole plan on the wrong assumption.

fwup does not write imx-boot, and never touches the eMMC boot partitions. It writes only the user area, /dev/mmcblk2.

The bootloader is flashed to mmcblk2boot0 separately, once, on the initial bring-up — and OTA never touches it again. I had assumed a firmware update carried the bootloader with it, and had started sketching how fwup would set the eMMC ext_csd boot-enable bit as part of an upgrade.

None of that work was needed. Boot-partition activation is a one-time bring-up concern, not a per-update one. Realising that deleted a whole class of imagined problem — and it also means an OTA update can never brick the bootloader, which is a reassuring property to get for free.

The partition layout#

/dev/mmcblk2   (user area, 29.2 GB)
 ├─ MBR
 ├─ U-Boot env      raw @ block 8064 (0x3F0000), 128 KB
 ├─ rootfs A        512 MB   ┐ squashfs, read-only
 ├─ rootfs B        512 MB   ┘ the inactive one receives the update
 └─ app data        f2fs, expands to fill the rest

/dev/mmcblk2boot0  (4 MB)  imx-boot — written once by uuu, never by OTA
/dev/mmcblk2boot1  (4 MB)  redundant copy

Standard Nerves A/B: an update writes the inactive slot and reboots into it, and a bad image reverts automatically. Application data survives because it is a separate partition.

Where this leaves things#

Every file authored, every version pinned, nothing built yet. The next post is the first build — which did not work, for reasons that were entirely my own and took a while to see.