The model you ship at launch is the worst model the product will ever run — or should be. OTA model updates turn a static embedded product into one that learns from its fleet and improves in the field.
This guide covers the mechanics: packaging models for OTA, staged rollouts, health checks, rollback, and the data flywheel that makes the whole thing worth building.
Models as data, not firmware
The enabling decision is architectural: store the model as a versioned blob in a dedicated flash region, separate from firmware. The runtime (TFLM interpreter, NPU driver) lives in firmware; the model is data the runtime loads — so model updates ride a small payload through your existing OTA channel instead of a full firmware reflash.
Versioning is metadata: model ID, version, checksum, minimum runtime version, and the input/output contract (feature shape, class order). A model whose feature pipeline doesn't match the firmware's is a brick; the contract check prevents it.
Staged rollout and health checks
Treat model rollout like firmware rollout — 1% of fleet, then 10%, then all — with health metrics gating each stage. For a model, health is behavioral: inference success rate, output sanity (class distributions in range, scores not saturated), and where possible, business metrics like false-alarm rate reported back.
Canary cohorts need representativeness: a rollout validated on dev units in the lab misses field diversity. Sample the canary group across hardware revisions, environments, and usage profiles.
- Model blob: versioned payload in dedicated flash partition
- Contract check: feature shape + class order vs firmware
- Staged rollout: 1% → 10% → 100% with health gates
- Rollback: previous blob kept as fallback image
Rollback: the non-negotiable
Every update mechanism needs a safe path back. Keep the previous model blob in a fallback slot, verify the new one (signature + boot-test inference) before committing, and auto-revert on failure — crash, inference timeout, or health-metric regression.
Same discipline as firmware OTA: A/B partitions or a known-good fallback image, signed payloads, and a boot check that runs one inference against a canned input before marking the slot active.
The field-data flywheel
The point of all this plumbing: field data flows back, models retrain, better models flow out. Devices log compact signals — anomaly events, low-confidence windows, feature distributions (not raw sensitive data) — upstream for analysis. Those logs become the next training set, closing a loop no launch-day model can beat.
This is where edge architecture pays double: the device infers locally and cheaply, while the fleet-level data shapes every future version. Products that build the flywheel early compound; products that treat the model as a fixed asset stagnate.
Frequently Asked Questions
How big is an OTA model update?
TinyML models are small by design — a 50–200 KB int8 model is a modest payload even over LPWAN-class links, trivial over BLE/Wi-Fi. Delta updates (binary diff against the previous blob) shrink it further when bandwidth is constrained. This is a genuine advantage over full-firmware updates.
What if the new model performs worse in the field?
That's what staged rollout and health metrics exist for — behavioral checks catch regressions on the canary cohort before fleet-wide damage, and the rollback path restores the previous blob automatically. Define health metrics before the first rollout, including a floor on basic output sanity.
Can I update the feature pipeline, not just the model?
Feature code usually lives in firmware, so changing it means a firmware OTA — which is why keeping the feature contract stable across model versions is a core design rule. Version the input contract explicitly and gate model rollout on contract compatibility.
How do I keep track of which model runs on which device?
Fleet inventory is table stakes: each device reports its model version in telemetry, dashboards show the version distribution across the fleet, and rollout tooling targets cohorts by version, hardware rev, and region. Without this, you can't stage, can't measure, and can't roll back selectively.
Building something that should run AI on-device?
Edgehound designs, compresses, and deploys TinyML models on microcontrollers — from feasibility audit to field-ready firmware.