TinyML: Machine Learning on a Microcontroller Budget

The microcontroller in a smoke detector, a washing machine or a factory sensor has a few hundred kilobytes of memory, runs at a fraction of a phone's clock speed, and might be expected to last years on a coin cell. TinyML is the practice of running machine-learning models on that hardware — not a scaled-down version of cloud AI, but a distinct engineering discipline where the budget is measured in kilobytes and microwatts. It's how "Hey Siri" wakes your phone without draining it, and it's spreading into billions of devices that have never had a network connection.

Why bother

Microcontrollers outsell every other kind of processor combined — tens of billions a year. Putting even simple intelligence on them changes what a device can do without a cloud, a phone or a gateway:

• Always-on sensing at negligible power. A model that listens for a keyword or watches for a vibration signature can run continuously for months on a battery, waking more expensive hardware only when something happens.
• Privacy by architecture. Audio and images processed on the chip are never stored or transmitted. The device outputs "glass broke", not a recording.
• Zero latency, zero connectivity dependence. Anomaly detection on a pump in a remote station works with no network at all.
• Cost. A microcontroller costs cents to a few dollars. Adding a model to a device that already has one is nearly free at the bill-of-materials level.

Typical applications

• Keyword spotting and voice commands — the canonical example, in phones, earbuds, appliances and cars.
• Predictive maintenance — accelerometer and current-draw models on motors, pumps and bearings that flag drift from normal.
• Gesture and activity recognition from inertial sensors in wearables, tools and controllers.
• Low-resolution vision — person presence for lighting and HVAC, occupancy counting, simple defect detection — on image sensors with sub-milliwatt budgets.
• Environmental sensing — classifying gas mixtures, detecting acoustic events (glass, alarms, coughs), monitoring crops and livestock.
• Health devices — arrhythmia detection on a patch, fall detection on a pendant.

The hardware

Targets range from 32-bit Arm Cortex-M microcontrollers (M4, M7, M33 and the newer M55 with vector extensions) and RISC-V equivalents, through chips with dedicated micro-NPUs (Arm Ethos-U, and offerings from Syntiant, Ambiq and others), to specialist ultra-low-power inference chips. Typical resources: 64 KB–1 MB of RAM, a few hundred KB to a few MB of flash, clock speeds from tens to a few hundred MHz, and power budgets from microwatts (always-on) to tens of milliwatts (active inference). There is no operating system to speak of, no floating-point unit on the cheapest parts, and no dynamic memory allocation if you value your sanity.

Making a model fit

The workflow starts with a model trained normally on a workstation, then compressed aggressively:

• Architecture choice. Small convolutional networks, depthwise-separable convolutions, tiny recurrent or attention layers. Model size is designed in, not optimised out.
• Quantisation to 8-bit integers is essentially mandatory — it cuts memory 4× and lets inference run on integer-only hardware. Quantisation-aware training recovers most of the accuracy lost.
• Pruning removes low-value weights and channels, and distillation from a larger model often yields a smaller network with better accuracy than training the small one directly.
• Operator fusion and static memory planning by the compiler so the model runs without allocation and within a fixed RAM budget.

The toolchain has consolidated: TensorFlow Lite for Microcontrollers (now LiteRT) is the reference runtime; Arm's CMSIS-NN provides optimised kernels; Edge Impulse offers an end-to-end platform from data collection to deployed firmware; and compilers such as Apache TVM, and vendor tools like STM32Cube.AI and NXP's eIQ, generate hardware-specific code. Benchmarking through MLPerf Tiny lets you compare chips honestly.

Design realities

The model is often the easy part. The harder engineering:

• Data collection on the actual sensor and enclosure — a model trained on clean lab audio fails on a microphone behind a plastic grille.
• Power budgeting across sensing, inference and radio, with duty cycling so the model runs only when a cheap threshold suggests it should.
• Over-the-air updates for models and firmware, securely signed, with rollback — because the model will need to change and the device will be in a wall.
• Monitoring without data. You can't upload every inference. Aggregate confidence statistics and periodic sampled uploads are how you know a fleet's models still work.
• Security. A model on a device can be extracted; a device with a network can be attacked. Secure boot and encrypted model storage matter — see IoT Security.

Where it's going

Micro-NPUs are pushing the ceiling upward — small vision transformers and even tiny language models are appearing on high-end microcontrollers. Energy-harvesting devices (solar, vibration, RF) combined with TinyML promise sensors with no battery at all. And the boundary with edge AI on more capable hardware keeps shifting as what was once a gateway workload fits on a chip. The constant is the discipline: know your memory budget, your power budget and your sensor, and design the model for all three.