Skip to main content

aen-sim-vision

Two minimal pipelines in one image — a vision path and a wake-word path — both running real TensorFlow Lite Micro inference on the Alif Ensemble E8 Cortex-M55-HP, driven by the studio hardware simulator via tan renode.

Source: examples/aen/aen-sim-vision/.

:::warning Sim-only, and read the placeholder notes below This example exists to exercise the simulator flow. There is no bench or HiL run behind it, the bundled model is a placeholder, and the wake-word decision is a threshold rather than a classifier. It is not a plain board.yaml project either — see How it is built. :::

The inference runs in software, not on the Ethos-U NPU​

This is deliberate, and the SDK is explicit about it: no NPU is used. The inference is the software path — TFLM on the M55, Helium-accelerated — which is exactly what a sim validates: the pipeline and the result, not the Ethos-U hardware dispatch.

The reasoning, from the SDK:

  • Renode ships no functional Ethos-U model. No Ethos-U55/U85 peripheral in the Renode device library executes the Vela command stream and produces real output tensors.
  • A stub NPU would be worse than none. Modelling only the registers and the "operation complete" IRQ, without computing, would make the firmware read back uncomputed output memory — the sim would display a wrong inference result, defeating the point of a functional simulator.
  • There is nothing to build a real model against. Arm's Ethos-U RTL and the Vela command-stream binary format are licensed/closed, so neither a Verilator co-simulation nor a faithful re-implementation is possible.

Real Ethos-U dispatch (the alp_inference NPU backend with a Vela-compiled model) is a bench/HiL concern, validated on real E8 silicon rather than in Renode. See aen-npu-inference for that path.

What it does​

A doorbell-polled loop, with studio (the sim host) writing into fixed shared-memory windows and ringing a doorbell:

PathInput windowWhat the firmware does
VisionSIM_FRAME — 1024 B, GRAY8 32×32Computes the frame's mean intensity, maps it to x ∈ [0, 2π], runs TFLM, renders y as a filled bar into the SSD1306 mono framebuffer.
Wake-wordSIM_AUDIO — 1024 B u8 PCMComputes mean absolute deviation from 0x80 (loudness), runs the same TFLM model for a confidence value.

TFLM is real: CONFIG_TENSORFLOW_LITE_MICRO=y, a MicroInterpreter, a 2048 B arena, and a MicroMutableOpResolver<1> carrying only AddFullyConnected().

:::caution Two honest caveats about the pipelines

  • The model is a placeholder sine model. It cannot classify anything.
  • The wake-word decision is a plain energy threshold, not the model. Because the sine model can't classify, a loudness gate stands in as the detector; the model's output is logged as conf_milli but does not gate the decision.
  • The vision path's "frame" feature is a single scalar (mean intensity), not a parsed camera frame descriptor. :::

How it is built​

This example has no board.yaml — it is a plain Zephyr app, not an orchestrator project, so it does not appear in the SDK's example catalog.

Build it ITCM-linked (Renode 1.16.1's Cortex-M55 mis-executes a high-MRAM vector table; ITCM at 0x0 sidesteps it):

west build -b alp_e1m_aen801_m55_hp/ae822fa0e5597ls0/rtss_hp \
    examples/aen/aen-sim-vision \
    -- -DEXTRA_DTC_OVERLAY_FILE=$PWD/tests/renode/aen_m55_itcm_run.overlay

tan renode --board E1M-AEN801 --image-bundle build

The sim platform is resolved by SKU from the SDK's Renode metadata; the .resc script is generated at runtime.

:::caution --sim-mode is wired but not yet ported tan renode --sim-mode — the studio hardware-simulator harness — is present for surface stability only and fails with renode.sim-mode-unported (tan-cli issue #674). Plain tan renode, as shown above, boots the built image under Renode without the studio injection wiring, so nothing writes the SIM_FRAME / SIM_AUDIO windows or rings the doorbell on that path. :::

:::note The sim platform description carries its own caveat The E8 .repl used here self-declares VALIDATION STATUS: UNVERIFIED — its addresses are sourced from SDK metadata and the upstream Zephyr Alif devicetree, but it had not been booted in Renode at authoring time. An advisory CI workflow is what proves it. :::

Console output​

The README documents no fixed expected-output block. The lines the firmware logs are:

aen-sim-vision: ready (frame=%p doorbell=%p display=%p audio=%p audio_doorbell=%p)
inference: frame_mean=%u y_milli=%d fill=%d
wakeword: %s energy=%u conf_milli=%d

See also​

Questions about this page? Discuss in Community Forum