<alp/inference.h> — NPU Dispatcher

A single API for running TFLite Micro models on the right silicon-specific NPU back-end.

Supported back-ends

Back-end	Build gate	Available on
Arm Ethos-U55	CONFIG_ALP_TFLM_ETHOS_U55	Every E1M-AEN SKU (E3 / E4 / E5 / E6 / E7 / E8) — two instances per SoC
Arm Ethos-U85	CONFIG_ALP_TFLM_ETHOS_U85	E1M-AEN401 / AEN601 / AEN801 (Alif E4 / E6 / E8 only) — one instance per SoC, Transformer-capable, generative-AI forward path
Arm Ethos-U65	CONFIG_ALP_TFLM_ETHOS_U65	E1M-N93 family (NXP i.MX 93)
Renesas DRP-AI3	CMake option ALP_SDK_USE_DRPAI_V2N (default OFF) — not a Zephyr Kconfig symbol	E1M-X V2N family
DEEPX DX-M1	CMake option ALP_SDK_USE_DEEPX_DXM1 (default OFF)	E1M-X V2N-M1 family (non-TFLM runtime — DXNN)
CPU	(fallback, always available)	Any target — reference kernels

Which Ethos-U dispatchers compile in is silicon-determined — the SDK's loader reads the active SoM preset's capabilities: block and emits CONFIG_ALP_SDK_INFERENCE_* for every dispatcher the silicon can drive (TFLM CPU always; Ethos-U per caps). DRP-AI3 and DEEPX DX-M1 are gated by CMake options instead — see Linux-only NPU backends below. The customer's board.yaml does not carry an inference.backend: knob.

On the i.MX 93 (N93) family the Ethos-U65 backend registration is gated by CONFIG_ALP_SDK_INFERENCE_BACKEND_ETHOS_U_N93 — note the BACKEND_ infix; the symbol is not CONFIG_ALP_SDK_INFERENCE_ETHOS_U_N93.

Apps pick a backend per handle at runtime via alp_inference_open(.backend = ALP_INFERENCE_BACKEND_AUTO | _CPU | _ETHOS_U | _DRPAI | _DEEPX_DXM1). AUTO picks the highest-priority match for the active SoM (order: U85 → U65 → U55 → DRP-AI → DEEPX DX-M1 → CPU). On V2M101 (V2N-M1) the dispatcher set carries both DRP-AI3 and DEEPX DX-M1; apps can open two handles and dispatch concurrently to either NPU.

The U55 / U85 split is driven by the loader's requires_cap matcher reading each SoM preset's capabilities: block — pick the SKU, the right CONFIG_* follows automatically.

Header

#include <alp/inference.h>

Quick example

extern const uint8_t my_model_tflite[];
extern const size_t  my_model_tflite_len;

alp_inference_t *infer = alp_inference_open(&(alp_inference_config_t){
    .backend         = ALP_INFERENCE_BACKEND_AUTO,
    .model_bytes     = my_model_tflite,
    .model_len       = my_model_tflite_len,
    .tensor_arena_kb = 256,
});
if (infer == NULL) {
    int err = alp_last_error();
    return err;
}

// Get input tensor metadata
alp_tensor_t input;
alp_inference_get_input(infer, 0, &input);
memcpy(input.data, my_image, input.byte_size);

// Run
alp_inference_invoke(infer);

// Read output
alp_tensor_t output;
alp_inference_get_output(infer, 0, &output);
// process output.data ...

alp_inference_close(infer);

Opening a handle

alp_inference_open() error set

alp_inference_open() returns NULL on failure and stamps alp_last_error() with one of:

ALP_ERR_INVAL, ALP_ERR_NOT_PRESENT_ON_THIS_SOC, ALP_ERR_NOT_IMPLEMENTED, ALP_ERR_NOSUPPORT, ALP_ERR_NOMEM, ALP_ERR_IO.

arena == NULL is rejected for Ethos-U models

A model that dispatches to Ethos-U with cfg->arena == NULL now returns ALP_ERR_INVAL at open, instead of a silent bus abort. A hard, named failure at open() beats an unexplained abort later.

The dispatch decision is op-driven: the backend looks for the ethos-u custom op in the model flatbuffer. It does not key off cfg->format. So a model you assumed was a plain CPU TFLite blob will still take the Ethos-U path — and still require an arena — if the ethos-u custom op is present in it.

Backend registry

Ethos-U registers per silicon, not once globally:

Backend	Silicon ref	Priority
Ethos-U	alif:ensemble:e3, e4, e5, e6, e7, e8	100 each
Ethos-U	nxp:imx9:imx93	100
CPU / TFLM	"*"	50

The CPU/TFLM fallback matches everything at priority 50, so an SoC with no NPU entry still resolves.

Ethos-U accelerator sizing

Accelerator sizing is derived from SoC metadata, per target core — not from the SoC maximum. The build picks the most capable NPU variant present (U85 > U65 > U55) and reads MAC/cycle from the SoC metadata, preferring the NPU entry whose paired_core matches the slice being built.

That last clause matters on a multi-core SoM: two cores on the same SoC can legitimately build against different accelerator configurations, because each is paired with a different NPU instance.

Valid accelerator configurations:

Variant	Configurations
Ethos-U55	ETHOS_U55_64, ETHOS_U55_128, ETHOS_U55_256
Ethos-U65	ETHOS_U65_128, ETHOS_U65_256, ETHOS_U65_512
Ethos-U85	ETHOS_U85_128, ETHOS_U85_256, ETHOS_U85_512, ETHOS_U85_1024, ETHOS_U85_2048

Linux-only NPU backends (DRP-AI3 / DEEPX DX-M1)

Both are real backends now — and both are opt-in and unproven.

Renesas DRP-AI3 (RZ/V2N)

The RZ/V2N DRP-AI3 backend runs against the real MERA DRP-AI TVM runtime (MeraDrpRuntimeWrapper).

It is gated by a CMake option, not a Zephyr Kconfig symbol — DRP-AI3 is A55/Linux-only, so a Zephyr symbol would be the wrong mechanism:

Option	Default	Effect
ALP_SDK_USE_DRPAI_V2N	OFF	Compiles in the DRP-AI3 backend.
ALP_SDK_DRPAI_REQUIRED	OFF	Turns a missing DRP-AI stack from a warning into a hard configure error.

The orchestrator emits -DALP_SDK_USE_DRPAI_V2N=ON automatically when the board declares capabilities.drp_ai, so you do not normally set it by hand.

:::warning Not run on silicon The DRP-AI3 backend has NOT run on silicon. It is header-checked against the real runtime surface only. Treat every result from it as unverified until it has been on a bench. :::

DEEPX DX-M1

DEEPX DX-M1 mirrors this exactly: CMake option ALP_SDK_USE_DEEPX_DXM1, default OFF, a real backend against dxrt::InferenceEngine.

Vendor escape-hatch headers

The per-vendor escape hatches moved out of <alp/vendors/...> and into the <alp/ext/<vendor>/...> tree that the rest of the SDK's vendor extensions already use:

Vendor	Header	Was
Renesas	<alp/ext/renesas/inference.h>	<alp/vendors/renesas/drpai.h>
DEEPX	<alp/ext/deepx/inference.h>	<alp/vendors/deepx/dxm1.h>

The old paths no longer exist — update any #include that still names them.

board.yaml

cores:
  m55_hp:
    inference:
      default_arena_kib: 256   # the only inference knob —
                               # dispatcher set is silicon-determined (SoM preset's
                               # capabilities:), app picks the runtime backend
                               # per handle via alp_inference_open().

There is no inference.backend: field — the build compiles in every dispatcher the SoM's capabilities: block declares (TFLM CPU always; Ethos-U per silicon, via CONFIG_ALP_SDK_INFERENCE_*). DRP-AI3 and DEEPX DX-M1 are the exception: they are compiled in by the CMake options ALP_SDK_USE_DRPAI_V2N / ALP_SDK_USE_DEEPX_DXM1 (both default OFF), which the orchestrator turns on from the board's declared capabilities — capabilities.drp_ai drives -DALP_SDK_USE_DRPAI_V2N=ON. Apps still select per-handle at runtime via alp_inference_open(.backend = ...).

Model formats

• TFLite Micro — the universal input format. Vela-compiled for Ethos-U backends.
• DRP-AI binary — Renesas DRP-AI3 path consumes models converted via the DRP-AI Translator toolchain.
• DXNN — DEEPX path consumes .dxnn files compiled from ONNX by the DEEPX compiler.

The dispatcher abstracts the format so application code only sees TFLite Micro. The build system invokes the right compiler for the active backend.

Portable model packages (.alpmodel)

:::caution ABI-EXPERIMENTAL <alp/model.h> and alp_inference_open_alpmodel() are [ABI-EXPERIMENTAL] — pin your SDK to a commit if you depend on them. :::

tan model build compiles a source model for every NPU back-end the SoM declares into one fat multi-backend .alpmodel package — a 24-byte header + CBOR manifest + per-backend blobs + a capability requires envelope. At runtime alp_inference_open_alpmodel() loads the package, a selection engine picks the blob that matches the active SoM, and the handle dispatches through the same backend registry as alp_inference_open(). One model, portable across NPUs with no source changes.

alp_inference_t *alp_inference_open_alpmodel(const alp_model_open_opts_t *opts);

typedef struct {
    const void             *data;        // package bytes, or NULL to use `path`
    size_t                  size;        // byte count when `data` is set
    const char             *path;        // storage path (Linux), or NULL
    alp_inference_backend_t backend;     // AUTO, or a forced backend
    size_t                  arena_bytes; // 0 = size from the manifest
    void                   *arena;       // caller arena, or NULL for backend default
} alp_model_open_opts_t;

extern const uint8_t detector_alpmodel[];
extern const size_t  detector_alpmodel_len;

// MCU: embed the package bytes; AUTO lets the loader pick the best blob.
alp_inference_t *infer = alp_inference_open_alpmodel(&(alp_model_open_opts_t){
    .data    = detector_alpmodel,
    .size    = detector_alpmodel_len,
    .backend = ALP_INFERENCE_BACKEND_AUTO,
});
if (infer == NULL) {
    int err = alp_last_error();   // ALP_ERR_NO_FIT / _NO_BACKEND / _NOT_FOUND / _VERSION / _INVAL
    return err;
}
// The returned handle works with every alp_inference_* accessor unchanged
// (get_input / invoke / get_output / close — see Quick example above).

On Linux, point at a storage path instead of embedding the bytes:

.path = "/lib/firmware/detector.alpmodel",

Selection. The loader picks the blob whose backend is available on the active SoC, whose silicon_ref is compatible, and that fits the device NPU arena envelope (ALP_SOC_NPU_ARENA_SRAM_KIB); ties break by the SoM's preferred backend. backend = AUTO falls back to a CPU blob if present; pinning an explicit NPU backend bypasses CPU fallback. Failure stamps alp_last_error():

Code	Value	Meaning
ALP_ERR_VERSION	-11	Package container version is newer than this loader supports.
ALP_ERR_NO_BACKEND	-12	No blob for any backend available on this SoM (and no CPU fallback).
ALP_ERR_NO_FIT	-13	A backend matched but no blob fits the device NPU envelope, and no CPU fallback.
ALP_ERR_NOT_FOUND	-14	An explicitly-requested backend is absent from the package.

A bad magic / truncated / corrupt package returns ALP_ERR_INVAL.

Building a package

For the full build-and-package walkthrough — per-backend compile: config, partial-coverage packages, arena sizing, and the read-side parser — see the AI Model Packaging guide.

tan model build reads the models: block of your board.yaml, derives the target back-ends from som.sku, and emits one .alpmodel per model:

tan model build --board board.yaml --out build/models

Option	Default	What it picks
--board	board.yaml	Path to the board.yaml.
--out	build/models	Output directory for the .alpmodels.
--metadata-root	the SDK's metadata/	Path to the metadata/ root.

Read-side parser (standalone firmware)

Hand-written firmware that wants to inspect a package without the loader uses <alp/model.h> directly — gated on CONFIG_ALP_SDK_MODEL_READER:

#include <alp/model.h>

alp_model_t m;
alp_status_t rc = alp_model_parse(detector_alpmodel, detector_alpmodel_len, &m);
// rc == ALP_OK; ALP_ERR_INVAL (bad magic / truncated / CBOR error);
//               ALP_ERR_VERSION (container newer than this reader).

alp_model_parse() decodes the manifest once into a bounded, stack-friendly alp_model_t view (no heap; the blobs reference the source buffer, which must outlive the view). It carries the model identity plus up to ALP_MODEL_MAX_TARGETS (8) per-backend target entries (backend, silicon_ref, blob_format, arena/SRAM requirements, and the blob slice). The container magic is ALP_MODEL_MAGIC ("ALPM"), container version 1.

Tensor arena

tensor_arena_kb reserves working memory for the interpreter. Right-size this against the model's arena_size reported by Vela or the equivalent. Too small and alp_inference_open returns NULL with ALP_ERR_OUT_OF_RANGE; too large just wastes RAM.

For a model that dispatches to Ethos-U, cfg->arena == NULL is rejected outright with ALP_ERR_INVAL — see arena == NULL is rejected for Ethos-U models.

Off-device training

The SDK is inference-only. Train your model offline in TensorFlow or PyTorch, export to TFLite (or ONNX for the DEEPX path), then deploy through <alp/inference.h>.

Reference applications

App	Stack
examples/aen/edgeai-vision-aen	camera → ISP → Ethos-U inference → OLED overlay
examples/iot-connected-camera	camera → DRP-AI inference → MQTT publish

Default-initialiser macro

Unlike the peripheral macros, ALP_INFERENCE_CONFIG_DEFAULT(id) (v0.10) takes the model blob pointer as its identity argument:

alp_inference_config_t cfg = ALP_INFERENCE_CONFIG_DEFAULT(model_blob);
cfg.model_size = model_blob_len;        // mandatory — the default 0 is invalid

It sets model_data from id plus: model_size = 0, format = ALP_INFERENCE_MODEL_TFLITE, backend = ALP_INFERENCE_BACKEND_AUTO (let the dispatcher pick), arena_bytes = 0 and arena = NULL (SDK-managed arena).

You must still set model_size — a zero size is never valid.

See the shared macro contract for the C++ compound-literal caveat that applies to every ALP_*_CONFIG_DEFAULT.

See also

• E1M-AEN inference
• E1M-X V2N-M1 inference
• <alp/camera.h>