<alp/inference.h> — NPU Dispatcher

A single API for running TFLite Micro models on the right silicon-specific NPU back-end.

Supported back-ends

Back-end	TFLite Micro Kconfig	Available on
Arm Ethos-U55	CONFIG_ALP_TFLM_ETHOS_U55	Every E1M-AEN SKU (E3 / E4 / E5 / E6 / E7 / E8) — two instances per SoC
Arm Ethos-U85	CONFIG_ALP_TFLM_ETHOS_U85	E1M-AEN401 / AEN601 / AEN801 (Alif E4 / E6 / E8 only) — one instance per SoC, Transformer-capable, generative-AI forward path
Arm Ethos-U65	CONFIG_ALP_TFLM_ETHOS_U65	E1M-N93 family (NXP i.MX 93)
Renesas DRP-AI3	CONFIG_ALP_TFLM_DRP_AI	E1M-X V2N family
DEEPX DX-M1	ALP_SDK_INFERENCE_DEEPX_DX	E1M-X V2N-M1 family (non-TFLM runtime — DXNN)
CPU	(fallback, always available)	Any target — reference kernels

Which dispatchers compile in is silicon-determined — the SDK's loader reads the active SoM preset's capabilities: block and emits CONFIG_ALP_SDK_INFERENCE_* for every dispatcher the silicon can drive (TFLM CPU always; Ethos-U / DRP-AI3 / DEEPX DX-M1 per caps). The customer's board.yaml does not carry an inference.backend: knob.

Apps pick a backend per handle at runtime via alp_inference_open(.backend = ALP_INFERENCE_BACKEND_AUTO | _CPU | _ETHOS_U | _DRPAI | _DEEPX_DXM1). AUTO picks the highest-priority match for the active SoM (order: U85 → U65 → U55 → DRP-AI → DEEPX DX-M1 → CPU). On V2M101 (V2N-M1) the dispatcher set carries both DRP-AI3 and DEEPX DX-M1; apps can open two handles and dispatch concurrently to either NPU.

The U55 / U85 split is driven by the loader's requires_cap matcher reading each SoM preset's capabilities: block — pick the SKU, the right CONFIG_* follows automatically.

Header

#include <alp/inference.h>

Quick example

extern const uint8_t my_model_tflite[];
extern const size_t  my_model_tflite_len;

alp_inference_t *infer = alp_inference_open(&(alp_inference_config_t){
    .backend         = ALP_INFERENCE_BACKEND_AUTO,
    .model_bytes     = my_model_tflite,
    .model_len       = my_model_tflite_len,
    .tensor_arena_kb = 256,
});
if (infer == NULL) {
    int err = alp_last_error();
    return err;
}

// Get input tensor metadata
alp_tensor_t input;
alp_inference_get_input(infer, 0, &input);
memcpy(input.data, my_image, input.byte_size);

// Run
alp_inference_invoke(infer);

// Read output
alp_tensor_t output;
alp_inference_get_output(infer, 0, &output);
// process output.data ...

alp_inference_close(infer);

board.yaml

cores:
  m55_hp:
    inference:
      default_arena_kib: 256   # the only inference knob —
                               # dispatcher set is silicon-determined (SoM preset's
                               # capabilities:), app picks the runtime backend
                               # per handle via alp_inference_open().

There is no inference.backend: field — the build compiles in every dispatcher the SoM's capabilities: block declares (TFLM CPU always; Ethos-U / DRP-AI3 / DEEPX DX-M1 per silicon). Apps select per-handle at runtime via alp_inference_open(.backend = ...).

Model formats

• TFLite Micro — the universal input format. Vela-compiled for Ethos-U backends.
• DRP-AI binary — Renesas DRP-AI3 path consumes models converted via the DRP-AI Translator toolchain.
• DXNN — DEEPX path consumes .dxnn files compiled from ONNX by the DEEPX compiler.

The dispatcher abstracts the format so application code only sees TFLite Micro. The build system invokes the right compiler for the active backend.

Portable model packages (.alpmodel)

:::caution ABI-EXPERIMENTAL <alp/model.h> and alp_inference_open_alpmodel() are [ABI-EXPERIMENTAL] — pin your SDK to a commit if you depend on them. :::

alp model build compiles a source model for every NPU back-end the SoM declares into one fat multi-backend .alpmodel package — a 24-byte header + CBOR manifest + per-backend blobs + a capability requires envelope. At runtime alp_inference_open_alpmodel() loads the package, a selection engine picks the blob that matches the active SoM, and the handle dispatches through the same backend registry as alp_inference_open(). One model, portable across NPUs with no source changes.

alp_inference_t *alp_inference_open_alpmodel(const alp_model_open_opts_t *opts);

typedef struct {
    const void             *data;        // package bytes, or NULL to use `path`
    size_t                  size;        // byte count when `data` is set
    const char             *path;        // storage path (Linux), or NULL
    alp_inference_backend_t backend;     // AUTO, or a forced backend
    size_t                  arena_bytes; // 0 = size from the manifest
    void                   *arena;       // caller arena, or NULL for backend default
} alp_model_open_opts_t;

extern const uint8_t detector_alpmodel[];
extern const size_t  detector_alpmodel_len;

// MCU: embed the package bytes; AUTO lets the loader pick the best blob.
alp_inference_t *infer = alp_inference_open_alpmodel(&(alp_model_open_opts_t){
    .data    = detector_alpmodel,
    .size    = detector_alpmodel_len,
    .backend = ALP_INFERENCE_BACKEND_AUTO,
});
if (infer == NULL) {
    int err = alp_last_error();   // ALP_ERR_NO_FIT / _NO_BACKEND / _NOT_FOUND / _VERSION / _INVAL
    return err;
}
// The returned handle works with every alp_inference_* accessor unchanged
// (get_input / invoke / get_output / close — see Quick example above).

On Linux, point at a storage path instead of embedding the bytes:

.path = "/lib/firmware/detector.alpmodel",

Selection. The loader picks the blob whose backend is available on the active SoC, whose silicon_ref is compatible, and that fits the device NPU arena envelope (ALP_SOC_NPU_ARENA_SRAM_KIB); ties break by the SoM's preferred backend. backend = AUTO falls back to a CPU blob if present; pinning an explicit NPU backend bypasses CPU fallback. Failure stamps alp_last_error():

Code	Value	Meaning
ALP_ERR_VERSION	-11	Package container version is newer than this loader supports.
ALP_ERR_NO_BACKEND	-12	No blob for any backend available on this SoM (and no CPU fallback).
ALP_ERR_NO_FIT	-13	A backend matched but no blob fits the device NPU envelope, and no CPU fallback.
ALP_ERR_NOT_FOUND	-14	An explicitly-requested backend is absent from the package.

A bad magic / truncated / corrupt package returns ALP_ERR_INVAL.

Building a package

For the full build-and-package walkthrough — per-backend compile: config, partial-coverage packages, arena sizing, and the read-side parser — see the AI Model Packaging guide.

alp model build reads the models: block of your board.yaml, derives the target back-ends from som.sku, and emits one .alpmodel per model:

alp model build --board board.yaml --out build/models

Option	Default	What it picks
--board	board.yaml	Path to the board.yaml.
--out	build/models	Output directory for the .alpmodels.
--metadata-root	the SDK's metadata/	Path to the metadata/ root.

Read-side parser (standalone firmware)

Hand-written firmware that wants to inspect a package without the loader uses <alp/model.h> directly — gated on CONFIG_ALP_SDK_MODEL_READER:

#include <alp/model.h>

alp_model_t m;
alp_status_t rc = alp_model_parse(detector_alpmodel, detector_alpmodel_len, &m);
// rc == ALP_OK; ALP_ERR_INVAL (bad magic / truncated / CBOR error);
//               ALP_ERR_VERSION (container newer than this reader).

alp_model_parse() decodes the manifest once into a bounded, stack-friendly alp_model_t view (no heap; the blobs reference the source buffer, which must outlive the view). It carries the model identity plus up to ALP_MODEL_MAX_TARGETS (8) per-backend target entries (backend, silicon_ref, blob_format, arena/SRAM requirements, and the blob slice). The container magic is ALP_MODEL_MAGIC ("ALPM"), container version 1.

Tensor arena

tensor_arena_kb reserves working memory for the interpreter. Right-size this against the model's arena_size reported by Vela or the equivalent. Too small and alp_inference_open returns NULL with ALP_ERR_OUT_OF_RANGE; too large just wastes RAM.

Off-device training

The SDK is inference-only. Train your model offline in TensorFlow or PyTorch, export to TFLite (or ONNX for the DEEPX path), then deploy through <alp/inference.h>.

Reference applications

App	Stack
examples/aen/edgeai-vision-aen	camera → ISP → Ethos-U inference → OLED overlay
examples/iot-connected-camera	camera → DRP-AI inference → MQTT publish

See also

• E1M-AEN inference
• E1M-X V2N-M1 inference
• <alp/camera.h>