v2n-drpai-inference
Run a compiled model through the RZ/V2N's on-die DRP-AI3 NPU via <alp/inference.h>, on one or more still frames given on the command line, and print per-image results plus timing. A Linux/Yocto user-space app on the Cortex-A55 cluster — the A55-side sibling of v2n-m1-deepx-inference.
Source: examples/v2n/v2n-drpai-inference/. New in alp-sdk v0.17.0.
[UNTESTED on silicon]The app builds as a standalone C translation unit against the real <alp/inference.h> header, and its top-N selection logic is unit-tested. Linking against a real libalp_sdk and running end to end have not been verified, and nothing has run against real DRP-AI silicon through this app. The SDK's DRP-AI backend itself has run on silicon (YOLOX-S / VOC on an E1M-V2M103, about 40 ms per frame).
What it does
- Opens a DRP-AI-compiled model bundle (a
drpai_dirtar) through the portable surface, withbackend = ALP_INFERENCE_BACKEND_DRPAIandformat = ALP_INFERENCE_MODEL_DRPAI. - Feeds one or more raw, pre-processed frames, one inference call each, and prints each image's output size, wall-clock invoke time, and the top-5 raw output values.
- Handles DRP-AI being unavailable the way the documented
NOSUPPORTcontract requires:alp_inference_open()returnsNULL, and the program reports why (backend not compiled in / driver absent / driver busy) and exits cleanly.
Input: raw frames, not JPEG/PNG
Decoding an image file needs a codec outside the SDK's portable surface, so the app reads raw pre-processed frames: flat 640×640×3 float32 NCHW buffers, exactly 4,915,200 bytes each, planar (all of channel 0, then channel 1, then channel 2) — not interleaved HWC. The runtime cannot catch a layout mistake: alp_inference_get_input() reports only a byte count, so an HWC file of the same size passes the size check and reaches the NPU with the wrong channel order. Getting NCHW right is the caller's job.
The target model is YOLOX-S trained on VOC. Matching the vendor sample, each frame is letterboxed to a square (image at the top, padding RGB 114), resized bilinearly to 640×640, in RGB channel order, raw 0–255 with no normalisation. The README carries a Pillow + NumPy generator that follows those steps.
Output: raw scores, not detections
The program prints the largest raw values of the output tensor, clearly labelled as raw and undecoded. A YOLOX decoder (grids, sigmoid, box regression, non-maximum suppression) is not implemented — a team productising this demo adds it once a board confirms the raw tensor's layout.
No bundle ships, and the SDK cannot compile one for this model yet
There is no supported path in the SDK today to produce the drpai_dir bundle for this model. The SDK's DRP-AI adapter only handles the 224×224 classifier geometry and rejects this detector's input shape. Compile the bundle outside the SDK with the Renesas DRP-AI TVM (RUHMI) toolchain, then tar the compiler's object directory with relative paths:
tar -cf model.tar -C <obj_dir> .
board.yaml
som:
sku: E1M-V2N101
preset: e1m-x-evk
cores:
a55_cluster:
app: ./src # Linux userspace app -- <alp/inference.h> only
recipe: alp-drpai-inference # meta-alp-sdk/recipes-examples/alp-drpai-inference
m33_sm:
os: "off" # DRP-AI3 inference is wholly A55/Linux-owned
diagnostics:
log_level: info
There is no chips: entry — DRP-AI3 is the SoC's on-die NPU, not a board-populated IC — and no inference: block, because the DRP-AI backend takes no caller-supplied arena. DRP-AI3 is on-die in every RZ/V2N-family SKU (V2N and V2M), but only E1M-V2N101 is declared and built here.
Build and run
Yocto (recommended). The alp-drpai-inference recipe packages this binary into alp-image-edge whenever the DRP-AI3 backend is built: add meta-rz-drpai to bblayers.conf, and point the build at an account-gated Renesas RUHMI checkout in local.conf:
RUHMI_DRPAI_TVM_DIR = "/path/to/built/rzv_drp-ai_tvm"
Without RUHMI the SDK's DRP-AI backend is not compiled and the open fails with ALP_ERR_NOSUPPORT, even though the driver and runtime are in the image. After boot:
v2n-drpai-inference <model.tar> <frame0.bin> [frame1.bin ...]
Standalone cross-compile against the Yocto SDK sysroot:
cmake -DCMAKE_TOOLCHAIN_FILE=<sysroot>/toolchain.cmake \
-S examples/v2n/v2n-drpai-inference -B build/drpai-demo
cmake --build build/drpai-demo
See also
<alp/inference.h>referencev2n-two-models— DRP-AI3 and the DEEPX DX-M1 at the same timev2n-m1-deepx-inference— the DEEPX sibling, sameNOSUPPORTcontract shape- Model packaging
- Examples overview