BEVFormer-tiny (R50) Deployment

Introduction

BEVFormer-tiny (R50) is an autonomous-driving multi-camera 3D object detection model focused on efficient edge deployment. It was proposed by the SenseTime and Tsinghua University teams. Built on the classic ResNet-50 backbone, it uses a unique Spatio-Temporal Attention mechanism to losslessly convert 2D image features from multiple surround-view cameras into a unified surround-view Bird's-Eye View (BEV) feature. As the lightweight Tiny variant, it streamlines Transformer layers and optimizes grid resolution to dramatically reduce memory footprint and inference latency, while still fully preserving velocity estimation and temporal trajectory tracking for dynamic objects. With its excellent energy-efficiency ratio, BEVFormer-tiny has become an industry benchmark for high-precision 3D spatial perception and vehicle-road coordination on low-compute automotive chips and Roadside edge computing units (RSCU).

This chapter demonstrates how to perform BEVFormer-tiny (R50) deployment, loading, and recognition on edge devices. The following deployment method is provided:

• AidLite Python API

In this case, model inference runs on the device-side NPU computing unit, and the relevant interfaces are called through code to receive user input and return results.

• Device: Rhino Pi-X1
• System: Ubuntu 22.04
• Source model: BEVFormer-tiny (R50)
• Quantized model precision: FP16 quantization
• Model Farm Reference: BEVFormer-tiny (R50)

Supported Platforms

Platform	Running Method
Rhino Pi-X1	Ubuntu 22.04, AidLux

Preparation

1. Rhino Pi-X1 hardware
2. Ubuntu 22.04 system or AidLux system

Download the BEVFormer-tiny (R50) Model Resources

mms list BEVFormer-tiny

#------------------------ You can see the BEVFormer-tiny (R50) model ------------------------
Model                 Precision  Chipset  SOC               Backend
--------------------  ---------  -------  ----------------  -------
BEVFormer-tiny (R50)  FP16       qcs8550  Qualcomm QCS8550  QNN2.40

# Download BEVFormer-tiny (R50)
mms get -m 'BEVFormer-tiny (R50)' -p fp16 -c qcs8550 -b qnn2.40 -d /home/aidlux/models-test
cd /home/aidlux/models-test
# Unzip
unzip BevFormer-Tiny-Resnet50_qcs8550_fp16.zip

AidLite SDK Installation

Developers can also refer to the README.md in the model folder to install the SDK.

• Ensure the QNN backend version is ≥ 2.40
• Ensure aidlite-sdk and aidlite-qnnxxx versions are 2.4.x or higher

# Check AidLite & QNN versions
dpkg -l | grep aidlite
#------------------------ The output is similar to the following ------------------------
ii  aidlite-qnn240 2.4.1.271 arm64 aidlux aidlite qnn240 backend plugin
ii  aidlite-sdk 2.4.1.271 arm64 aidlux inference module sdk

Update QNN & AidLite versions:

# Install AidLite SDK
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
sudo aid-pkg install aidlite-qnn240

# aidlite sdk c++ check
python3 -c "import aidlite; print(aidlite.get_library_version())"

# aidlite sdk python check
python3 -c "import aidlite; print(aidlite.get_py_library_version())"

AidLite Python API Deployment

Run the Python API Example

cd /home/aidlux/models-test/

python3 code/python/run_test.py \
  --backbone_model ./models/QCS8550/FP16/backbone_context.bin.aidem \
  --scene_start_encoder_model ./models/QCS8550/FP16/scene_start_encoder_context.bin.aidem \
  --encoder_model ./models/QCS8550/FP16/temporal_encoder_context.bin.aidem \
  --decoder_model ./models/QCS8550/FP16/decoder_context.bin.aidem \
  --asset_manifest ./code/python/datasets/sample4/asset_manifest.json \
  --nms_contract ./code/python/configs/nms_runtime_contract.json \
  --model_type QNN240 \
  --invoke_nums 4 \
  --output_dir ./outputs/final_sample4

You can see the model inference time (in ms) and detection results in the command line:

========================================
BEVFormer W8A8 QCS8550 Performance
========================================
model platform   : QCS8550
frames           : 4
QNN invoke (ms)  : mean=401.968  min=400.651  max=403.298
  backbone       : 18.089
  encoder        : 344.693 (scene_start)  345.820 (temporal)
  decoder        : 38.341

Output files
  summary JSON   : /home/aidlux/models-test/outputs/final_sample4/bevformer_demo_summary.json
  camera-grid PNG: 4 frame(s)
  camera-grid GIF: /home/aidlux/models-test/outputs/final_sample4/sample4_camera_grid.gif
========================================

After the example program runs, the final dynamic detection image sample4_camera_grid.gif will be generated under /home/aidlux/models-test/outputs/final_sample4/.