SAM2 Deployment

Introduction

Segment-Anything-Model-2 (SAM 2) is the next generation of visual segmentation foundation models released by Meta AI. It supports real-time interactive segmentation of any object in both images and videos. Compared to the previous version, SAM 2 utilizes a unified Transformer architecture and integrates a streaming memory mechanism for video segmentation, allowing it to achieve precise and continuous object tracking with minimal interaction prompts.

This chapter demonstrates the deployment, loading, and inference process of SAM 2 on edge devices. The following deployment method is provided:

• AidLite Python API

In this case, model inference runs on the device-side NPU computing unit. The code calls the relevant interfaces to receive user input and return results.

• Device: Rhino Pi-X1
• System: Ubuntu 22.04
• Source Model: SAM2 Tiny
• Model Precision: FP16
• Model Farm Reference: SAM2 Tiny

Supported Platforms

Platform	Execution Method
Rhino Pi-X1	Ubuntu 22.04, AidLux

Prerequisites

1. Rhino Pi-X1 hardware.
2. Ubuntu 22.04 system or AidLux system.

Download SAM2-tiny-FP16 Model Resources

mms list sam2-tiny

#------------------------ SAM2-tiny models available ------------------------
Model      Precision  Chipset          Backend
-----      ---------  -------          -------
SAM2-tiny  FP16       Qualcomm QCS8550  QNN2.36

# Download SAM2-tiny-FP16
mms get -m SAM2-tiny -p fp16 -c qcs8550 -b qnn2.36 -d /home/aidlux/sam2-tiny
cd /home/aidlux/sam2-tiny
# Unzip
unzip sam2_qcs8550_qnn2.36_fp16_aidlite.zip

💡Note

Developers can also search and download models on the Model Farm website.

AidLite SDK Installation

• Ensure the QNN backend version is ≥ 2.36.
• Ensure the versions of aidlite-sdk and aidlite-qnnxxx are 2.3.x.

# Check AidLite & QNN versions
dpkg -l | grep aidlite
#------------------------ You should see output similar to this ------------------------
ii  aidlite-qnn236       2.3.0.230         arm64        aidlux aidlite qnn236 backend plugin
ii  aidlite-sdk           2.3.0.230         arm64        aidlux inference module sdk

Update QNN & AidLite versions:

# Install AidLite SDK
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
sudo aid-pkg install aidlite-qnn236

# AidLite SDK C++ check
python3 -c "import aidlite; print(aidlite.get_library_version())"

# AidLite SDK Python check
python3 -c "import aidlite; print(aidlite.get_py_library_version())"

AidLite Python API Deployment

Run the Python API Example

cd /home/aidlux/sam2-tiny/model_farm_sam2_qcs8550_qnn2.36_fp16_aidlite

# --imgs: Input image
# --point-coordinates: Prompt coordinates
# --invoke_nums: Loop iterations
python3 python/run_test.py --imgs python/truck.jpg --point-coordinates "500,375;1100,600" --invoke_nums 10

The terminal will display the model inference latency (in ms) and segmentation results:

====================================
Encoder model invoke 10 times:
 --mean_invoke_time is 216.16535186767578 
 --max_invoke_time is 221.62842750549316 
 --min_invoke_time is 214.51592445373535 
 --var_invoketime is 3.8429782512139354
====================================
[2026-02-10 15:26:06.024] [info] [license.cpp:520] the device is licensed, with license ID: aidf70cf68561ab0f1555c98a624584610b
PluginManager::load_plugin(): plugin(=aidlite_qnn236) has been loaded
w:1800, h:1200
====================================
Decoder model invoke 10 times:
 --mean_invoke_time is 6.911730766296387 
 --max_invoke_time is 7.040739059448242 
 --min_invoke_time is 6.859779357910156 
 --var_invoketime is 0.0025213859089490143
====================================
Result images saved to 'python/mask_results.jpg'

The result image mask_results.jpg is saved in the path /home/aidlux/sam2-tiny/model_farm_sam2_qcs8550_qnn2.36_fp16_aidlite/python.