Stable Diffusion Deployment

Introduction

Stable Diffusion is an open-source text-to-image generation model developed by Stability AI. Based on the Latent Diffusion Model (LDM) architecture, it generates high-quality images from natural language descriptions. By operating in a compressed latent space rather than pixel space, it significantly reduces computational requirements while maintaining high fidelity.

The model is trained on billions of image-text pairs and supports various resolutions (e.g., 512x512 to 1024x1024). It can produce realistic, artistic, or abstract visual content and is widely used in creative design, gaming assets, and education. Its open-source nature allows for local deployment and fine-tuning through parameters like prompts and sampling steps.

This chapter demonstrates how to deploy Stable Diffusion V2.1 on edge devices using the AidLite Python API. The inference runs on the device-side NPU for optimal performance.

• Device: Rhino Pi-X1
• System: Ubuntu 22.04
• Source Model: Stable Diffusion V2.1
• Precision: W8A16 (INT8 Weights, FP16 Activations)
• Model Farm Reference: Stable Diffusion

Supported Platforms

Platform	Execution Method
Rhino Pi-X1	Ubuntu 22.04, AidLux

Prerequisites

1. Rhino Pi-X1 hardware.
2. Ubuntu 22.04 system or AidLux system.

Download Stable-Diffusion-v2.1-W8A16 Model Resources

mms list Stable-Diffusion

#------------------------ Stable-Diffusion models available ------------------------
Model                   Precision  Chipset              Backend
-----                   ---------  -------              -------
Stable-Diffusion-v2.1   W8A16      Qualcomm QCS8550     QNN2.31

# Download Stable-Diffusion-v2.1-W8A16
mms get -m Stable-Diffusion-v2.1 -p w8a16 -c qcs8550 -b qnn2.31 -d /home/aidlux/stable_diffusion
cd /home/aidlux/stable_diffusion
# Unzip
unzip stable_diffusion_v2_1_qcs8550_qnn2.31_w8a16_aidlite.zip

💡Note

Developers can also search and download models directly from the Model Farm website.

AidLite SDK Installation

• Ensure the QNN backend version is ≥ 2.31.
• Ensure aidlite-sdk and aidlite-qnnxxx versions are 2.3.x.

# Check AidLite & QNN versions
dpkg -l | grep aidlite
#------------------------ Example output ------------------------
ii  aidlite-qnn236       2.3.0.230         arm64        aidlux aidlite qnn236 backend plugin
ii  aidlite-sdk           2.3.0.230         arm64        aidlux inference module sdk

Update SDK:

# Install/Update AidLite SDK
sudo aid-pkg update
sudo aid-pkg install aidlite-sdk
sudo aid-pkg install aidlite-qnn236

AidLite Python API Deployment

Run the Python API Example

# Install Python dependencies
cd /home/aidlux/stable_diffusion/model_farm_stable_diffusion_v2_1_qcs8550_qnn2.31_w8a16_aidlite
pip install -r python/requirements.txt

# Run inference
# --prompt: Your text description
# --save_path: Output image path
python3 python/run_test.py --prompt 'Chinese garden at wildlife river and mountain range, highly detailed, digital illustration, artstation, concept art, matte, sharp focus, illustration, dramatic, sunset, hearthstone, art by Artgerm and Greg Rutkowski and Alphonse Mucha' --save_path python/generate_img.jpg

Inference Output (Example):

Vae model loading successful!
>> Text model invoketime: 9.657859802246094 ms
>> Text model invoketime: 9.041786193847656 ms
>>>>>Step 0 Running...
>>>>>Step 1 Running...
>>>>>Step 2 Running...
>>>>>Step 3 Running...
>>>>>Step 4 Running...
>>>>>Step 5 Running...
>>>>>Step 6 Running...
>>>>>Step 7 Running...
>>>>>Step 8 Running...
>>>>>Step 9 Running...
>>>>>Step 10 Running...
>>>>>Step 11 Running...
>>>>>Step 12 Running...
>>>>>Step 13 Running...
>>>>>Step 14 Running...
>>>>>Step 15 Running...
>>>>>Step 16 Running...
>>>>>Step 17 Running...
>>>>>Step 18 Running...
>>>>>Step 19 Running...
>> Vae model invoketime: 218.69373321533203 ms
>>>Generated image has been saved to python/generate_img.jpg
Invoke time :4.754089832305908 s

The generated image generate_img.jpg will be saved in the python/ directory.