LLM Service Deployment

LLM Toolchain Installation

The A8550MA1 supports on-device deployment of 0.5B–14B parameter LLMs, with a complete LLM development toolchain.
Log into the system via Browser Remote Login. For detailed documentation, refer to AidGenSE.

# 1. Download the application from the App Center:
# AidGenSE 2.0.9
# Install the aidgense service
sudo aid-pkg install aidgense -v 2.0.9

Password: aidlux

Online Model Pull & Run

# View available remote LLMs
aidllm remote-list api
# Pull the A8550-optimized Qwen2.5-VL-3B multimodal model
aidllm pull api aplux/qwen2.5-vl-3b-instruct-392x392-qnn2.36-w4a16-qcs8550
# View installed models
aidllm list api
# Start the LLM API service and view logs
aidllm start api -m qwen2.5-vl-3b-instruct-392x392-qnn2.36-w4a16-qcs8550 --show-log
# Verify service status
aidllm status api
# Normal output: Api server status: Running

Local Model Manual Deployment

# 1. Upload the local model folder (e.g., qwen2.5-1.5b-instruct-q4_k_m) to /home/aidlux/
# 2. Copy to the model directory
cp -r qwen2.5-1.5b-instruct-q4_k_m /opt/aidlux/app/aid-openai-api/res/models/

# 3. Modify the config file /opt/aidlux/app/aid-openai-api/api_cfg.json
{
  "model_cfg_list": [
    {
      "model_id": "qwen2.5-1.5b-instruct-q4_k_m",
      "model_create": "1752735571243",
      "model_owner": "aplux",
      "cfg_path": "./models/qwen2.5-1.5b-instruct-q4_k_m/qwen2.5-1.5b-instruct-q4_k_m.json",
      "prompt_template_type": "qwen2"
    }
  ],
  "default_model_id": "qwen2.5-1.5b-instruct-q4_k_m"
}

# 4. Start the service
aidllm start api -m qwen2.5-1.5b-instruct-q4_k_m

OpenAI-Compatible API Calls

AidGen provides an API interface that is fully compatible with OpenAI, with the default port set to 8888.

Text Chat Example

curl http://192.168.100.3:8888/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen2.5-VL-3B-392x392-8625",
    "messages": [
      {"role": "user", "content": "Briefly introduce the A8550MA1 compute card."}
    ]
  }'

Multimodal Image Analysis Example

curl http://192.168.100.3:8888/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen2.5-VL-3B-392x392-8625",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe the content of this image."},
          {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAAAAAAAD/..."}}
        ]
      }
    ]
  }'