C++ API Documentation

Plugin Overview

qtivcomposer Plugin

The qtivcomposer plugin uses GPU hardware to merge multiple input video streams into a single output stream. Pad properties determine how the input buffers from each stream are composited into the output buffer.

Attribute	Purpose
Position	Sets the X and Y placement of each input frame in the output image.
Crops	Optionally crops the source image.
Dimensions	Scales frames up or down and adjusts the resolution.
Rotate	Rotates a frame by right-angle increments.
Flip-horizontal / Flip-vertical	Controls how each input frame is mirrored in the composed output.
Alpha	Adds an alpha-blending value to a frame.
Z order	Controls layer stacking. By default, stacking follows sink pad creation order, such as sink_0, sink_1, and so on.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "video_composer0"
parent	Parent object. - Access: read, write - Type: GstObject
engine	Backend engine used for conversion. - Access: read, write - Type: GstVideoConverterBackend - Default: 1, gles - 1: gles uses the OpenGLES-based converter - 2: fcv uses the FastCV-based converter - Note: the Qualcomm Computer Vision SDK (fcv) backend is currently unsupported
latency	Extra delay for live pipelines, allowing upstream elements more time to generate buffers. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
start-time-selection	Determines how output start time is selected. - Access: read, write - Type: GstAggregatorStartTimeSelection - Default: 0, zero - 0: zero starts at runtime 0 - 1: first starts from the first observed input runtime - 2: set uses the start-time property
start-time	Start time used when start-time-selection is set to set. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 18446744073709551615
background	Background color. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: unsigned integer - Range: 0 to 4294967295 - Default: 4286611584

waylandsink Plugin

The waylandsink plugin is implemented on top of GStreamer's GstVideoSink class. It works with the Wayland Weston compositor, creates a window automatically, and renders incoming video frames into that window.

Qualcomm extends waylandsink with window-positioning support, a GBM-based zero-copy buffer backend, and an XDG shell backend.

tip

In the current release, the x, y, width, and height properties are not supported.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "waylandsink0"
parent	Parent object. - Access: read, write - Type: GstObject
sync	Synchronizes rendering to the pipeline clock. - Access: read, write - Type: boolean - Default: true
max-lateness	Maximum lateness allowed before a buffer is dropped, in nanoseconds. -1 means unlimited. - Access: read, write - Type: 64-bit integer - Range: -1 to 9223372036854775807 - Default: 20000000
qos	Emits QoS events upstream. - Access: read, write - Type: boolean - Default: true
async	Transitions to PAUSED asynchronously. - Access: read, write - Type: boolean - Default: true
ts-offset	Timestamp offset in nanoseconds. - Access: read, write - Type: 64-bit integer - Range: -9223372036854775808 to 9223372036854775807 - Default: 0
enable-last-sample	Enables the last-sample property. - Access: read, write - Type: boolean - Default: true
last-sample	The last sample received by the sink. - Access: read - Type: GstSample boxed pointer
blocksize	Byte size of each buffer pull. 0 means default. - Access: read, write - Type: unsigned integer - Range: 0 to 4294967295 - Default: 4096
render-delay	Extra sink-side render delay in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
throttle-time	Interval between rendered buffers. 0 disables throttling. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
max-bitrate	Maximum rendered bitrate in bits per second. 0 disables the limit. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
show-preroll-frame	Controls whether preroll frames are rendered. - Access: read, write - Type: boolean - Default: true
display	Wayland display name used for the connection if not supplied through GstContext. - Access: read, write - Type: string - Default: null
xdg-shell	Uses the XDG shell protocol. - Access: read, write - Type: boolean - Default: true
fullscreen	Controls fullscreen display. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: boolean - Default: false

qtivsplit Plugin

The qtivsplit plugin uses GPU hardware to split one input video stream into multiple output streams. The number of streams is user-defined and matches the number of source pads.

Each source pad's mode property determines how the input stream is split.

Mode	Description
none	Copies incoming frames from the sink pad to every source pad. Additional color conversion and scaling may be performed based on the negotiated GstCaps.
single-roi-meta	Checks whether GstVideoRegionOfInterestMeta is present in the input buffer. The number of source pads must match the maximum expected ROI metadata entries per GstBuffer. If there are too few pads, unmatched ROI metadata is ignored.
batch-roi-meta	Processes ROI metadata attached to the buffer. Unlike single-roi-meta, each metadata entry is forwarded as an individual buffer.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "videosplit0"
parent	Parent object. - Access: read, write - Type: GstObject
engine	Backend engine used for conversion. - Access: read, write - Type: GstVideoConverterBackend - Default: 1, gles - 1: gles uses the OpenGLES-based converter - 2: fcv uses the Qualcomm Computer Vision SDK-based converter - Note: the Qualcomm Computer Vision SDK (fcv) backend is currently unsupported

qtivtransform Plugin

The qtivtransform plugin uses GPU hardware to scale, flip, rotate, crop, and color-convert input YUV or RGB frames according to element properties.

Attribute	Purpose
Destination	Sets the X and Y position of the input frame in the output image and defines the output dimensions.
Crop	Optionally crops the source frame.
Rotate	Rotates the frame by right-angle increments.
Flip-horizontal / Flip-vertical	Mirrors the frame horizontally or vertically.

If no transformation properties are specified on the source pad caps, the plugin negotiates capabilities identical to the input stream and runs in pass-through mode until the parameters change.

The plugin uses Qualcomm's IB2C library on the Adreno GPU to perform these transformations. The library is wrapped by the custom GstC2dVideoConverter abstraction, which exposes APIs for creating, configuring, and processing input and output buffers.

The custom GstImageBufferPool can allocate either GBM or ION output buffers, depending on downstream negotiation:

• GBM allocation uses Qualcomm's libgbm implementation.
• ION allocation is performed by issuing IOCTL commands to the kernel.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "videotransform0"
parent	Parent object. - Access: read, write - Type: GstObject
engine	Backend engine used for conversion. - Access: read, write - Type: GstVideoConverterBackend - Default: 1, gles - 1: gles uses the OpenGLES-based converter - 2: fcv uses the Qualcomm Computer Vision SDK-based converter - Note: the Qualcomm Computer Vision SDK (fcv) backend is currently unsupported
flip-horizontal	Mirrors the frame horizontally. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: boolean - Default: false
flip-vertical	Mirrors the frame vertically. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: boolean - Default: false
rotate	Rotates the frame. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: GstVideoTransformRotate - Default: 0, none - 0: none - 1: 90CW - 2: 90CCW - 3: 180
crop	Input crop rectangle in the format <X, Y, WIDTH, HEIGHT>. - Note: this property does not support time synchronization. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: array of gint-typed GstValue entries
destination	Output destination rectangle in the format <X, Y, WIDTH, HEIGHT>. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: array of gint-typed GstValue entries
background	Background color. - Access: read, write; modifiable in NULL, READY, PAUSED, and PLAYING - Type: unsigned integer - Range: 0 to 4294967295 - Default: 4286611584

qtisocketsink Plugin

The qtisocketsink plugin sends GstBuffer objects backed by file descriptors (FDs) to other processes through UNIX domain sockets. qtisocketsrc is the corresponding receiving element.

qtisocketsink requires a UNIX domain socket file with the .sock suffix. That file is passed through the socket property so the FD can be transferred.

Buffers received from other processes are tracked through reference counting. The counter is incremented or decremented as buffers are sent or returned through the socket.

Some commonly used qtisocketsink settings are listed below.

Setting	Description
async	When set to false, the socket transitions to PAUSED asynchronously.
max-lateness	Maximum lateness allowed before a buffer is dropped, in nanoseconds.
max-bitrate	Maximum rendered bitrate in bits per second.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "fdsocketsink0"
parent	Parent object. - Access: read, write - Type: GstObject
sync	Synchronizes rendering to the pipeline clock. - Access: read, write - Type: boolean - Default: true
max-lateness	Maximum lateness allowed before a buffer is dropped. -1 means unlimited. - Access: read, write - Type: 64-bit integer - Range: -1 to 9223372036854775807 - Default: -1
qos	Emits QoS events upstream. - Access: read, write - Type: boolean - Default: false
async	Transitions to PAUSED asynchronously. - Access: read, write - Type: boolean - Default: true
ts-offset	Timestamp offset in nanoseconds. - Access: read, write
enable-last-sample	Enables the last-sample property. - Access: read, write - Type: boolean - Default: true
last-sample	Last sample received by the sink. - Access: read - Type: GstSample boxed pointer
blocksize	Byte size of each buffer pull. 0 means default. - Access: read, write - Type: unsigned integer - Range: 0 to 4294967295 - Default: 4096
render-delay	Extra sink-side render delay in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
throttle-time	Interval between rendered buffers. 0 disables throttling. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
max-bitrate	Maximum rendered bitrate in bits per second. 0 disables the limit. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
socket	Path to the UNIX domain socket. - Access: read, write; only modifiable in NULL or READY - Type: string - Default: null

qtisocketsrc Plugin

The qtisocketsrc plugin receives FD-backed GstBuffer objects from another process through a UNIX domain socket, typically from a qtisocketsink element.

The plugin requires a socket file with the .sock suffix. After the connection is established, it polls or waits for incoming GstBuffer objects and constructs buffer blocks from the received frame data.

Common buffer-related settings include:

• blocksize: byte size of each read operation.
• num-buffers: number of buffers pushed downstream before EOF is sent.

Property	Description
name	Object name. - Access: read, write - Type: string - Default: "fdsocketsrc0"
parent	Parent object. - Access: read, write - Type: GstObject
blocksize	Byte size of each read operation. -1 means default. - Access: read, write - Type: unsigned integer - Range: 0 to 4294967295 - Default: 4096
num-buffers	Number of buffers output before EOS is sent. -1 means unlimited. - Access: read, write - Type: integer - Range: -1 to 2147483647 - Default: -1
typefind	Runs type finding before negotiation. Deprecated and currently non-functional. - Access: read, write, deprecated - Type: boolean - Default: false
do-timestamp	Applies the current stream time to outgoing buffers. - Access: read, write - Type: boolean - Default: false
socket	Path to the UNIX domain socket. - Access: read, write; only modifiable in NULL or READY - Type: string - Default: null
timeout	Socket connection timeout. - Access: read, write; only modifiable in NULL or READY - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0

v4l2h264dec Plugin

The v4l2h264dec plugin decodes video streams through the V4L2 API. On supported platforms, it uses the hardware H.264 decoder to provide hardware-accelerated H.264 decoding.

tip

In the current release, only capture-io-mode and output-io-mode are writable. Both properties can only be used together with (5) dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT.

Property	Description
automatic-request-sync-point-flags	Flags used when requesting sync points automatically. - Access: read, write - Type: GstVideoDecoderRequestSyncPointFlags - Default: 0x00000003, corrupt-output+discard-input
automatic-request-sync-points	Requests sync points automatically when needed. - Access: read, write - Type: boolean - Default: false
capture-io-mode	Capture-side I/O mode matched to the source pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
device	Device path. - Access: read - Type: string - Default: "/dev/video0"
device-fd	Device file descriptor. - Access: read - Type: integer - Range: -1 to 2147483647 - Default: -1
device-name	Device name. - Access: read - Type: string - Default: null
discard-corrupted-frames	Drops frames marked as corrupted instead of outputting them. - Access: read, write - Type: boolean - Default: false
extra-controls	Extra V4L2 control IDs for the device. - Access: read, write - Type: boxed GstStructure pointer
max-errors	Maximum consecutive decoder errors allowed before reporting a stream error. - Access: read, write - Type: integer - Range: -1 to 2147483647 - Default: 10
min-force-key-unit-interval	Minimum interval between forced key-unit requests, in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
name	Object name. - Access: read, write - Type: string - Default: "v4l2h264dec0"
output-io-mode	Output-side I/O mode matched to the sink pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
parent	Parent object. - Access: read, write - Type: GstObject
qos	Handles QoS events from downstream. - Access: read, write - Type: boolean - Default: true

v4l2h265dec Plugin

The v4l2h265dec plugin decodes video streams through the V4L2 API. On supported platforms, it uses the hardware H.265 decoder to provide hardware-accelerated H.265 decoding.

• V4L2 is the standard Linux interface for video devices and is responsible for video capture, encoding, and decoding.
• H.265 provides better compression efficiency than H.264, and hardware decoding reduces system resource usage.
• GstVideoDecoder is the GStreamer base class that keeps the plugin aligned with standard decoding flows.

tip

In the current release, only capture-io-mode and output-io-mode are writable. Both properties can only be used together with (5) dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT.

Property	Description
automatic-request-sync-point-flags	Flags used when requesting sync points automatically. - Access: read, write - Type: GstVideoDecoderRequestSyncPointFlags - Default: 0x00000003, corrupt-output+discard-input
automatic-request-sync-points	Requests sync points automatically when needed. - Access: read, write - Type: boolean - Default: false
capture-io-mode	Capture-side I/O mode matched to the source pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
device	Device path. - Access: read - Type: string - Default: "/dev/video0"
device-fd	Device file descriptor. - Access: read - Type: integer - Range: -1 to 2147483647 - Default: -1
device-name	Device name. - Access: read - Type: string - Default: null
discard-corrupted-frames	Drops frames marked as corrupted instead of outputting them. - Access: read, write - Type: boolean - Default: false
extra-controls	Extra V4L2 control IDs for the device. - Access: read, write - Type: boxed GstStructure pointer
max-errors	Maximum consecutive decoder errors allowed before reporting a stream error. - Access: read, write - Type: integer - Range: -1 to 2147483647 - Default: 10
min-force-key-unit-interval	Minimum interval between forced key-unit requests, in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
name	Object name. - Access: read, write - Type: string - Default: "v4l2h265dec0"
output-io-mode	Output-side I/O mode matched to the sink pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
parent	Parent object. - Access: read, write - Type: GstObject
qos	Handles QoS events from downstream. - Access: read, write - Type: boolean - Default: true

v4l2h264enc Plugin

The v4l2h264enc plugin encodes video through the V4L2 API. On supported platforms, it uses the hardware H.264 encoder to provide hardware-accelerated H.264 encoding.

• V4L2 is the standard Linux interface for video devices and is responsible for video capture and encoding.
• Hardware H.264 encoding improves throughput while reducing CPU usage compared with software encoding.
• GstVideoEncoder is the base class that keeps the element compatible with standard GStreamer encoding flows.

Property	Description
capture-io-mode	Capture-side I/O mode matched to the source pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
device	Device path. - Access: read - Type: string - Default: "/dev/video1"
device-fd	Device file descriptor. - Access: read - Type: integer - Range: -1 to 2147483647 - Default: -1
device-name	Device name. - Access: read - Type: string - Default: null
extra-controls	Extra V4L2 control IDs for the device. - Access: read, write - Type: boxed GstStructure pointer
min-force-key-unit-interval	Minimum interval between forced key-unit requests, in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
name	Object name. - Access: read, write - Type: string - Default: "v4l2h264enc0"
output-io-mode	Output-side I/O mode matched to the sink pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
parent	Parent object. - Access: read, write - Type: GstObject
qos	Handles QoS events from downstream. - Access: read, write - Type: boolean - Default: true

Control	Type	Description
horizontal_flip	boolean	value=0 by default; flags=execute-on-write
vertical_flip	boolean	value=0 by default; flags=execute-on-write
rotate	integer	min=0, max=270, step=90, value=0; flags=execute-on-write and modify-layout

v4l2h265enc Plugin

The v4l2h265enc plugin encodes video through the V4L2 API. On supported platforms, it uses the hardware H.265 encoder to provide hardware-accelerated H.265 encoding.

• V4L2 is the standard Linux interface for video devices and is responsible for video capture and encoding.
• Hardware H.265 encoding provides better compression efficiency than H.264 while reducing CPU usage through dedicated hardware blocks.
• GstVideoEncoder is the base class that keeps the element compatible with standard GStreamer encoding flows.

Property	Description
capture-io-mode	Capture-side I/O mode matched to the source pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
device	Device path. - Access: read - Type: string - Default: "/dev/video1"
device-fd	Device file descriptor. - Access: read - Type: integer - Range: -1 to 2147483647 - Default: -1
device-name	Device name. - Access: read - Type: string - Default: null
extra-controls	Extra V4L2 control IDs for the device. - Access: read, write - Type: boxed GstStructure pointer
min-force-key-unit-interval	Minimum interval between forced key-unit requests, in nanoseconds. - Access: read, write - Type: unsigned 64-bit integer - Range: 0 to 18446744073709551615 - Default: 0
name	Object name. - Access: read, write - Type: string - Default: "v4l2h265enc0"
output-io-mode	Output-side I/O mode matched to the sink pad. - Access: read, write - Type: GstV4l2IOMode - Default: 0, auto
parent	Parent object. - Access: read, write - Type: GstObject
qos	Handles QoS events from downstream. - Access: read, write - Type: boolean - Default: true

C++ API Overview

Structure	Member	Type	Description
Image	data	const char *	Pointer to the image data buffer.
	data_mask	const char *	Pointer to the mask buffer. Reserved.
	stream_id	string	Unique stream identifier. Required and useful in multi-threaded scenarios.
	width	int	Image width.
	height	int	Image height.
	width_padding	int	Width padding. Used only on 6490 QL.
	height_padding	int	Height padding. Used only on 6490 QL.
	size	uint64_t	Buffer size occupied by the image data.
	fps	double	Frame rate.
	idx	uint64_t	Frame counter.
	frame_time	string	Frame timestamp. Used only on 6490 QL.
	enable_mask_drawing	bool	Whether mask drawing is enabled.
	pipe_status	int	Pipeline status.
	stride	int	Stride value.
	scanline	int	Scanline value.
	pts	int64_t	Presentation timestamp.
	format	string	Image format.

Public Function	Description
void clogger(const char* path_and_prefix)	Creates log files and sets the file path and filename prefix.
void set_log_level(GSTLogLevel log_level)	Sets the log level.
typedef function<int8_t(const Image &)> GetImageCB	Callback type used by stream-start APIs to provide image buffers to application code.
int start_stream(string stream_id, GetImageCB &cb)	Starts a stream by stream_id. This is the recommended interface for most applications.
int start_stream_input_file(string file_path, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)	Starts a file-based stream.
int start_stream_input_rtsp(string rtsp_addr, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)	Starts an RTSP stream.
int start_stream_input_rtmp(string rtmp_addr, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)	Starts an RTMP stream.

Enum	Member	Type	Description
GSTLogLevel	SINFO	uint8_t	Info log level.
	SWARNING	uint8_t	Warning log level.
	SERROR	uint8_t	Error log level.
	SDEBUG	uint8_t	Debug log level.
	SOFF	uint8_t	Logging disabled.
StreamType	EMPTY	uint8_t	No output protocol.
	WAYLANDSINK	uint8_t	Wayland display output.
	RTSPSINK	uint8_t	RTSP output.
	RTMPSINK	uint8_t	RTMP output.
	MP4	uint8_t	MP4 file output.

Image Structure

Image describes frame metadata, buffer pointers, and pipeline-side image properties.

Member	Type	Description
data	const char *	Pointer to the image data buffer.
data_mask	const char *	Pointer to the mask data buffer. Reserved.
stream_id	string	Unique stream identifier. Required and useful in multi-threaded scenarios.
width	int	Image width.
height	int	Image height.
width_padding	int	Width padding. Used only on 6490 QL.
height_padding	int	Height padding. Used only on 6490 QL.
size	uint64_t	Buffer size occupied by the image data.
fps	double	Frame rate.
idx	uint64_t	Frame counter.
frame_time	string	Frame timestamp. Used only on 6490 QL.
enable_mask_drawing	bool	Whether mask drawing is enabled.
pipe_status	int	Pipeline status.
stride	int	Stride value.
scanline	int	Scanline value.
pts	int64_t	Presentation timestamp.
format	string	Image format.

warning

Unless otherwise stated, all definitions belong to the Aidlux::AidStream namespace:

using namespace Aidlux::AidStream;

Start a Stream by Stream ID

API	int start_stream(string stream_id, GetImageCB &cb)
Description	Starts a stream by stream_id. Stream definitions are read from /usr/local/share/aidstream-gst/conf/aidstream-gst.conf. Each stream_id must be unique. This is the strongly recommended interface for production use.
Parameters	stream_id: unique stream configuration identifier. cb: callback function. See GetImageCB. For configuration details, see AidStream configuration reference.
Return Value	0 for success, -1 for failure

Example:

int res = start_stream("stream1", my_cb);

Start a File Stream

API	int start_stream_input_file(string file_path, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)
Description	Starts a file-based stream without relying on a configuration file.
Parameters	file_path: source MP4 file. cb: callback function. display_type: output type. target_path: output target path. stream_id: unique stream identifier. Note: width and height are no longer effective.
Return Value	0 for success, -1 for failure

Example:

int res = start_stream_input_file("./test.mp4", my_cb, StreamType::WAYLANDSINK, "<output-stream-url>/aidstream-gst-test-6");

Start an RTSP Stream

API	int start_stream_input_rtsp(string rtsp_addr, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)
Description	Starts an RTSP stream without relying on a configuration file.
Parameters	rtsp_addr: source RTSP address. cb: callback function. display_type: output type. target_path: output target path. stream_id: unique stream identifier. Note: width and height are no longer effective.
Return Value	0 for success, -1 for failure

Example:

int res = start_stream_input_rtsp("<input-stream-url>/h264/ch1/main/av_stream", my_cb, StreamType::RTSPSINK, "<output-stream-url>/aidstream-gst-test-3");

Start an RTMP Stream

API	int start_stream_input_rtmp(string rtmp_addr, GetImageCB cb, StreamType display_type, string target_path, int width, int height, string stream_id)
Description	Starts an RTMP stream without relying on a configuration file.
Parameters	rtmp_addr: source RTMP address. cb: callback function. display_type: output type. target_path: output target path. stream_id: unique stream identifier. Note: width and height are no longer effective.
Return Value	0 for success, -1 for failure

Example:

int res = start_stream_input_rtmp("<input-stream-url>/h264/ch1/main/av_stream", my_cb, StreamType::EMPTY, "<output-stream-url>/aidstream-gst-test-2");

Create Logs with clogger

API	void clogger(const char* path_and_prefix)
Description	Creates log files and sets the file path and filename prefix.
Parameters	path_and_prefix: log filename with path, for example "./aidclog_aidstream_".
Return Value	None

Example:

const char* log_path = "./aidclog_aidstream_";
Aidlux::AidStream::clogger(log_path);

Set the Log Level

API	void set_log_level(GSTLogLevel log_level)
Description	Sets the log level.
Parameters	log_level: a GSTLogLevel enum value. See GSTLogLevel.
Return Value	None

Example:

Aidlux::AidStream::set_log_level(GSTLogLevel::SINFO);

GSTLogLevel Enum

Member	Type	Description
SINFO	uint8_t	Info log level.
SWARNING	uint8_t	Warning log level.
SERROR	uint8_t	Error log level.
SDEBUG	uint8_t	Debug log level.
SOFF	uint8_t	Logging disabled.

StreamType Enum

Member	Type	Description
EMPTY	uint8_t	No output protocol.
WAYLANDSINK	uint8_t	Wayland display output.
RTSPSINK	uint8_t	RTSP output.
RTMPSINK	uint8_t	RTMP output.
MP4	uint8_t	MP4 file output.

ScenarioType Enum

Member	Type	Description
Pulling_A	int	Read stream → (custom frame processing).
PullingEncode_B	int	Encode stream → stream output.
PullingInferenceOutput_C	int	Read stream → (decode) → AI processing → output results.
PullingInferenceEncode_D	int	Read stream → (decode) → AI processing → stream output.
PullingMultiCompInference_E	int	Merge multiple streams, run inference (Qualcomm QNN via qtimlqnn).
PullingMultiComp_F	int	Merge multiple streams into one.
PullingMultiCompInference_G	int	Merge multiple streams, run inference, save as MP4, and push RTSP.
PullingMultiCompInference_H	int	Merge multiple streams, run inference, save as MP4, and push RTSP.

TaskType Enum

Member	Type	Description
MultiVideoDecode	int	Multi-video decoding task.
MultiVideoEncode	int	Multi-video encoding task.
VideoDecode	int	Video decoding task. Required for all stream pull operations.
InferenceCB	int	Inference callback task. Invokes inference within the callback.
VideoEncode	int	Video encoding task. Required when encoding is needed.
CustomCB	int	Custom callback task. For user-defined callback operations.
InferenceQTI	int	QTI inference task using the Qualcomm qtimlqnn plugin.

AidStreamStatusCode-Enum

Member	Type	Value	Description
AS_OK	int	0	Success.
AS_ERR_INVALID_PARAM	int	-1	Invalid parameter.
AS_ERR_STREAM_EXISTS	int	-2	Stream already exists.
AS_ERR_STREAM_NOT_FOUND	int	-3	Specified stream not found.
AS_ERR_PIPELINE_CREATE	int	-4	Pipeline creation failed.
AS_ERR_NOT_RUNNING	int	-5	Stream is not in running state.
AS_ERR_NOT_SYNCED	int	-6	Stream is not synchronized.
AS_ERR_PUSH_BUFFER	int	-7	Buffer push failed.
AS_ERR_INTERNAL	int	-8	Internal error.

HevcAuInfo Structure

HevcAuInfo describes the parsed HEVC access unit information.

Member	Type	Description
has_vps	bool	Whether the AU contains a VPS (Video Parameter Set).
has_sps	bool	Whether the AU contains an SPS (Sequence Parameter Set).
has_pps	bool	Whether the AU contains a PPS (Picture Parameter Set).
is_keyframe	bool	Whether the AU is a keyframe.
has_slice	bool	Whether the AU contains slice data.

Callback Function: GetImageCB

API	typedef std::function<int8_t(const Image &)> GetImageCB
Description	Callback type used by stream-start APIs to hand image buffers to application code. Users implement the callback themselves; the SDK only defines the callback signature.
Parameters	The callback receives an Image instance containing buffer pointers, dimensions, frame rate, and related metadata. See Image.
Return Value	User-defined

Example:

int8_t my_get_img_cb(const Image &img)
{
    printf("width: %d, height: %d, fps: %f\n", img.width, img.height, img.fps);
    return 0;
}

AidStream Configuration Reference

When using int start_stream(string stream_id, GetImageCB &cb), input and output streams can be configured through /usr/local/share/aidstream-gst/conf/aidstream-gst.conf.

Requirements:

• The configuration file must be in JSON format.
• Parameters marked as required must be provided.
• Parameter names are case-sensitive.

Parameter	Type	Required	Description
streamsId	string	Yes	Stream ID used to uniquely identify one stream configuration.
inputType	string	Yes	Input protocol type. Supported values: rtsp, rtmp, file. Unsupported values or invalid addresses cause startup failure.
inputAddr	string	Yes	Input stream address.
outputType	string	Yes	Output protocol type. Supported values: rtsp, wayland, mp4, empty. empty means output is sent to fakesink without pushing or rendering.
outputAddr	string	Yes	Output stream address.
decodeType	string	No	Hardware decode format. Supported values: H264, H265.
encodeType	string	No	Hardware encode format. Supported values: H264, H265.
audio	bool	No	Enables audio.
saveMp4	bool	No	Saves output as MP4.
outputFile	string	No	Output path for saved MP4 files.
width	int	No	Output video width.
height	int	No	Output video height.
fps	int	No	Output frame rate.
rolling_save	bool	No	Enables rolling recording.
rollingFps	int	No	Frame rate used for rolling recording.
manual_rebuild	bool	No	Enables manual pipeline rebuild.
network_push	bool	No	Enables network pushing.

tip

Placeholder conventions:

• <input-stream-url>: for example, rtsp://admin:aidlux123@192.168.110.234:554
• <output-stream-url>: for example, rtsp://192.168.111.115:8554

Update these addresses according to your local network environment.

Reference example:

{
  "streams": [
    {
      "streamsId": "1",
      "properties": {
        "inputType": "rtsp",
        "inputAddr": "<input-stream-url>/h264/ch1/main/av_stream",
        "outputType": "rtsp",
        "outputAddr": "<output-stream-url>/aidstream-gst-rtsp-test-1",
        "decodeType": "H264",
        "encodeType": "H264"
      }
    }
  ]
}

StreamIns Structure

StreamIns is used as the parameter to start() and exposes a richer stream-configuration model for programmatic startup.

Member	Type	Default	Description
stream_id	string	"empty"	Unique stream identifier.
p_stream_id	string	"empty_p"	Parent stream identifier, used to link parent-child stream relationships.
inputType	string	""	Input stream type: rtsp, rtmp, file, or usbcam.
inputAddr	string	""	Input source address.
outputType	string	""	Output type: rtsp, wayland, mp4, or empty.
outputAddr	string	""	Output target address.
decodeType	string	"H264"	Hardware decode format: H264 or H265.
encodeType	string	"H264"	Hardware encode format: H264 or H265.
audio_switch	bool	false	Enables audio.
save_mp4	bool	false	Saves output as MP4.
network_push	bool	false	Enables network push output. When enabled, separate processing is triggered.
output_file	string	"/tmp/saved_output.mp4"	Output path for MP4 files.
width	int	0	Video width.
height	int	0	Video height.
fps	int	0	Video frame rate.
rolling_save	bool	false	Enables rolling recording.
rollingFps	int	-1	Frame rate used for rolling recording MP4 playback.
manual_rebuild	bool	true	Whether to manually reconnect on EOS or error (true = manual, false = auto).

AidStreamEngine Class

AidStreamEngine provides a newer engine API driven by ScenarioType and TaskType.

Method	Description
AidStreamEngine(ScenarioType scenario_type)	Constructor. Initializes the engine with a scenario type.
~AidStreamEngine()	Destructor.
int add_task(TaskType task_type, std::unordered_map<std::string, std::string>& config)	Adds a processing task. Returns 0 on success, -1 on failure.
void set_callback(TaskType task_type, GetImageCB &callback)	Sets the callback function for the specified task type.
int start()	Starts the engine. Returns 0 on success.
static void stop()	Stops the engine. This is a static method.

Start a Stream with StreamIns

API	int start(StreamIns &stream_ins, GetImageCB &cb)
Description	Starts a stream by configuring all stream parameters programmatically without a configuration file.
Parameters	stream_ins: stream configuration structure containing the full stream definition. cb: callback function.
Return Value	0 for success, -1 or another AidStreamStatusCode value for failure

Refer to [AidStreamStatusCode Enum](#aidstreamstatuscode-enum)

Stop and Release a Stream

API	void gstreamer_pipeline_shutdown(string stream_id)
Description	Stops a stream. You can stop a single stream by stream_id, or stop all streams by passing "all_stream".
Parameters	stream_id: target stream ID or "all_stream".
Return Value	None

Retrieve Rolling Recording Files

API	int get_rolling_save_video(std::string stream_id, std::string output_video, int delay_seconds = 5)
Description	Retrieves the MP4 file currently being generated by the rolling-recording pipeline, merging and exporting a video clip of the specified duration from the rolling buffer. Useful for persisting video clips after exception or event detection.
Parameters	stream_id: stream identifier. output_video: output video file path. delay_seconds: delay in seconds before retrieval (default: 5).
Return Value	0 for success, -1 for failure

USB Camera Interface

API	int start_stream_input_usbcam(StreamIns &stream_ins, GetImageCB cb, StreamType display_type, std::string stream_id)
Description	Starts stream processing from a USB camera.
Parameters	stream_ins: USB camera initialization settings. cb: callback function. display_type: output stream type. stream_id: unique stream identifier.
Return Value	0 for success, -1 for failure

Splice Stream APIs

The splice-stream APIs are intended for raw-stream stitching and mixed display scenarios.

API	int start_splice_stream(std::string stream_id, GetImageCB &cb)
Description	Starts decoding and output for a splice stream.
Parameters	stream_id: unique stream identifier. cb: callback function.
Return Value	0 for success, -1 for failure

API	int stop_splice_stream(std::string stream_id)
Description	Stops a splice stream.
Parameters	stream_id: unique stream identifier.
Return Value	0 for success, -1 for failure

API	int decode_put_frame_splice(const uint8_t *data, size_t size, const std::string &stream_id, int64_t pts = -1, bool is_keyframe = false)
Description	Pushes a video frame into the splice buffer.
Parameters	data: video frame data pointer. size: data size. stream_id: unique stream identifier. pts: presentation timestamp (default: -1). is_keyframe: whether this is a keyframe (default: false).
Return Value	0 for success, -1 for failure

API	HevcAuInfo parse_hevc_au_info_splice(const uint8_t *data, size_t size)
Description	Parses H.265 (HEVC) access unit metadata before splicing.
Parameters	data: HEVC data pointer. size: data size.
Return Value	HevcAuInfo structure containing VPS/SPS/PPS/keyframe/slice information.

Get Stream Properties

API	int get_properties(std::string stream_id, StreamIns &stream_obj)
Description	Reads the stream configuration properties for the specified stream_id from the configuration file and populates the StreamIns object.
Parameters	stream_id: unique stream identifier. stream_obj: reference to a StreamIns object to receive the configuration.
Return Value	0 for success, -1 for failure

Get Stream Status

API	int get_stream_status()
Description	Returns the current running status of the stream. Returns 1 when the stream is actively running.
Parameters	None
Return Value	1: stream is running, 0: stream is idle, -1: uninitialized

Get Image Data

API	cv::Mat get_data(int height, int width, char* data)
Description	Converts raw image data of the specified height and width into an OpenCV cv::Mat object in CV_8UC3 format.
Parameters	height: image height. width: image width. data: pointer to image data buffer.
Return Value	OpenCV cv::Mat object

V4L2 Image Conversion

Converts V4L2-sourced image data to OpenCV BGR-format cv::Mat objects.

API	int v4l2_get_mat(cv::Mat &destMat, Image img)
Description	Converts image data from Image into a BGR-format cv::Mat, cropping to the actual dimensions (removing padding). Designed for the 6490 QL platform.
Parameters	destMat: output BGR-format matrix. img: Image object containing buffer data and dimensions.
Return Value	0 for success, -1 for failure (empty data)

API	int v4l2_get_mat_ori(cv::Mat &destMat, Image img)
Description	Converts image data from Image into a BGR-format cv::Mat (preserves padding, no cropping). Designed for the 6490 QL platform.
Parameters	destMat: output BGR-format matrix. img: Image object containing buffer data and dimensions.
Return Value	0 for success, -1 for failure (empty data)

Start Audio Stream

API	int start_audio_stream(std::string stream_id, GetImageCB &cb)
Description	Starts audio stream processing, used alongside video streams.
Parameters	stream_id: unique stream identifier. cb: callback function.
Return Value	0 for success, -1 for failure

Start Split Stream

API	int start_split_stream(std::string stream_id_1, GetImageCB &cb1, std::string stream_id_2, GetImageCB &cb2)
Description	Starts a single input stream that splits into two output streams. Both stream_id values must share the same parent stream identifier (p_stream_id) and input configuration.
Parameters	stream_id_1: first output stream identifier. cb1: callback for the first stream. stream_id_2: second output stream identifier. cb2: callback for the second stream.
Return Value	0 for success, -1 for failure