Skip to main content

Application Test - General Object Detection (YOLO26)

Navigation bar

Run Talowe talowe_yolo26_detection_cam on CAM126B-P: the camera detects 80 COCO classes in real time, draws boxes, and pushes H.264 RTP to the computer. Open yolo26.sdp in VLC to view the picture.

Source: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo

Inference: RKNN | Capture: V4L2 /dev/video12 NV12 1920x1080 | Encoding: MPP | Stream receive: UDP 5000

Information

Talowe's original project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flash the IPC firmware and you can demonstrate, no need to rebuild rootfs.
Encoding uses a direct NV12 copy from the camera, and boxes are drawn on NV12, avoiding RGB→NV12 causing VLC to be all black.

Firmware

The procedure on this page is written for CAM126B-P Buildroot IPC (rkipc, /dev/video12, S99rkipc).

  • For flashing, see CAM126B-P Buildroot IPC (Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware
  • Reflashing is not required: if the board already has this IPC firmware, demonstrate directly
  • If it is currently Debian / another image, first flash Buildroot IPC according to that page, then follow the steps below

For IPC streaming, account and password, see Firmware Usage - IPC Streaming.

Warning

This Demo shares camera /dev/video12 and UDP 5000 with Human Pose, YOLOv5 Detection, and YOLO11, and cannot run simultaneously.
run_yolo26.sh stops the pose / YOLOv5 / rkipc processes first.


Demonstration (see the effect directly)​

Environment as the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.

RoleAddressDescription
Stream-receiving computer (Mac)10.0.0.49Install VLC, open yolo26.sdp
CAM126B-Pssh root@10.0.2.0Default password see Firmware Usage
Program directory/root/rknn_yolo26_detection_camAlready deployed package

1. Open VLC on the computer first​

Copy yolo26.sdp to the computer, e.g. ~/yolo26.sdp. The content is fixed as follows (port 5000, do not change to the board IP):

v=0
o=- 0 0 IN IP4 0.0.0.0
s=YOLO26
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000

The package also contains a play.sdp with the same content (s=YOLO26). For this page use yolo26.sdp; do not mix it with the pose play.sdp or YOLOv5 yolov5.sdp.

open -a VLC ~/yolo26.sdp

Or VLC → File → Open File → select yolo26.sdp.
The first time it may ask the local firewall to allow UDP 5000; click Allow.

VLC will stay black waiting for the stream; this is normal. Start the board side in the next step.

2. Start streaming on the board​

ssh root@10.0.2.0
cd /root/rknn_yolo26_detection_cam
./run_yolo26.sh 10.0.0.49

run_yolo26.sh will: stop other Demos / rkipc → open /dev/video12 → infer and draw boxes → push H.264 to 10.0.0.49:5000.

When you see logs like the following, the stream is out:

v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
[DET] person | box:( 412 , 201)( 890 , 980) | size: 478*779 | conf: 87.32%

VLC should show a live picture: detection boxes, class name, confidence, and FPS in the upper-left corner. COCO objects such as people, cups, keyboards, and phones will be boxed when they enter the frame.

Manual equivalent command (when not using the script):

/etc/init.d/S99rkipc stop
cd /root/rknn_yolo26_detection_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_yolo26_detection_cam ./model/yolo26.rknn /dev/video12 10.0.0.49

The third parameter must be the computer IP, not 10.0.2.0.

3. After the demonstration, restore IPC​

After stopping the Demo with Ctrl + C:

killall rknn_yolo26_detection_cam rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start

The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.


Board-side package contents​

root@rv1126bp-buildroot:~/rknn_yolo26_detection_cam# ls
lib model play.sdp rknn_yolo26_detection_cam run_yolo26.sh yolo26.sdp
FilePurpose
rknn_yolo26_detection_camMain program
run_yolo26.shStop rkipc and start; parameter is the computer IP
yolo26.sdpCopy to the computer for VLC (recommended)
play.sdpSame content as yolo26.sdp, s=YOLO26
model/yolo26.rknnDetection model (about 4.2 MB, 80 COCO classes)
model/coco_80_labels_list.txtClass names; post-processing reads ./model/coco_80_labels_list.txt
lib/librknnrt.so librga.so librockchip_mpp.soNPU / RGA / encoding

Startup parameters:

./rknn_yolo26_detection_cam <model_path> <camera_index> <target_ipv4>
  • <camera_index>: ISP main path; on this board it is /dev/video12 (rkisp_mainpath in v4l2-ctl --list-devices)
  • <target_ipv4>: LAN IP of the stream-receiving computer; port is hardcoded 5000

The 80 COCO classes include person, bicycle, car, bottle, cup, chair, laptop, cell phone, etc. Thresholds are in postprocess.h: BOX_THRESH default 0.25, NMS_THRESH default 0.45.


Compile from source and deploy​

You can skip this section if you only want to demonstrate. Follow the steps below when changing code or changing boards.

Machines​

RoleExample
Local machine~/SDK/talowe-rv1126b-aidemo (Mac only edits code and rsyncs)
Build serverssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64)
Boardssh root@10.0.2.0

Use the distribution cross-compiler; no need to build the full SDK host:

# build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake

Sync source​

3rdparty/opencv is about 525 MB and is not used by this Demo; do not sync it.

ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam'

rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam/

Confirm model/yolo26.rknn is about 4.2 MB and already in the project model/.

Cross-compile​

ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam/cpp
chmod +x build-linux.sh
./build-linux.sh

Output:

~/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam/cpp/install/rv1126b_linux_aarch64/rknn_yolo26_detection_cam/
# lib model play.sdp rknn_yolo26_detection_cam run_yolo26.sh yolo26.sdp

CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the board's existing librockchip_mpp (a copy of the so is included in the package).

Push to the board​

The board's Buildroot usually has no rsync; create a tar on the build machine and extract it:

ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_yolo26_detection_cam/cpp/install/rv1126b_linux_aarch64 rknn_yolo26_detection_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'

You can also use this when USB is connected via adb:

adb push rknn_yolo26_detection_cam /root/

Then go back to steps 1 and 2 in Demonstration above.


Data path​

IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ copy one NV12 frame for encoding (boxes / text)
→ RGA convert to RGB888
→ RKNN YOLO26 (80 COCO classes)
→ draw boxes / class names / FPS on NV12
→ MPP H.264 (each IDR carries SPS/PPS)
→ RTP PT=96 UDP → computer:5000
→ VLC opens yolo26.sdp

The resolution and port are written in cpp/main.cc, cpp/run_yolo26.sh, yolo26.sdp / play.sdp; when changing 1080p / 5000, change all three places.


FAQ​

SymptomHandling
VLC stays blackClose VLC first, reopen yolo26.sdp, then start the board side; the third parameter must be the computer IP; allow UDP 5000 locally
Camera open failed / VIDIOC_S_FMT failedPose / YOLOv5 / rkipc is still running. ./run_yolo26.sh stops them
All black after opening the cameraISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is inserted
Picture present but no boxesThe object must fully enter the frame; thresholds are in postprocess.h BOX_THRESH (default 0.25)
Class names are all null / garbledYou must start from the package root directory; post-processing reads ./model/coco_80_labels_list.txt
Web page / RTSP gone after demonstrationrkipc not restored: killall rknn_yolo26_detection_cam then /etc/init.d/S99rkipc start
Build missing aarch64-linux-gnu-gccOn the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
The RKNN model can not be foundyolo26.rknn was not placed in the project model/ before compiling

Camera node confirmation:

v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12

Differences from pose / YOLOv5 / YOLO11​

Human PoseYOLOv5This page YOLO26YOLO11
Projecttalowe_yolov8_pose_camtalowe_yolov5_detection_camtalowe_yolo26_detection_camtalowe_yolo11_detection_cam
Modelyolov8_pose.rknnyolov5.rknn ~7.4 MByolo26.rknn ~4.2 MByolo11.rknn ~4.2 MB
OutputBoxes + 17-keypoint skeletonBoxes + 80 COCO class namesBoxes + 80 COCO class namesBoxes + 80 COCO class names
Stream receiveVLC play.sdpVLC yolov5.sdpVLC yolo26.sdpVLC yolo11.sdp
With rkipcMust stopMust stopMust stopMust stop

The directory name in the README is talowe_yolov26_detection_cam; the actual repository directory is talowe_yolo26_detection_cam.


Download source code and prebuilt packages​

Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc

The cloud drive contains two archives:

PackagePurpose
talowe-rv1126b-aidemoSource code, cross-compile yourself
talowe-rv1126b-aidemo-appAlready compiled board-side programs (aarch64); copy to /root/ on the board and they run

Notes for sending to customers / using yourself:

  1. run_*.sh pushes H.264 to 10.0.0.49:5000 by default. This is an example computer IP and must be changed to the stream-receiving computer's own LAN IP: ./run_xxx.sh <computer IP>.
  2. Firmware must be CAM126B-P Buildroot IPC. The Demo and rkipc share /dev/video12 and cannot run simultaneously; the script stops rkipc first.
  3. Open VLC on the computer with the sdp corresponding to this page (this page is yolo26.sdp, port 5000); do not mix it with other Demos' sdp.
  4. In talowe-rv1126b-aidemo-app, please ignore rknn_yolov5_demo (leftover from the old image inference). For live camera detection use rknn_yolov5_detection_cam.
  5. Most Demos in the source have been changed to V4L2 + RGA + MPP; no need to include the entire 3rdparty/opencv (about 11 × 525 MB). Only PP-OCR cross-compilation still needs the in-project OpenCV static library.

Summary​

  1. Open yolo26.sdp in VLC on the computer (wait for UDP 5000)
  2. On the board cd /root/rknn_yolo26_detection_cam && ./run_yolo26.sh 10.0.0.49
  3. After screenshot / demonstration killall rknn_yolo26_detection_cam, /etc/init.d/S99rkipc start
AI Translation Notice

This page was translated into English by AI and is provided for reference only. The original document is written in Chinese. If there is any discrepancy, the Chinese version prevails.

本页英文内容由 AI 自动翻译生成,仅供参考;原文为中文,如有差异请以中文原文为准。

查看中文原文 / View the Chinese original →