Application Test - General Object Detection (YOLO11)

Run Talowe
talowe_yolo11_detection_camon CAM126B-P: the camera detects 80 COCO classes in real time, draws boxes, and pushes H.264 RTP to the computer. Openyolo11.sdpin VLC to view the picture.Source: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo
Inference: RKNN | Capture: V4L2
/dev/video12NV12 1920x1080| Encoding: MPP | Stream receive: UDP5000
Talowe's original project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flash the IPC firmware and you can demonstrate, no need to rebuild rootfs.
Encoding uses a direct NV12 copy from the camera, and boxes are drawn on NV12, avoiding RGB→NV12 causing VLC to be all black.
The procedure on this page is written for rkipc, /dev/video12, S99rkipc).
- For flashing, see CAM126B-P Buildroot IPC (
Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware Reflashing is not required : if the board already has this IPC firmware, demonstrate directly- If it is currently Debian / another image, first flash Buildroot IPC according to that page, then follow the steps below
For IPC streaming, account and password, see Firmware Usage - IPC Streaming.
This Demo shares camera /dev/video12 and UDP 5000 with Human Pose, YOLOv5, YOLO26, and Facial Features, and cannot run simultaneously.
run_yolo11.sh stops the other Demos / rkipc first.
Demonstration (see the effect directly)
Environment as the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.
| Role | Address | Description |
|---|---|---|
| Stream-receiving computer (Mac) | 10.0.0.49 | Install VLC, open yolo11.sdp |
| CAM126B-P | ssh root@10.0.2.0 | Default password see Firmware Usage |
| Program directory | /root/rknn_yolo11_detection_cam | Already deployed package |
1. Open VLC on the computer first
Copy yolo11.sdp to the computer, e.g. ~/yolo11.sdp. The content is fixed as follows (port 5000, do not change to the board IP):
v=0
o=- 0 0 IN IP4 0.0.0.0
s=YOLO11
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000
The package also contains a play.sdp with the same content (s=YOLO11). For this page use yolo11.sdp; do not mix it with the pose / YOLOv5 / YOLO26 sdp.
open -a VLC ~/yolo11.sdp
Or VLC → File → Open File → select yolo11.sdp.
The first time it may ask the local firewall to allow UDP 5000; click Allow.
VLC will stay black waiting for the stream; this is normal. Start the board side in the next step.
2. Start streaming on the board
ssh root@10.0.2.0
cd /root/rknn_yolo11_detection_cam
./run_yolo11.sh 10.0.0.49
run_yolo11.sh will: stop other Demos / rkipc → open /dev/video12 → infer and draw boxes → push H.264 to 10.0.0.49:5000.
When you see logs like the following, the stream is out:
v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
[DET] person | box:( 412 , 201)( 890 , 980) | size: 478*779 | conf: 87.32%
VLC should show a live picture: detection boxes, class name, confidence, and FPS in the upper-left corner. COCO objects such as people, cups, keyboards, and phones will be boxed when they enter the frame.
Manual equivalent command (when not using the script):
/etc/init.d/S99rkipc stop
cd /root/rknn_yolo11_detection_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_yolo11_detection_cam ./model/yolo11.rknn /dev/video12 10.0.0.49
The third parameter must be the 10.0.2.0.
3. After the demonstration, restore IPC
After stopping the Demo with Ctrl + C:
killall rknn_yolo11_detection_cam rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start
The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.
Board-side package contents
root@rv1126bp-buildroot:~/rknn_yolo11_detection_cam# ls
lib model play.sdp rknn_yolo11_detection_cam run_yolo11.sh yolo11.sdp
| File | Purpose |
|---|---|
rknn_yolo11_detection_cam | Main program |
run_yolo11.sh | Stop rkipc and start; parameter is the computer IP |
yolo11.sdp | Copy to the computer for VLC (recommended) |
play.sdp | Same content as yolo11.sdp, s=YOLO11 |
model/yolo11.rknn | Detection model (about 4.2 MB, 80 COCO classes) |
model/coco_80_labels_list.txt | Class names; post-processing reads ./model/coco_80_labels_list.txt |
lib/librknnrt.so librga.so librockchip_mpp.so | NPU / RGA / encoding |
Startup parameters:
./rknn_yolo11_detection_cam <model_path> <camera_index> <target_ipv4>
<camera_index>: ISP main path; on this board it is/dev/video12(rkisp_mainpathinv4l2-ctl --list-devices)<target_ipv4>: LAN IP of the stream-receiving computer; port is hardcoded 5000
The 80 COCO classes include person, bicycle, car, bottle, cup, chair, laptop, cell phone, etc. Thresholds are in postprocess.h: BOX_THRESH default 0.25, NMS_THRESH default 0.45.
Compile from source and deploy
You can skip this section if you only want to demonstrate. Follow the steps below when changing code or changing boards.
Machines
| Role | Example |
|---|---|
| Local machine | ~/SDK/talowe-rv1126b-aidemo (Mac only edits code and rsyncs) |
| Build server | ssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64) |
| Board | ssh root@10.0.2.0 |
Use the distribution cross-compiler; no need to build the full SDK host:
# build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake
Sync source
3rdparty/opencv is about 525 MB and is not used by this Demo; do not sync it.
ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam'
rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam/
Confirm model/yolo11.rknn is about 4.2 MB and already in the project model/.
Cross-compile
ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam/cpp
chmod +x build-linux.sh
./build-linux.sh
Output:
~/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam/cpp/install/rv1126b_linux_aarch64/rknn_yolo11_detection_cam/
# lib model play.sdp rknn_yolo11_detection_cam run_yolo11.sh yolo11.sdp
CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the board's existing librockchip_mpp (a copy of the so is included in the package).
Push to the board
The board's Buildroot usually has no rsync; create a tar on the build machine and extract it:
ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_yolo11_detection_cam/cpp/install/rv1126b_linux_aarch64 rknn_yolo11_detection_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'
You can also use this when USB is connected via adb:
adb push rknn_yolo11_detection_cam /root/
Then go back to steps 1 and 2 in Demonstration above.
Data path
IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ copy one NV12 frame for encoding (boxes / text)
→ RGA convert to RGB888
→ RKNN YOLO11 (80 COCO classes)
→ draw boxes / class names / FPS on NV12
→ MPP H.264 (each IDR carries SPS/PPS)
→ RTP PT=96 UDP → computer:5000
→ VLC opens yolo11.sdp
The resolution and port are written in cpp/main.cc, cpp/run_yolo11.sh, yolo11.sdp / play.sdp; when changing 1080p / 5000, change all three places.
FAQ
| Symptom | Handling |
|---|---|
| VLC stays black | Close VLC first, reopen yolo11.sdp, then start the board side; the third parameter must be the computer IP; allow UDP 5000 locally |
Camera open failed / VIDIOC_S_FMT failed | Other Demos / rkipc are still running. ./run_yolo11.sh stops them |
| All black after opening the camera | ISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is inserted |
| Picture present but no boxes | The object must fully enter the frame; thresholds are in postprocess.h BOX_THRESH (default 0.25) |
Class names are all null / garbled | You must start from the package root directory; post-processing reads ./model/coco_80_labels_list.txt |
| Web page / RTSP gone after demonstration | rkipc not restored: killall rknn_yolo11_detection_cam then /etc/init.d/S99rkipc start |
Build missing aarch64-linux-gnu-gcc | On the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu |
The RKNN model can not be found | yolo11.rknn was not placed in the project model/ before compiling |
Camera node confirmation:
v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12
Differences from pose / YOLOv5 / YOLO26
| Human Pose | YOLOv5 | YOLO26 | This page YOLO11 | |
|---|---|---|---|---|
| Project | talowe_yolov8_pose_cam | talowe_yolov5_detection_cam | talowe_yolo26_detection_cam | talowe_yolo11_detection_cam |
| Model | yolov8_pose.rknn | yolov5.rknn ~7.4 MB | yolo26.rknn ~4.2 MB | yolo11.rknn ~4.2 MB |
| Output | Boxes + 17-keypoint skeleton | Boxes + 80 COCO class names | Boxes + 80 COCO class names | Boxes + 80 COCO class names |
| Stream receive | VLC play.sdp | VLC yolov5.sdp | VLC yolo26.sdp | VLC yolo11.sdp |
| With rkipc | Must stop | Must stop | Must stop | Must stop |
The directory name in the README is talowe_yolov11_detection_cam; the actual repository directory is talowe_yolo11_detection_cam.
Download source code and prebuilt packages
Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc
The cloud drive contains two archives:
| Package | Purpose |
|---|---|
talowe-rv1126b-aidemo | Source code, cross-compile yourself |
talowe-rv1126b-aidemo-app | Already compiled board-side programs (aarch64); copy to /root/ on the board and they run |
Notes for sending to customers / using yourself:
run_*.shpushes H.264 to10.0.0.49:5000by default. This is an example computer IP and must be changed to the stream-receiving computer's own LAN IP:./run_xxx.sh <computer IP>.- Firmware must be CAM126B-P Buildroot IPC. The Demo and
rkipcshare/dev/video12and cannot run simultaneously; the script stopsrkipcfirst. - Open VLC on the computer with the sdp corresponding to this page (this page is
yolo11.sdp, port 5000); do not mix it with other Demos' sdp. - In
talowe-rv1126b-aidemo-app, please ignorerknn_yolov5_demo(leftover from the old image inference). For live camera detection userknn_yolov5_detection_cam. - Most Demos in the source have been changed to V4L2 + RGA + MPP; no need to include the entire
3rdparty/opencv(about 11 × 525 MB). Only PP-OCR cross-compilation still needs the in-project OpenCV static library.
Summary
- Open
yolo11.sdpin VLC on the computer (wait for UDP 5000) - On the board
cd /root/rknn_yolo11_detection_cam && ./run_yolo11.sh 10.0.0.49 - After screenshot / demonstration
killall rknn_yolo11_detection_cam,/etc/init.d/S99rkipc start