Application Test - Helmet Detection

Run Talowe
talowe_helmet_detection_camon CAM126B-P: the camera detects in real time whether people wear helmets (helmet/no-helmet), draws boxes, and pushes H.264 RTP to the computer. Openhelmet.sdpin VLC to view the picture.Source: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo
Inference: RKNN | Capture: V4L2
/dev/video12NV12 1920x1080| Encoding: MPP | Stream receive: UDP5000
Talowe's original project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flash the IPC firmware and you can demonstrate, no need to rebuild rootfs.
Encoding uses a direct NV12 copy from the camera, and boxes are drawn on NV12, avoiding RGB→NV12 causing VLC to be all black.
The procedure on this page is written for rkipc, /dev/video12, S99rkipc).
- For flashing, see CAM126B-P Buildroot IPC (
Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware Reflashing is not required : if the board already has this IPC firmware, demonstrate directly- If it is currently Debian / another image, first flash Buildroot IPC according to that page, then follow the steps below
For IPC streaming, account and password, see Firmware Usage - IPC Streaming.
This Demo shares camera /dev/video12 and UDP 5000 with Human Pose, YOLOv5, YOLO26, YOLO11, and PP-OCR, and cannot run simultaneously.
run_helmet.sh stops the other Demos / rkipc first.
Demonstration (see the effect directly)
Environment as the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.
| Role | Address | Description |
|---|---|---|
| Stream-receiving computer (Mac) | 10.0.0.49 | Install VLC, open helmet.sdp |
| CAM126B-P | ssh root@10.0.2.0 | Default password see Firmware Usage |
| Program directory | /root/rknn_helmet_detection_cam | Already deployed package |
1. Open VLC on the computer first
Copy helmet.sdp to the computer, e.g. ~/helmet.sdp. The content is fixed as follows (port 5000, do not change to the board IP):
v=0
o=- 0 0 IN IP4 0.0.0.0
s=Helmet
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000
The package also contains a play.sdp with the same content (s=Helmet). For this page use helmet.sdp; do not mix it with the pose / YOLO / OCR sdp.
open -a VLC ~/helmet.sdp
Or VLC → File → Open File → select helmet.sdp.
The first time it may ask the local firewall to allow UDP 5000; click Allow.
VLC will stay black waiting for the stream; this is normal. Start the board side in the next step.
2. Start streaming on the board
ssh root@10.0.2.0
cd /root/rknn_helmet_detection_cam
./run_helmet.sh 10.0.0.49
run_helmet.sh will: stop other Demos / rkipc → open /dev/video12 → infer and draw boxes → push H.264 to 10.0.0.49:5000.
When you see logs like the following, the stream is out:
v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
[DET] helmet | box:( 412 , 201)( 890 , 480) | size: 478*279 | conf: 87.32%
[DET] no-helmet | box:( 100 , 180)( 320 , 460) | size: 220*280 | conf: 76.10%
VLC should show a live picture: detection boxes, class name, confidence, and FPS in the upper-left corner.
Wearing a helmet is a green box helmet; not wearing is a red box no-helmet. A complete face / head in the frame gives a better result.
Manual equivalent command (when not using the script):
/etc/init.d/S99rkipc stop
cd /root/rknn_helmet_detection_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_helmet_detection_cam ./model/helmet_detection.rknn /dev/video12 10.0.0.49
The third parameter must be the 10.0.2.0.
3. After the demonstration, restore IPC
After stopping the Demo with Ctrl + C:
killall rknn_helmet_detection_cam rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start
The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.
Board-side package contents
root@rv1126bp-buildroot:~/rknn_helmet_detection_cam# ls
lib model play.sdp helmet.sdp rknn_helmet_detection_cam run_helmet.sh
| File | Purpose |
|---|---|
rknn_helmet_detection_cam | Main program |
run_helmet.sh | Stop rkipc and start; parameter is the computer IP |
helmet.sdp | Copy to the computer for VLC (recommended) |
play.sdp | Same content as helmet.sdp, s=Helmet |
model/helmet_detection.rknn | Detection model (about 11 MB, two classes) |
model/coco_2_labels_list.txt | Class names helmet / no-helmet |
lib/librknnrt.so librga.so librockchip_mpp.so | NPU / RGA / encoding |
Startup parameters:
./rknn_helmet_detection_cam <model_path> <camera_index> <target_ipv4>
<camera_index>: ISP main path; on this board it is/dev/video12(rkisp_mainpathinv4l2-ctl --list-devices)<target_ipv4>: LAN IP of the stream-receiving computer; port is hardcoded 5000
Two classes: helmet (wearing), no-helmet (not wearing). Thresholds are in postprocess.h: BOX_THRESH default 0.35, NMS_THRESH default 0.45.
Compile from source and deploy
You can skip this section if you only want to demonstrate. Follow the steps below when changing code or changing boards.
Machines
| Role | Example |
|---|---|
| Local machine | ~/SDK/talowe-rv1126b-aidemo (Mac only edits code and rsyncs) |
| Build server | ssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64) |
| Board | ssh root@10.0.2.0 |
Use the distribution cross-compiler; no need to build the full SDK host:
# build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake
Sync source
3rdparty/opencv is about 525 MB and is not used by this Demo; do not sync it.
ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_helmet_detection_cam'
rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_helmet_detection_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_helmet_detection_cam/
Confirm model/helmet_detection.rknn is about 11 MB and already in the project model/.
Cross-compile
ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_helmet_detection_cam/cpp
chmod +x build-linux.sh
./build-linux.sh
Output:
~/talowe-rv1126b-aidemo/talowe_helmet_detection_cam/cpp/install/rv1126b_linux_aarch64/rknn_helmet_detection_cam/
# helmet.sdp lib model play.sdp rknn_helmet_detection_cam run_helmet.sh
CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the board's existing librockchip_mpp (a copy of the so is included in the package).
Push to the board
The board's Buildroot usually has no rsync; create a tar on the build machine and extract it:
ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_helmet_detection_cam/cpp/install/rv1126b_linux_aarch64 rknn_helmet_detection_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'
You can also use this when USB is connected via adb:
adb push rknn_helmet_detection_cam /root/
Then go back to steps 1 and 2 in Demonstration above.
Data path
IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ copy one NV12 frame for encoding (boxes / text)
→ RGA convert to RGB888
→ RKNN helmet_detection (helmet / no-helmet)
→ draw boxes / class names / FPS on NV12
→ MPP H.264 (each IDR carries SPS/PPS)
→ RTP PT=96 UDP → computer:5000
→ VLC opens helmet.sdp
The resolution and port are written in cpp/main.cc, cpp/run_helmet.sh, helmet.sdp / play.sdp; when changing 1080p / 5000, change all three places.
FAQ
| Symptom | Handling |
|---|---|
| VLC stays black | Close VLC first, reopen helmet.sdp, then start the board side; the third parameter must be the computer IP; allow UDP 5000 locally |
Camera open failed / VIDIOC_S_FMT failed | Other Demos / rkipc are still running. ./run_helmet.sh stops them |
| All black after opening the camera | ISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is inserted |
| Picture present but no boxes | The head must fully enter the frame; thresholds are in postprocess.h BOX_THRESH (default 0.35) |
Class names are all null / garbled | You must start from the package root directory; post-processing reads ./model/coco_2_labels_list.txt |
| Web page / RTSP gone after demonstration | rkipc not restored: killall rknn_helmet_detection_cam then /etc/init.d/S99rkipc start |
Build missing aarch64-linux-gnu-gcc | On the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu |
The RKNN model can not be found | helmet_detection.rknn was not placed in the project model/ before compiling |
Camera node confirmation:
v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12
Differences from pose / YOLO / OCR
| Human Pose | YOLO11 | PP-OCR | This page helmet | |
|---|---|---|---|---|
| Project | talowe_yolov8_pose_cam | talowe_yolo11_detection_cam | talowe_ppocr_cam | talowe_helmet_detection_cam |
| Model | yolov8_pose.rknn | yolo11.rknn ~4.2 MB | det + rec two models | helmet_detection.rknn ~11 MB |
| Output | Boxes + 17-keypoint skeleton | 80 COCO classes | Text boxes + recognized strings | helmet / no-helmet |
| Stream receive | VLC play.sdp | VLC yolo11.sdp | VLC ppocr.sdp | VLC helmet.sdp |
| With rkipc | Must stop | Must stop | Must stop | Must stop |
The network structure is the same as YOLO11, only the classes are changed to helmet binary classification.
Download source code and prebuilt packages
Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc
The cloud drive contains two archives:
| Package | Purpose |
|---|---|
talowe-rv1126b-aidemo | Source code, cross-compile yourself |
talowe-rv1126b-aidemo-app | Already compiled board-side programs (aarch64); copy to /root/ on the board and they run |
Notes for sending to customers / using yourself:
run_*.shpushes H.264 to10.0.0.49:5000by default. This is an example computer IP and must be changed to the stream-receiving computer's own LAN IP:./run_xxx.sh <computer IP>.- Firmware must be CAM126B-P Buildroot IPC. The Demo and
rkipcshare/dev/video12and cannot run simultaneously; the script stopsrkipcfirst. - Open VLC on the computer with the sdp corresponding to this page (this page is
helmet.sdp, port 5000); do not mix it with other Demos' sdp. - In
talowe-rv1126b-aidemo-app, please ignorerknn_yolov5_demo(leftover from the old image inference). For live camera detection userknn_yolov5_detection_cam. - Most Demos in the source have been changed to V4L2 + RGA + MPP; no need to include the entire
3rdparty/opencv(about 11 × 525 MB). Only PP-OCR cross-compilation still needs the in-project OpenCV static library.
Summary
- Open
helmet.sdpin VLC on the computer (wait for UDP 5000) - On the board
cd /root/rknn_helmet_detection_cam && ./run_helmet.sh 10.0.0.49 - After screenshot / demonstration
killall rknn_helmet_detection_cam,/etc/init.d/S99rkipc start