Application Test - Mask Wearing Detection

Run Talowe
talowe_mask_detection_camon CAM126B-P: the camera detects in real time whether people wear masks (mask/no_mask/mask_weared_incorrect), draws boxes, and pushes H.264 RTP to the computer. Openmask.sdpin VLC to view the picture.Source: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo
Inference: RKNN | Capture: V4L2
/dev/video12NV12 1920x1080| Encoding: MPP | Stream receive: UDP5000
Talowe's original project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flash the IPC firmware and you can demonstrate, no need to rebuild rootfs.
Encoding uses a direct NV12 copy from the camera, and boxes are drawn on NV12, avoiding RGB→NV12 causing VLC to be all black.
The procedure on this page is written for rkipc, /dev/video12, S99rkipc).
- For flashing, see CAM126B-P Buildroot IPC (
Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware Reflashing is not required : if the board already has this IPC firmware, demonstrate directly- If it is currently Debian / another image, first flash Buildroot IPC according to that page, then follow the steps below
For IPC streaming, account and password, see Firmware Usage - IPC Streaming.
This Demo shares camera /dev/video12 and UDP 5000 with Human Pose, YOLO11, Facial Features, Helmet, and Segmentation, and cannot run simultaneously.
run_mask.sh stops the other Demos / rkipc first.
Demonstration (see the effect directly)
Environment as the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.
| Role | Address | Description |
|---|---|---|
| Stream-receiving computer (Mac) | 10.0.0.49 | Install VLC, open mask.sdp |
| CAM126B-P | ssh root@10.0.2.0 | Default password see Firmware Usage |
| Program directory | /root/rknn_mask_detection_cam | Already deployed package |
1. Open VLC on the computer first
Copy mask.sdp to the computer, e.g. ~/mask.sdp. The content is fixed as follows (port 5000, do not change to the board IP):
v=0
o=- 0 0 IN IP4 0.0.0.0
s=Mask
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000
The package also contains a play.sdp with the same content (s=Mask). For this page use mask.sdp; do not mix it with other Demos' sdp.
open -a VLC ~/mask.sdp
Or VLC → File → Open File → select mask.sdp.
The first time it may ask the local firewall to allow UDP 5000; click Allow.
VLC will stay black waiting for the stream; this is normal. Start the board side in the next step.
2. Start streaming on the board
ssh root@10.0.2.0
cd /root/rknn_mask_detection_cam
./run_mask.sh 10.0.0.49
run_mask.sh will: stop other Demos / rkipc → open /dev/video12 → infer and draw boxes → push H.264 to 10.0.0.49:5000.
When you see logs like the following, the stream is out:
v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
[DET] mask | box:( 412 , 201)( 890 , 480) | size: 478*279 | conf: 87.32%
[DET] no_mask | box:( 100 , 180)( 320 , 460) | size: 220*280 | conf: 76.10%
[DET] mask_weared_incorrect | box:( 520 , 240)( 780 , 510) | size: 260*270 | conf: 64.50%
VLC should show a live picture: detection boxes, class name, confidence, and FPS in the upper-left corner.
Worn correctly is a green box mask; not worn is a red box no_mask; worn incorrectly is an orange box mask_weared_incorrect. A complete face in the frame gives a better result.
Manual equivalent command (when not using the script):
/etc/init.d/S99rkipc stop
cd /root/rknn_mask_detection_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_mask_detection_cam ./model/mask_detection.rknn /dev/video12 10.0.0.49
The third parameter must be the 10.0.2.0.
3. After the demonstration, restore IPC
After stopping the Demo with Ctrl + C:
killall rknn_mask_detection_cam rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start
The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.
Board-side package contents
root@rv1126bp-buildroot:~/rknn_mask_detection_cam# ls
lib model play.sdp mask.sdp rknn_mask_detection_cam run_mask.sh
| File | Purpose |
|---|---|
rknn_mask_detection_cam | Main program |
run_mask.sh | Stop rkipc and start; parameter is the computer IP |
mask.sdp | Copy to the computer for VLC (recommended) |
play.sdp | Same content as mask.sdp, s=Mask |
model/mask_detection.rknn | Detection model (about 8.0 MB, three classes) |
model/coco_3_labels_list.txt | Class names mask / no_mask / mask_weared_incorrect |
lib/librknnrt.so librga.so librockchip_mpp.so | NPU / RGA / encoding |
Startup parameters:
./rknn_mask_detection_cam <model_path> <camera_index> <target_ipv4>
<camera_index>: ISP main path; on this board it is/dev/video12(rkisp_mainpathinv4l2-ctl --list-devices)<target_ipv4>: LAN IP of the stream-receiving computer; port is hardcoded 5000
Three classes: mask (wearing), no_mask (not wearing), mask_weared_incorrect (worn incorrectly). Thresholds are in postprocess.h: BOX_THRESH default 0.35, NMS_THRESH default 0.45.
Compile from source and deploy
You can skip this section if you only want to demonstrate. Follow the steps below when changing code or changing boards.
Machines
| Role | Example |
|---|---|
| Local machine | ~/SDK/talowe-rv1126b-aidemo (Mac only edits code and rsyncs) |
| Build server | ssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64) |
| Board | ssh root@10.0.2.0 |
Use the distribution cross-compiler; no need to build the full SDK host:
# build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake
Sync source
3rdparty/opencv is about 525 MB and is not used by this Demo; do not sync it.
ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_mask_detection_cam'
rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_mask_detection_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_mask_detection_cam/
Confirm model/mask_detection.rknn is about 8.0 MB and already in the project model/.
Cross-compile
ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_mask_detection_cam/cpp
chmod +x build-linux.sh
./build-linux.sh
Output:
~/talowe-rv1126b-aidemo/talowe_mask_detection_cam/cpp/install/rv1126b_linux_aarch64/rknn_mask_detection_cam/
# lib mask.sdp model play.sdp rknn_mask_detection_cam run_mask.sh
CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the board's existing librockchip_mpp (a copy of the so is included in the package).
Push to the board
The board's Buildroot usually has no rsync; create a tar on the build machine and extract it:
ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_mask_detection_cam/cpp/install/rv1126b_linux_aarch64 rknn_mask_detection_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'
You can also use this when USB is connected via adb:
adb push rknn_mask_detection_cam /root/
Then go back to steps 1 and 2 in Demonstration above.
Data path
IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ copy one NV12 frame for encoding (boxes / text)
→ RGA convert to RGB888
→ RKNN mask_detection (mask / no_mask / mask_weared_incorrect)
→ draw boxes / class names / FPS on NV12
→ MPP H.264 (each IDR carries SPS/PPS)
→ RTP PT=96 UDP → computer:5000
→ VLC opens mask.sdp
The resolution and port are written in cpp/main.cc, cpp/run_mask.sh, mask.sdp / play.sdp; when changing 1080p / 5000, change all three places.
FAQ
| Symptom | Handling |
|---|---|
| VLC stays black | Close VLC first, reopen mask.sdp, then start the board side; the third parameter must be the computer IP; allow UDP 5000 locally |
Camera open failed / VIDIOC_S_FMT failed | Other Demos / rkipc are still running. ./run_mask.sh stops them |
| All black after opening the camera | ISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is inserted |
| Picture present but no boxes | The face must fully enter the frame; thresholds are in postprocess.h BOX_THRESH (default 0.35) |
Class names are all null / garbled | You must start from the package root directory; post-processing reads ./model/coco_3_labels_list.txt |
| Web page / RTSP gone after demonstration | rkipc not restored: killall rknn_mask_detection_cam then /etc/init.d/S99rkipc start |
Build missing aarch64-linux-gnu-gcc | On the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu |
The RKNN model can not be found | mask_detection.rknn was not placed in the project model/ before compiling |
Camera node confirmation:
v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12
Differences from helmet / detection
| Helmet | YOLOv5 | This page mask | |
|---|---|---|---|
| Project | talowe_helmet_detection_cam | talowe_yolov5_detection_cam | talowe_mask_detection_cam |
| Model | helmet_detection.rknn ~11 MB | yolov5.rknn ~7.4 MB | mask_detection.rknn ~8.0 MB |
| Output | helmet / no-helmet | 80 COCO classes | mask / no_mask / mask_weared_incorrect |
| Stream receive | VLC helmet.sdp | VLC yolov5.sdp | VLC mask.sdp |
| With rkipc | Must stop | Must stop | Must stop |
The network structure is the same as YOLOv5 Detection; only the classes are changed to mask three-class classification.
Download source code and prebuilt packages
Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc
The cloud drive contains two archives:
| Package | Purpose |
|---|---|
talowe-rv1126b-aidemo | Source code, cross-compile yourself |
talowe-rv1126b-aidemo-app | Already compiled board-side programs (aarch64); copy to /root/ on the board and they run |
Notes for sending to customers / using yourself:
run_*.shpushes H.264 to10.0.0.49:5000by default. This is an example computer IP and must be changed to the stream-receiving computer's own LAN IP:./run_xxx.sh <computer IP>.- Firmware must be CAM126B-P Buildroot IPC. The Demo and
rkipcshare/dev/video12and cannot run simultaneously; the script stopsrkipcfirst. - Open VLC on the computer with the sdp corresponding to this page (this page is
mask.sdp, port 5000); do not mix it with other Demos' sdp. - In
talowe-rv1126b-aidemo-app, please ignorerknn_yolov5_demo(leftover from the old image inference). For live camera detection userknn_yolov5_detection_cam. - Most Demos in the source have been changed to V4L2 + RGA + MPP; no need to include the entire
3rdparty/opencv(about 11 × 525 MB). Only PP-OCR cross-compilation still needs the in-project OpenCV static library.
Summary
- Open
mask.sdpin VLC on the computer (wait for UDP 5000) - On the board
cd /root/rknn_mask_detection_cam && ./run_mask.sh 10.0.0.49 - After screenshot / demonstration
killall rknn_mask_detection_cam,/etc/init.d/S99rkipc start