Application Test - Garbage Classification Detection

Run Talowe
talowe_yolov8_garbage_sort_camon CAM126B-P: the camera detects four garbage classes in real time (recyclable / kitchen / hazardous / other), draws boxes, and pushes H.264 RTP to the computer. Opengarbage.sdpin VLC to view the picture.Source: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo
Inference: RKNN | Capture: V4L2
/dev/video12NV12 1920x1080| Encoding: MPP | Stream receive: UDP5000
Talowe's original project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flash the IPC firmware and you can demonstrate, no need to rebuild rootfs.
Encoding uses a direct NV12 copy from the camera, and boxes are drawn on NV12, avoiding RGB→NV12 causing VLC to be all black.
The directory name in the README is talowe_garbage_classification_cam; the actual repository directory is talowe_yolov8_garbage_sort_cam.
The procedure on this page is written for rkipc, /dev/video12, S99rkipc).
- For flashing, see CAM126B-P Buildroot IPC (
Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware Reflashing is not required : if the board already has this IPC firmware, demonstrate directly- If it is currently Debian / another image, first flash Buildroot IPC according to that page, then follow the steps below
For IPC streaming, account and password, see Firmware Usage - IPC Streaming.
This Demo shares camera /dev/video12 and UDP 5000 with Human Pose, YOLO11, PP-OCR, and Helmet, and cannot run simultaneously.
run_garbage.sh stops the other Demos / rkipc first.
Demonstration (see the effect directly)
Environment as the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.
| Role | Address | Description |
|---|---|---|
| Stream-receiving computer (Mac) | 10.0.0.49 | Install VLC, open garbage.sdp |
| CAM126B-P | ssh root@10.0.2.0 | Default password see Firmware Usage |
| Program directory | /root/rknn_yolov8_garbage_sort_cam | Already deployed package |
1. Open VLC on the computer first
Copy garbage.sdp to the computer, e.g. ~/garbage.sdp. The content is fixed as follows (port 5000, do not change to the board IP):
v=0
o=- 0 0 IN IP4 0.0.0.0
s=Garbage
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000
The package also contains a play.sdp with the same content (s=Garbage). For this page use garbage.sdp; do not mix it with the pose / YOLO / OCR / helmet sdp.
open -a VLC ~/garbage.sdp
Or VLC → File → Open File → select garbage.sdp.
The first time it may ask the local firewall to allow UDP 5000; click Allow.
VLC will stay black waiting for the stream; this is normal. Start the board side in the next step.
2. Start streaming on the board
ssh root@10.0.2.0
cd /root/rknn_yolov8_garbage_sort_cam
./run_garbage.sh 10.0.0.49
run_garbage.sh will: stop other Demos / rkipc → open /dev/video12 → infer and draw boxes → push H.264 to 10.0.0.49:5000.
When you see logs like the following, the stream is out:
v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
[DET] recyclable waste | box:( 412 , 201)( 890 , 480) | size: 478*279 | conf: 87.32%
[DET] kitchen waste | box:( 100 , 180)( 320 , 460) | size: 220*280 | conf: 76.10%
VLC should show a live picture: detection boxes, English class names, confidence, and FPS in the upper-left corner. Point bottles, cartons, kitchen waste, batteries, etc. at the lens. Box colors:
| Class (picture / log) | Chinese | Color |
|---|---|---|
recyclable waste | Recyclable waste | Green |
kitchen waste | Kitchen waste | Yellow |
hazardous waste | Hazardous waste | Red |
other waste | Other waste | Blue |
Manual equivalent command (when not using the script):
/etc/init.d/S99rkipc stop
cd /root/rknn_yolov8_garbage_sort_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_yolov8_garbage_sort_cam ./model/garbage_sort.rknn /dev/video12 10.0.0.49
The third parameter must be the 10.0.2.0.
3. After the demonstration, restore IPC
After stopping the Demo with Ctrl + C:
killall rknn_yolov8_garbage_sort_cam rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start
The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.
Board-side package contents
root@rv1126bp-buildroot:~/rknn_yolov8_garbage_sort_cam# ls
garbage.sdp lib model play.sdp rknn_yolov8_garbage_sort_cam run_garbage.sh
| File | Purpose |
|---|---|
rknn_yolov8_garbage_sort_cam | Main program |
run_garbage.sh | Stop rkipc and start; parameter is the computer IP |
garbage.sdp | Copy to the computer for VLC (recommended) |
play.sdp | Same content as garbage.sdp, s=Garbage |
model/garbage_sort.rknn | Detection model (about 3.8 MB, 4 classes) |
model/coco_4_labels_list.txt | English names of the four classes |
lib/librknnrt.so librga.so librockchip_mpp.so | NPU / RGA / encoding |
Startup parameters:
./rknn_yolov8_garbage_sort_cam <model_path> <camera_index> <target_ipv4>
<camera_index>: ISP main path; on this board it is/dev/video12(rkisp_mainpathinv4l2-ctl --list-devices)<target_ipv4>: LAN IP of the stream-receiving computer; port is hardcoded 5000
Thresholds are in postprocess.h: BOX_THRESH default 0.25, NMS_THRESH default 0.45.
Compile from source and deploy
You can skip this section if you only want to demonstrate. Follow the steps below when changing code or changing boards.
Machines
| Role | Example |
|---|---|
| Local machine | ~/SDK/talowe-rv1126b-aidemo (Mac only edits code and rsyncs) |
| Build server | ssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64) |
| Board | ssh root@10.0.2.0 |
Use the distribution cross-compiler; no need to build the full SDK host:
# build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake
Sync source
3rdparty/opencv is about 525 MB and is not used by this Demo; do not sync it.
ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam'
rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam/
Confirm model/garbage_sort.rknn is about 3.8 MB and already in the project model/.
Cross-compile
ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam/cpp
chmod +x build-linux.sh
./build-linux.sh
Output:
~/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam/cpp/install/rv1126b_linux_aarch64/rknn_yolov8_garbage_sort_cam/
# garbage.sdp lib model play.sdp rknn_yolov8_garbage_sort_cam run_garbage.sh
CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the board's existing librockchip_mpp (a copy of the so is included in the package).
Push to the board
The board's Buildroot usually has no rsync; create a tar on the build machine and extract it:
ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_yolov8_garbage_sort_cam/cpp/install/rv1126b_linux_aarch64 rknn_yolov8_garbage_sort_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'
You can also use this when USB is connected via adb:
adb push rknn_yolov8_garbage_sort_cam /root/
Then go back to steps 1 and 2 in Demonstration above.
Data path
IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ copy one NV12 frame for encoding (boxes / text)
→ RGA convert to RGB888
→ RKNN YOLOv8 garbage_sort (4 classes)
→ draw boxes / class names / FPS on NV12
→ MPP H.264 (each IDR carries SPS/PPS)
→ RTP PT=96 UDP → computer:5000
→ VLC opens garbage.sdp
The resolution and port are written in cpp/main.cc, cpp/run_garbage.sh, garbage.sdp / play.sdp; when changing 1080p / 5000, change all three places.
FAQ
| Symptom | Handling |
|---|---|
| VLC stays black | Close VLC first, reopen garbage.sdp, then start the board side; the third parameter must be the computer IP; allow UDP 5000 locally |
Camera open failed / VIDIOC_S_FMT failed | Other Demos / rkipc are still running. ./run_garbage.sh stops them |
| All black after opening the camera | ISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is inserted |
| Picture present but no boxes | The object must fully enter the frame; thresholds are in postprocess.h BOX_THRESH (default 0.25) |
Class names are all null / garbled | You must start from the package root directory; post-processing reads ./model/coco_4_labels_list.txt |
| Web page / RTSP gone after demonstration | rkipc not restored: killall rknn_yolov8_garbage_sort_cam then /etc/init.d/S99rkipc start |
Build missing aarch64-linux-gnu-gcc | On the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu |
The RKNN model can not be found | garbage_sort.rknn was not placed in the project model/ before compiling |
Camera node confirmation:
v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12
Differences from pose / YOLO / helmet
| YOLO11 | Helmet | This page garbage classification | |
|---|---|---|---|
| Project | talowe_yolo11_detection_cam | talowe_helmet_detection_cam | talowe_yolov8_garbage_sort_cam |
| Model | yolo11.rknn ~4.2 MB | helmet_detection.rknn ~11 MB | garbage_sort.rknn ~3.8 MB |
| Output | 80 COCO classes | helmet / no-helmet | Four garbage classes |
| Stream receive | VLC yolo11.sdp | VLC helmet.sdp | VLC garbage.sdp |
| With rkipc | Must stop | Must stop | Must stop |
Download source code and prebuilt packages
Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc
The cloud drive contains two archives:
| Package | Purpose |
|---|---|
talowe-rv1126b-aidemo | Source code, cross-compile yourself |
talowe-rv1126b-aidemo-app | Already compiled board-side programs (aarch64); copy to /root/ on the board and they run |
Notes for sending to customers / using yourself:
run_*.shpushes H.264 to10.0.0.49:5000by default. This is an example computer IP and must be changed to the stream-receiving computer's own LAN IP:./run_xxx.sh <computer IP>.- Firmware must be CAM126B-P Buildroot IPC. The Demo and
rkipcshare/dev/video12and cannot run simultaneously; the script stopsrkipcfirst. - Open VLC on the computer with the sdp corresponding to this page (this page is
garbage.sdp, port 5000); do not mix it with other Demos' sdp. - In
talowe-rv1126b-aidemo-app, please ignorerknn_yolov5_demo(leftover from the old image inference). For live camera detection userknn_yolov5_detection_cam. - Most Demos in the source have been changed to V4L2 + RGA + MPP; no need to include the entire
3rdparty/opencv(about 11 × 525 MB). Only PP-OCR cross-compilation still needs the in-project OpenCV static library.
Summary
- Open
garbage.sdpin VLC on the computer (wait for UDP 5000) - On the board
cd /root/rknn_yolov8_garbage_sort_cam && ./run_garbage.sh 10.0.0.49 - After screenshot / demonstration
killall rknn_yolov8_garbage_sort_cam,/etc/init.d/S99rkipc start