Application Test - Human Pose Detection (YOLOv8-Pose)

Run Talowe
talowe_yolov8_pose_camon CAM126B-P: real-time human keypoints from the camera, overlaid with the skeleton, then pushed to the computer via H.264 RTP; openplay.sdpwith VLC to view the image.Source code: https://github.com/hbt021211-coder/talowe-rv1126b-aidemo
Inference: RKNN | Capture: V4L2
/dev/video12NV12 1920x1080| Encoding: MPP | Stream receive: UDP5000
The original Talowe project uses OpenCV + GStreamer. The factory Buildroot IPC does not have these two libraries.
The current Demo has been changed to V4L2 + RGA + MPP; flashing the IPC firmware is enough to demonstrate, no need to rebuild the rootfs.
The process on this page is written for the rkipc, /dev/video12, S99rkipc).
- For flashing, see the CAM126B-P Buildroot IPC (
Rockchip-RV1126B-Buildroot-IPC) in Upgrade Firmware Re-flashing is not required : if the board already runs this IPC firmware, you can demonstrate directly- If it is currently Debian / another image, please first flash it to Buildroot IPC according to that page, then follow the steps below
For IPC streaming, account and password, see Firmware Usage - IPC Streaming.
Demonstration (See the Effect Directly)
The environment uses the three machines below. If the computer IP is not 10.0.0.49, change the script parameter to your IP.
| Role | Address | Description |
|---|---|---|
| Receiving computer (Mac) | 10.0.0.49 | Install VLC, open play.sdp |
| CAM126B-P | ssh root@10.0.2.0 | Default password see Firmware Usage |
| Program directory | /root/rknn_yolov8_pose_cam | Already deployed installation package |
1. Open VLC on the Computer First
Copy play.sdp to the computer, for example ~/play.sdp. The content is fixed as follows (port 5000; do not change it to the board IP):
v=0
o=- 0 0 IN IP4 0.0.0.0
s=Pose
c=IN IP4 0.0.0.0
t=0 0
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000
open -a VLC ~/play.sdp
Or VLC → File → Open File → select play.sdp.
The first time, the local firewall may ask to allow UDP 5000; click Allow.
VLC will stay on a black screen waiting for the stream, which is normal; start the board side in the next step.
2. Start Streaming on the Board
ssh root@10.0.2.0
cd /root/rknn_yolov8_pose_cam
./run_pose.sh 10.0.0.49
run_pose.sh will: stop rkipc (otherwise it occupies the camera) → open /dev/video12 → run inference and draw the skeleton → push H.264 to 10.0.0.49:5000.
A log similar to the following means the stream is out:
v4l2 opened /dev/video12 1920x1080 stride=1920 nbufs=4
mpp rtp -> 10.0.0.49:5000 1920x1080
Pipeline initialized. Starting inference, RTP 10.0.0.49:5000 ...
rknn_run time=32.xxms, FPS = 30.xx
VLC should show a real-time image: human boxes, skeleton, keypoints, and FPS in the upper-left corner. Inference is about 20–33 FPS.
Equivalent manual command (when not using the script):
/etc/init.d/S99rkipc stop
cd /root/rknn_yolov8_pose_cam
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH
./rknn_yolov8_pose_demo ./model/yolov8_pose.rknn /dev/video12 10.0.0.49
The third parameter must be the 10.0.2.0.
3. After the Demonstration, Restore IPC
After stopping the Demo with Ctrl + C:
killall rknn_yolov8_pose_demo rkaiq_3A_server 2>/dev/null
/etc/init.d/S99rkipc start
The web page / RTSP (rtsp://10.0.2.0:554/live/0) will recover.
The Demo and rkipc cannot occupy the camera at the same time.
Board-side Installation Package Contents
root@rv1126bp-buildroot:~/rknn_yolov8_pose_cam# ls
lib model play.sdp rknn_yolov8_pose_demo run_pose.sh
| File | Purpose |
|---|---|
rknn_yolov8_pose_demo | Main program |
run_pose.sh | Stop rkipc and start; the parameter is the computer IP |
play.sdp | Copy to the computer for VLC |
model/yolov8_pose.rknn | Pose model (about 5 MB, 1 class person, 17 keypoints) |
lib/librknnrt.so librga.so librockchip_mpp.so | NPU / RGA / encoding |
Startup parameters:
./rknn_yolov8_pose_demo <model_path> <camera_index> <target_ipv4>
<camera_index>: ISP main path; on this board it is/dev/video12(rkisp_mainpathinv4l2-ctl --list-devices)<target_ipv4>: LAN IP of the receiving computer; the port is hardcoded to 5000
Compile from Source and Deploy
You can skip this section if you only want to demonstrate. Follow the steps below when modifying code or changing the board.
Machines
| Role | Example |
|---|---|
| Local machine | ~/SDK/talowe-rv1126b-aidemo (Mac only modifies code, rsync) |
| Build server | ssh roy@10.0.0.18 (x86_64 Ubuntu, cross-compile aarch64) |
| Board | ssh root@10.0.2.0 |
The distribution cross-compiler is sufficient; you do not need to build the full SDK host:
# Build server (root)
apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake
Sync Source
3rdparty/opencv is about 525 MB and is no longer used by this Demo, so do not sync it.
ssh roy@10.0.0.18 'mkdir -p ~/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam'
rsync -av --progress \
--exclude '.git' \
--exclude 'cpp/3rdparty/opencv' \
--exclude 'cpp/build' \
--exclude 'cpp/install' \
~/SDK/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam/ \
roy@10.0.0.18:~/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam/
Confirm that model/yolov8_pose.rknn of about 5 MB is already in the project's model/.
Cross-compile
ssh roy@10.0.0.18
cd ~/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam/cpp
chmod +x build-linux.sh
./build-linux.sh
Output:
~/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam/cpp/install/rv1126b_linux_aarch64/rknn_yolov8_pose_cam/
# lib model play.sdp rknn_yolov8_pose_demo run_pose.sh
CMakeLists.txt / build-linux.sh use aarch64-linux-gnu-gcc and link against the librockchip_mpp already on the board (the installation package includes a copy of the so).
Push to the Board
The board's Buildroot generally has no rsync; create a tar from the build machine and extract it:
ssh roy@10.0.0.18 \
'tar czf - -C ~/talowe-rv1126b-aidemo/talowe_yolov8_pose_cam/cpp/install/rv1126b_linux_aarch64 rknn_yolov8_pose_cam' \
| ssh root@10.0.2.0 'tar xzf - -C /root'
If USB adb is connected, you can also use:
adb push rknn_yolov8_pose_cam /root/
Then return to steps 1 and 2 of Demonstration above.
Data Path
IMX / Sensor
→ rkisp_mainpath /dev/video12 (NV12 1920x1080)
→ RGA convert to RGB888
→ RKNN YOLOv8-Pose (640x640)
→ draw boxes / 17-point skeleton / FPS
→ RGA convert back to NV12
→ MPP H.264
→ RTP PT=96 UDP → computer:5000
→ VLC opens play.sdp
The resolution and port are written in cpp/main.cc, cpp/run_pose.sh, and play.sdp; change all three together when changing 1080p / 5000.
FAQ
| Symptom | Handling |
|---|---|
| VLC stays black | Open VLC first, then start the board; the third parameter must be the computer IP; allow UDP 5000 locally; computer and board on the same network segment |
Camera open failed / VIDIOC_S_FMT failed | rkipc is still running. ./run_pose.sh will stop it, or manually /etc/init.d/S99rkipc stop |
| Image is completely black after opening the camera | ISP 3A is not up. The script starts rkaiq_3A_server /etc/iqfiles; confirm the module is plugged in |
| No person, no skeleton | The person must be fully in frame; the threshold is BOX_THRESH in postprocess.h (default 0.5) |
| Web/RTSP gone after the demonstration | rkipc not restored: after killall rknn_yolov8_pose_demo run /etc/init.d/S99rkipc start |
Compilation missing aarch64-linux-gnu-gcc | On the build server apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu |
The RKNN model can not be found | yolov8_pose.rknn was not placed in the project's model/ before compiling |
Camera node confirmation:
v4l2-ctl --list-devices
rkisp_mainpath (platform:rkisp-vir0):
/dev/video12
Differences from the Official Yolov5 Image Demo
| Official Yolov5 | This Page Pose Cam | General Object Detection | |
|---|---|---|---|
| Input | Local bus.jpg | Camera real-time | Camera real-time |
| Output | On-board out.png | Computer VLC views RTP | Computer VLC views RTP |
| Stream receive | None | play.sdp | yolov5.sdp |
| With rkipc | Does not compete for the camera | Must stop rkipc | Must stop rkipc |
Source Code and Compiled Package Download
Link: https://pan.baidu.com/s/15rNLzfb12Af8Sh_DX-KVBw?pwd=fanc
Extraction code: fanc
The network disk contains two archives:
| Package | Purpose |
|---|---|
talowe-rv1126b-aidemo | Source code, cross-compile yourself |
talowe-rv1126b-aidemo-app | Already compiled board-side program (aarch64); copy to the board's /root/ to run |
Note when sending to customers / using it yourself:
run_*.shpushes H.264 to10.0.0.49:5000by default. This is an example computer IP; you must change it to the receiving computer's own LAN IP:./run_xxx.sh <computer IP>.- The firmware must be CAM126B-P Buildroot IPC. The Demo and
rkipcshare/dev/video12and cannot run at the same time; the script stopsrkipcfirst. - On the computer, use the sdp corresponding to this page to open VLC (this page is
play.sdp, port 5000); do not mix it with the sdp of other Demos. - In
talowe-rv1126b-aidemo-app, please ignorerknn_yolov5_demo(leftover old image inference). Userknn_yolov5_detection_camfor real-time camera detection. - Most Demos in the source code have been changed to V4L2 + RGA + MPP; there is no need to include all
3rdparty/opencv(about 11 × 525 MB). Only PP-OCR cross-compilation still needs the OpenCV static libraries in the project.
Summary
- Open
play.sdpwith VLC on the computer (waiting for UDP 5000) - On the board
cd /root/rknn_yolov8_pose_cam && ./run_pose.sh 10.0.0.49 - After screenshot / demonstration,
killall rknn_yolov8_pose_demo,/etc/init.d/S99rkipc start