网站建设网站结构图,怎样用ps设计网站模板,中铁建设集团门户密码,江苏省 前置审批 网站深度学习图像视觉 RKNN Toolkit2 部署 RK3588S边缘端 过程全记录 认识RKNN Toolkit2 工程文件学习路线#xff1a; Anaconda Miniconda安装.condarc 文件配置镜像源自定义conda虚拟环境路径创建Conda虚拟环境 本地训练环境本地转换环境安装 RKNN-Toolkit2#xff1a;添加 lin… 深度学习图像视觉 RKNN Toolkit2 部署 RK3588S边缘端 过程全记录 认识RKNN Toolkit2 工程文件学习路线 Anaconda Miniconda安装.condarc 文件配置镜像源自定义conda虚拟环境路径创建Conda虚拟环境 本地训练环境本地转换环境安装 RKNN-Toolkit2添加 linaro交叉编译工具链安装 Cmakerknn_model_zoo 文件1. 使用 rknn_model_zoo 转换模型 model.rknn以转换yolov8n.onnx 为例测试 2. 编译模型 model.rknn 生成 install 文件以编译 yolov8 生成 install 为例全过程记录转换编译模型 rknpu2 文件1. 以Aarch64 Linux Demo 编译构建以编译 Yolo-v5 demo 生成 install 为例全过程记录转换编译模型 Aarch64设备部署rknn_model_zoo examples 文件使用 rknn_model_zoo 编译构建输出文件install推送到开发板终端设备执行操作全记录 rknpu2 examples 文件使用 rknpu2 编译构建输出文件install推送到开发板视频演示指南 终端设备执行操作全记录 aarch64设备部署 C API 推理 ------ AArch64是由ARM公司为其ARMv8-A 64位指令集架构推出的一种新的编程模型它是ARMv8-A架构的一个子集。这种编程模型主要是为了在64位模式下提高程序性能和安全性。与ARM32即ARMv7-A的指令集不同AArch64在寄存器和指令方面有许多改进和增强。 ------ 因此ARM64和AArch64都是指同一个东西只是名称不同来源也不同。如果你在不同的地方看到这两个术语不要感到困惑它们都是指代同一个处理器架构。 作者量子孤岛 来源知乎 认识RKNN Toolkit2 工程文件 RKNN-Toolkit, RKNN Toolkit Lite, 和 RKNPU2 都与Rockchip芯片上的神经网络推理相关但它们在用途和应用场景上有所不同 RKNN-Toolkit: 这是一个用于PC平台的开发套件主要用于在Rockchip芯片上进行NPU神经处理单元模型的转换、优化和性能评估。 它提供了全面的功能包括模型导入如ONNX、TensorFlow、Torchscript等模型转换为RKNN格式以及在目标硬件上进行推理的接口。 RKNN-Toolkit通常用于开发环境允许开发者在强大的PC上进行模型的预处理和调试工作。 RKNN Toolkit Lite: 这是轻量级版本的RKNN工具设计用于在Rockchip的嵌入式设备或开发板上直接运行。 它可能包含了基本的模型加载和推理功能适合在资源有限的环境中使用。 RKNN Toolkit Lite可能不包含所有PC版本的高级特性例如详细的性能分析或模型转换工具但它更专注于在实际硬件上的高效运行。 RKNPU2: RKNPU2似乎是指Rockchip NPU的第二代软件栈或SDK它提供了与NPU硬件交互的底层接口。 这个SDK通常包含驱动程序、库文件和API使得开发者能够直接控制NPU进行神经网络计算。 RKNPU2可能被用于实现更底层的性能优化或者在没有完整RKNN-Toolkit的情况下进行定制化开发。 RKNPU2 SDK 提供了C语言编程接口专门设计用于带有Rockchip神经处理单元NPU的芯片平台。这个接口允许开发者直接在目标设备上编写C代码来集成和执行已经通过RKNN-Toolkit2转换和优化的RKNN模型。以下是一些关键点 模型转换 使用RKNN-Toolkit2开发者可以将常见的深度学习框架如TensorFlow, PyTorch等的模型转换为针对RockchipNPU优化的RKNN格式。 C语言接口 RKNPU2 SDK 提供的C API使得开发者能够在应用程序中加载、初始化和运行这些RKNN模型。这些API通常包括模型加载函数、推理接口、数据预处理和后处理函数等。 设备交互 通过这些接口开发者可以直接控制NPU进行高效的硬件加速计算充分利用NPU的并行处理能力。API会处理与硬件的低级别通信包括内存管理和指令调度。 资源管理 开发者需要管理模型的内存分配和释放以及在运行时管理输入和输出数据缓冲区。 性能优化 RKNPU2可能还包括一些工具和指导帮助开发者进行性能调优比如批量处理、多线程支持等。 部署和测试 一旦模型在目标设备上编译和链接开发者可以通过编写C程序来实现模型的部署并进行实际的推理任务。 错误处理和调试 C语言接口也会提供错误处理机制以便在遇到问题时能够捕获和诊断错误。 通过这种方式RKNPU2 SDK 和 C语言接口为开发者提供了一种灵活和高效的方法将AI模型集成到嵌入式系统中尤其适用于需要实时推理和低功耗要求的应用场景。 总结来说RKNN-Toolkit是全面的开发和调试工具适合在PC上进行模型准备RKNN Toolkit Lite是简化版适用于嵌入式设备上的推理而RKNPU2是NPU的软件开发包提供直接访问硬件的能力。根据开发需求和目标平台的不同开发者会选择适合的工具。
学习路线 Anaconda Miniconda安装
清华大学开源软件镜像站 Miniconda下载 Anaconda 镜像使用帮助 linux安装软件安装过程中根据提示输入enter或yes
bash Miniconda3-py312_24.3.0-0-Linux-x86_64.sh安装完后通过conda命令进行使用。
.condarc 文件
Windows 用户无法直接创建名为 .condarc 的文件可先执行 conda config --set show_channel_urls yes 配置镜像源
通过修改文件添加推荐 直接修改.condarc文件是最方便的。 找到系统用户下的 .condarc 的文件记事本打开并添加镜像源。
channels:- defaults
show_channel_urls: true
default_channels:- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmsys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudbioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudmenpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudpytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloudsimpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/clouddeepmodeling: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/自定义conda虚拟环境路径
找到系统用户下的 .condarc 的文件记事本打开并添加路径换成自己要保存的位置建议放在非C盘中。
##windows
envs_dirs:- E://Miniconda3//envs ##linux
envs_dirs:- /home/wlj/.conda/envs/创建Conda虚拟环境
# 列举所有环境
conda env list
# Python创建虚拟训练环境
conda create -n RKNN_yolov8_py3.10 python3.10
conda activate RKNN_yolov8_py3.10 # Python创建虚拟转换环境
conda create -n RKNN_Toolkit2_py3.10 python3.10
conda activate RKNN_Toolkit2_py3.10 # 退出虚拟环境
conda deactivate本地训练环境 本地转换环境 安装 RKNN-Toolkit2
注意事项 使用大于或等于 1.4.0 的 rknn-toolkit2 版本。 使用自己训练的模型时请对齐anchor等后处理参数否则会导致后处理分析误差。 demo需要librga.so的支持,编译使用请参考 https://github.com/airockchip/librga 。
# 从官方RKNN-Toolkit2仓库拉取最新版本(toolkit2版本是/v2.0.0-beta0)
git clone https://github.com/airockchip/rknn-toolkit2/tree/v2.0.0-beta0# 配置pip源
pip3 config set global.index-url https://mirror.baidu.com/pypi/simple# 安装依赖库根据rknn-toolkit2/packages/requirements_cp310-2.0.0b0.txt
pip3 install -r requirements_cp310-2.0.0b0.txt# 安装rknn_toolkit2 | rknn-toolkit2/packages/rknn_toolkit2-2.0.0b09bab5682-cp310-cp310-linux_x86_64.whl
# 根据系统的python版本选择不同的whl文件安装
pip3 install rknn_toolkit2-2.0.0b09bab5682-cp310-cp310-linux_x86_64.whl检测是否安装成功
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:~$ python3
Python 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] on linux
Type help, copyright, credits or license for more information.from rknn.api import RKNN添加 linaro交叉编译工具链
环境 ubnutu20.04.06 使用linaro交叉编译工具链下载 博客
解压交叉编译工具链 解压gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu
tar -xzvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz
tar -xvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tarsudo gedit ~/.bashrc 或 sudo vim ~/.bashrc
export GCC_COMPILER/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu
source ~/.bashrc安装 Cmake
第一种指令安装
sudo apt install cmake第二种解压tar 自定义 Cmake下载地址
tar -xzvf cmake-3.29.0-linux-x86_64.tar.gz#打开个人path配置
gedit ~/.bashrc 或 sudo vim ~/.bashrc
#在末尾添加如下的内容
export PATH/home/ubuntu/cmake-3.29.0-linux-x86_64/bin:$PATH
#接着在终端source一下.bashrc文件让path立即生效
source ~/.bashrc
#安装完毕测试版本
cmake --versionrknn_model_zoo 文件
rknn_model_zoo
├── 3rdparty # 第三方库
├── datasets # 数据集
├── examples # 示例代码
├── utils # 常用方法如文件操作画图等
├── build-android.sh # 用于目标为 Android 系统开发板的编译脚本
├── build-linux.sh # 用于目标为 Linux 系统开发板的编译脚本
└── ...1. 使用 rknn_model_zoo 转换模型 model.rknn
# 从官方rknn_model_zoo仓库拉取最新版本
git clone https://github.com/airockchip/rknn_model_zoo/tree/v2.0.0Convert to RKNN
以转换yolov8n.onnx 为例
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ tree
.
├── README.md
├── cpp
│ ├── CMakeLists.txt
│ ├── main.cc
│ ├── postprocess.cc
│ ├── postprocess.h
│ ├── rknpu1
│ │ └── yolov8.cc
│ ├── rknpu2
│ │ ├── yolov8.cc
│ │ └── yolov8_rv1106_1103.cc
│ └── yolov8.h
├── model
│ ├── bus.jpg
│ ├── coco_80_labels_list.txt
│ ├── dataset.txt
│ ├── download_model.sh
│ └── yolov8n.onnx
├── model_comparison
│ ├── yolov8_graph_comparison.jpg
│ └── yolov8_output_comparison.jpg
├── python
│ ├── convert.py
│ └── yolov8.py
└── result.png首先导入 GCC_COMPILER 例如 export GCC_COMPILER/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu如系统添加无需此步 然后执行
cd python
# 运行 convert.py 脚本将原始的 ONNX 模型转成 RKNN 模型
# 用法: python convert.py model_path [rk3566|rk3588|rk3562] [i8/fp] [output_path]
# output model will be saved as ../model/yolov8.rknn
python convert.py ../model/yolov8n.onnx rk3588 i8 ../model/yolov8n.rknn测试
# 如果想先在计算机端运行原始的 onnx 模型可以参考以下命令
# 用法: python yolov5.py --model_path {onnx_model} --img_show
python yolov8.py --model_path ../model/yolov8n.onnx --img_show2. 编译模型 model.rknn 生成 install 文件
For Linux develop board:
首先导入 GCC_COMPILER 例如 export GCC_COMPILER/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu如系统添加无需此步 然后执行
./build-linux.sh -t target -a arch -d build_demo_name [-b build_type] [-m]-t : target (rk356x/rk3588/rk3576/rv1106/rk1808/rv1126)-a : arch (aarch64/armhf)-d : demo name-b : build_type(Debug/Release)-m : enable address sanitizer, build_type need set to Debug
Note: rk356x represents rk3562/rk3566/rk3568, rv1106 represents rv1103/rv1106, rv1126 represents rv1109/rv1126# Here is an example for compiling yolov5 demo for 64-bit Linux RK3566.
./build-linux.sh -t rk356x -a aarch64 -d yolov5For Android development board:
# For Android develop boards, its require to set path for Android NDK compilation tool path according to the user environment
export ANDROID_NDK_PATH~/opts/ndk/android-ndk-r18b
./build-android.sh -t target -a arch -d build_demo_name [-b build_type] [-m]-t : target (rk356x/rk3588/rk3576)-a : arch (arm64-v8a/armeabi-v7a)-d : demo name-b : build_type (Debug/Release)-m : enable address sanitizer, build_type need set to Debug# Here is an example for compiling yolov5 demo for 64-bit Android RK3566.
./build-android.sh -t rk356x -a arm64-v8a -d yolov5以编译 yolov8 生成 install 为例
位置在rknn_model_zoo/examples/yolov8
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/rknn_model_zoo/examples/yolov8$ tree
.
├── README.md
├── cpp # C/C 版本的示例代码
│ ├── CMakeLists.txt
│ ├── main.cc
│ ├── postprocess.cc
│ ├── postprocess.h
│ ├── rknpu1
│ │ └── yolov8.cc
│ ├── rknpu2
│ │ ├── yolov8.cc
│ │ └── yolov8_rv1106_1103.cc
│ └── yolov8.h
├── model # 模型、测试图片等文件
│ ├── bus.jpg
│ ├── coco_80_labels_list.txt
│ ├── dataset.txt
│ ├── download_model.sh
│ ├── yolov8.rknn
│ └── yolov8n.onnx
├── model_comparison
│ ├── yolov8_graph_comparison.jpg
│ └── yolov8_output_comparison.jpg
├── python # 模型转换脚本和 Python 版本的示例代码
│ ├── convert.py
│ └── yolov8.py
└── result.pngcd /rknn_model_zoo
./build-linux.sh -t rk3588 -a aarch64 -d yolov8rknn_model_zoo目录下生成 install
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/rknn_model_zoo/install$ tree
.
└── rk3588_linux_aarch64└── rknn_yolov8_demo├── lib # 依赖库│ ├── librga.so│ └── librknnrt.so├── model # 存放模型、测试图片等文件│ ├── bus.jpg│ ├── coco_80_labels_list.txt│ └── yolov8.rknn└── rknn_yolov8_demo # 可执行文件全过程记录转换编译模型
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ ls
README.md cpp model model_comparison python result.png
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd python/
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/python$ python convert.py ../model/yolov8n.onnx rk3588
I rknn-toolkit2 version: 2.0.0b09bab5682
-- Config model
done
-- Loading model
I It is recommended onnx opset 19, but your onnx model opset is 12!
I Model converted from pytorch, opset_version should be set 19 in torch.onnx.export for successful convert!
I Loading : 100%|██████████████████████████████████████████████| 136/136 [00:0000:00, 35716.32it/s]
done
-- Building model
W build: found outlier value, this may affect quantization accuracyconst name abs_mean abs_std outlier valuemodel.0.conv.weight 2.44 2.47 -17.494model.22.cv3.2.1.conv.weight 0.09 0.14 -10.215model.22.cv3.1.1.conv.weight 0.12 0.19 13.361, 13.317model.22.cv3.0.1.conv.weight 0.18 0.20 -11.216
I GraphPreparing : 100%|████████████████████████████████████████| 161/161 [00:0000:00, 5348.48it/s]
I Quantizating : 100%|████████████████████████████████████████████| 161/161 [00:0500:00, 27.53it/s]
W build: The default input dtype of images is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 318 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of onnx::ReduceSum_326 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 331 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 338 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of onnx::ReduceSum_346 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 350 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 357 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of onnx::ReduceSum_365 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of 369 is changed from float32 to int8 in rknn model for performance!Please take care of this change when deploy rknn model with Runtime API!
I rknn building ...
I rknn buiding done.
done
-- Export rknn model
done
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/python$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd model
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/model$ ls
bus.jpg coco_80_labels_list.txt dataset.txt download_model.sh yolov8.rknn yolov8n.onnx
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8/model$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples/yolov8$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/examples$ cd ..
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo$ ./build-linux.sh -t rk3588 -a aarch64 -d yolov8
./build-linux.sh -t rk3588 -a aarch64 -d yolov8
/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnuBUILD_DEMO_NAMEyolov8
BUILD_DEMO_PATHexamples/yolov8/cpp
TARGET_SOCrk3588
TARGET_ARCHaarch64
BUILD_TYPERelease
ENABLE_ASANOFF
INSTALL_DIR/mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo
BUILD_DIR/mnt/d/Desktop/RK3588/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3588_linux_aarch64_Release
CC/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
CXX/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g-- !!!!!!!!!!!CMAKE_SYSTEM_NAME: Linux
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/d/Desktop/RK3588/rknn_model_zoo/build/build_rknn_yolov8_demo_rk3588_linux_aarch64_Release
[ 20%] Built target fileutils
[ 40%] Built target imagedrawing
[ 60%] Built target imageutils
[100%] Built target rknn_yolov8_demo
[ 20%] Built target fileutils
[ 40%] Built target imageutils
[ 60%] Built target imagedrawing
[100%] Built target rknn_yolov8_demo
Install the project...
-- Install configuration: Release
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo
-- Set runtime path of /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo to $ORIGIN/../lib
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/bus.jpg
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/coco_80_labels_list.txt
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/model/yolov8.rknn
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/lib/librknnrt.so
-- Installing: /mnt/d/Desktop/RK3588/rknn_model_zoo/install/rk3588_linux_aarch64/rknn_yolov8_demo/lib/librga.so
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo$rknpu2 文件
注意事项 使用大于或等于 1.4.0 的 rknn-toolkit2 版本。 使用自己训练的模型时请对齐anchor等后处理参数否则会导致后处理分析误差。 官网和rk预训练模型均检测80种目标。如果训练自己的模型则需要更改 include/postprocess.h 中的OBJ_CLASS_NUM和NMS_THRESHBOX_THRESH后处理参数。 demo需要librga.so的支持,编译使用请参考 https://github.com/airockchip/librga 。 由于硬件限制该demo的模型默认把 yolov5 模型的后处理部分移至cpu实现。本demo附带的模型均使用relu为激活函数相比silu激活函数精度略微下降性能大幅上升。
1. 以Aarch64 Linux Demo 编译构建
首先导入 GCC_COMPILER 例如 export GCC_COMPILER/home/cat/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu如系统添加无需此步 然后执行
#进入rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ls
CMakeLists.txt README.md README_CN.md build-android.sh build-linux.sh convert_rknn_demo include model src utils以编译 Yolo-v5 demo 生成 install 为例
catlubancat:~/rknpu2/examples/rknn_yolov5_demo$ tree
.
├── CMakeLists.txt
├── README.md
├── README_CN.md
├── build-android.sh
├── build-linux.sh
├── convert_rknn_demo
│ └── yolov5
│ ├── README.md
│ ├── README_CN.md
│ ├── bus.jpg
│ ├── dataset.txt
│ ├── onnx2rknn.py
│ └── onnx_models
│ ├── yolov5s_for_apk_demo.onnx
│ └── yolov5s_relu.onnx
├── include
│ ├── drm_func.h
│ ├── postprocess.h
│ ├── preprocess.h
│ └── rga_func.h
├── model
│ ├── RK3562
│ │ └── yolov5s-640-640.rknn
│ ├── RK3566_RK3568
│ │ └── yolov5s-640-640.rknn
│ ├── RK3576
│ │ └── yolov5s-640-640.rknn
│ ├── RK3588
│ │ └── yolov5s-640-640.rknn
│ ├── bus.jpg
│ └── coco_80_labels_list.txt
├── src
│ ├── main.cc
│ ├── main_video.cc
│ ├── postprocess.cc
│ └── preprocess.cc
└── utils├── drawing.cpp├── drawing.h├── mpp_decoder.cpp├── mpp_decoder.h├── mpp_encoder.cpp└── mpp_encoder.h11 directories, 32 files# such as:
# ./build-linux.sh -t target -a arch -b build_type]
./build-linux.sh -t rk3588 -a aarch64 -b Release在rknn_yolov5_demo目录下生成 install
全过程记录转换编译模型
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ls
CMakeLists.txt README.md README_CN.md build-android.sh build-linux.sh convert_rknn_demo include model src utils
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ll
total 20
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 ./
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 ../
-rwxrwxrwx 1 ubuntu20 ubuntu20 3948 Mar 25 11:53 CMakeLists.txt*
-rwxrwxrwx 1 ubuntu20 ubuntu20 3561 Mar 25 11:53 README.md*
-rwxrwxrwx 1 ubuntu20 ubuntu20 3496 Mar 25 11:53 README_CN.md*
-rwxrwxrwx 1 ubuntu20 ubuntu20 2632 Mar 25 11:53 build-android.sh*
-rwxrwxrwx 1 ubuntu20 ubuntu20 2722 Mar 25 11:53 build-linux.sh*
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 convert_rknn_demo/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 include/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 model/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 src/
drwxrwxrwx 1 ubuntu20 ubuntu20 4096 Mar 25 11:53 utils/
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$ ./build-linux.sh -t rk3588 -a aarch64 -b Release
./build-linux.sh -t rk3588 -a aarch64 -b Release
/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnuTARGET_SOCRK3588
TARGET_ARCHaarch64
BUILD_TYPERelease
BUILD_DIR/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
CC/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
CXX/home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g
-- Check for working CXX compiler: /home/ubuntu20/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/3rdparty/opencv/opencv-linux-aarch64 (found version 3.4.5)
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
Scanning dependencies of target rknn_yolov5_demo
Scanning dependencies of target rknn_yolov5_video_demo
[ 10%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/main_video.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/postprocess.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_decoder.cpp.o
[ 40%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_encoder.cpp.o
[ 50%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/drawing.cpp.o
[ 60%] Linking CXX executable rknn_yolov5_video_demo
[ 60%] Built target rknn_yolov5_video_demo
[ 70%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/main.cc.o
[ 90%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/postprocess.cc.o
[ 90%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/preprocess.cc.o
[100%] Linking CXX executable rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 60%] Built target rknn_yolov5_video_demo
[100%] Built target rknn_yolov5_demo
Install the project...
-- Install configuration: Release
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_demo
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librknnrt.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librga.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3588
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/RK3588/yolov5s-640-640.rknn
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/bus.jpg
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./model/coco_80_labels_list.txt
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/./rknn_yolov5_video_demo
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/librockchip_mpp.so
-- Installing: /mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo/install/rknn_yolov5_demo_Linux/lib/libmk_api.so
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn-toolkit2-2.0.0-beta0/rknpu2/examples/rknn_yolov5_demo$Aarch64设备部署
rknn_model_zoo examples 文件
使用 rknn_model_zoo 编译构建输出文件install推送到开发板
在 rknn_model_zoo 生成 install 文件夹
(RKNN_Toolkit2_py3.10) ubuntu20DESKTOP-EJ39PBE:/mnt/d/Desktop/RK3588/rknn_model_zoo/install$ tree
.
└── rk3588_linux_aarch64└── rknn_yolov8_demo├── lib│ ├── librga.so│ └── librknnrt.so├── model│ ├── bus.jpg│ ├── coco_80_labels_list.txt│ └── yolov8.rknn└── rknn_yolov8_demo拷贝 install 到 aarch64设备再
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ chmod 777 *
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ export LD_LIBRARY_PATH./lib
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ./rknn_yolov8_demo model/yolov8.rknn model/bus.jpg运行一张的速度太慢了批量去运行写一个batch_process.sh文件
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ vim batch_process.shexport LD_LIBRARY_PATH./lib
#!/bin/bash
# 指定图片文件夹路径
IMAGE_DIR./image_file
# 遍历图片文件夹中的所有图片文件
for image_file in $IMAGE_DIR/*.jpg; doif [ -f $image_file ]; thenecho Processing $image_file./rknn_yolov8_demo ./model/yolov8.rknn $image_filefi
done终端设备执行操作全记录
catlubancat:~$ ls
Desktop Downloads Pictures Python-3.10.14 Videos librga rknn_mobilenet_demo_Linux rknn_model_zoo
Documents Music Public Templates cmake-3.29.0-linux-aarch64 rk3588_linux_aarch64 rknn_model
catlubancat:~$ cd rk3588_linux_aarch64/
catlubancat:~/rk3588_linux_aarch64$ ls
rknn_yolov8_demo
catlubancat:~/rk3588_linux_aarch64$ cd rknn_yolov8_demo/
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ls
lib model rknn_yolov8_demo
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ll
total 984
drwxrwxr-x 4 cat cat 4096 May 13 15:06 ./
drwxrwxrwx 3 cat cat 4096 May 13 15:06 ../
drwxrwxr-x 2 cat cat 4096 May 13 15:06 lib/
drwxrwxr-x 2 cat cat 4096 May 13 15:06 model/
-rw-rw-r-- 1 cat cat 991216 May 13 15:06 rknn_yolov8_demo
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ chmod 777 *
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ export LD_LIBRARY_PATH./lib
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ./rknn_yolov8_demo model/yolov8.rknn model/bus.jpg
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 9
input tensors:index0, nameimages, n_dims4, dims[1, 640, 640, 3], n_elems1228800, size1228800, fmtNHWC, typeINT8, qnt_typeAFFINE, zp-128, scale0.003922
output tensors:index0, name318, n_dims4, dims[1, 64, 80, 80], n_elems409600, size409600, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-58, scale0.117659index1, nameonnx::ReduceSum_326, n_dims4, dims[1, 80, 80, 80], n_elems512000, size512000, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003104index2, name331, n_dims4, dims[1, 1, 80, 80], n_elems6400, size6400, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003173index3, name338, n_dims4, dims[1, 64, 40, 40], n_elems102400, size102400, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-45, scale0.093747index4, nameonnx::ReduceSum_346, n_dims4, dims[1, 80, 40, 40], n_elems128000, size128000, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003594index5, name350, n_dims4, dims[1, 1, 40, 40], n_elems1600, size1600, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003627index6, name357, n_dims4, dims[1, 64, 20, 20], n_elems25600, size25600, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-34, scale0.083036index7, nameonnx::ReduceSum_365, n_dims4, dims[1, 80, 20, 20], n_elems32000, size32000, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003874index8, name369, n_dims4, dims[1, 1, 20, 20], n_elems400, size400, fmtNCHW, typeINT8, qnt_typeAFFINE, zp-128, scale0.003922
model is NHWC input fmt
model input height640, width640, channel3
origin size640x640 crop size640x640
input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale1.000000 dst_box(0 0 639 639) allow_slight_change1 _left_offset0 _top_offset0 padding_w0 padding_h0
src width640 height640 fmt0x1 virAddr0x0x1d996ea0 fd0
dst width640 height640 fmt0x1 virAddr0x0x1dac2eb0 fd0
src_box(0 0 639 639)
dst_box(0 0 639 639)
color0x72
rga_api version 1.10.1_[0]
rknn_run
person (211 241 282 506) 0.864
bus (96 136 549 449) 0.864
person (109 235 225 535) 0.860
person (477 226 560 522) 0.848
person (79 327 116 513) 0.306
write_image path: out.png width640 height640 channel3 data0x1d996ea0
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ ls
lib model out.png rknn_yolov8_demo
catlubancat:~/rk3588_linux_aarch64/rknn_yolov8_demo$ rknpu2 examples 文件
使用 rknpu2 编译构建输出文件install推送到开发板
在rknpu2 / examples / rknn_yolov5_demo目录下生成 install 推送到aarch64设备开发板 install / rknn_yolov5_demo_Linux
export LD_LIBRARY_PATH./lib
#./rknn_yolov5_demo model/TARGET_PLATFORM/yolov5s-640-640.rknn model/bus.jpg
chmod 777 *
./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg注意如果在 lib 文件夹中找不到 librga.so请尝试搜索 librga.so 的位置并将其添加到LD_LIBRARY_PATH。使用以下命令添加到LD_LIBRARY_PATH。
export LD_LIBRARY_PATH./lib:LOCATION_LIBRGA.SOGITHUB的airockchip/librga地址
效果
视频演示指南
添加 test.mp4 视频测试
H264 H264型
通过 ffmpeg 转换为 h264
#ffmpeg -i xxx.mp4 -vcodec h264 xxx.h264
ffmpeg -i model/test.mp4 -vcodec h264 model/test.h264#./rknn_yolov5_video_demo model/TARGET_PLATFORM/yolov5s-640-640.rknn xxx.h264 264
./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/test.h264 264通过 ffmpeg 转换为 h265
#ffmpeg -i xxx.mp4 -vcodec hevc xxx.hevc
ffmpeg -i model/test.mp4 -vcodec hevc model/test.hevcH265
#./rknn_yolov5_video_demo model/TARGET_PLATFORM/yolov5s-640-640.rknn xxx.hevc 265./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/test.hevc 265RTSP
#./rknn_yolov5_video_demo model/TARGET_PLATFORM/yolov5s-640-640.rknn RTSP_URL 265./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn RTSP_URL 265效果
终端设备执行操作全记录
catlubancat:~$ ls
Desktop Music Python-3.10.14 cmake-3.29.0-linux-aarch64 rknn_yolov5_demo_Linux
Documents Pictures Templates librga
Downloads Public Videos rk3588_linux_aarch64
catlubancat:~$ cd rknn_yolov5_demo_Linux/
catlubancat:~/rknn_yolov5_demo_Linux$ ls
lib model out.h264 out.jpg rknn_yolov5_demo rknn_yolov5_video_demo
catlubancat:~/rknn_yolov5_demo_Linux$ rm -rf out.h264
catlubancat:~/rknn_yolov5_demo_Linux$ ls
lib model out.jpg rknn_yolov5_demo rknn_yolov5_video_demo
catlubancat:~/rknn_yolov5_demo_Linux$ ./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/test.hevc 265
Loading mode...
省略....time_gap-10found last frame reset decoder
waiting finish
catlubancat:~/rknn_yolov5_demo_Linux$ ls
lib model out.h264 out.jpg rknn_yolov5_demo rknn_yolov5_video_demoaarch64设备部署 C API 推理
