OpenCV_CUDA Environment Setup

1.Linux

It is more recommended to deploy on Linux; for some deep learning tasks, the running speed on Linux is significantlyfar higher thanWindows.

If you don't have a free hard drive to install Linux, you can consider WSL2 (Windows Subsystem for Linux 2, which runs on Windows). Although there is a slight performance loss, it is still much faster than Windows.

WSL2 Installation TutorialVinci Robotics Team Linux Introductory Tutorial

Bare-metal Linux**＞WSL2＞＞**Windows

Regarding cv_bridge: It is best to compile OpenCV before installing ROS, so that when installing ROS, cv_bridge will automatically point to the already installed OpenCV, and ROS will not install OpenCV separately. In this way, the only find_package and OpenCV on the computer can be found through the cv_bridge command, ensuring that the system environment is not polluted. For remedies, see the FAQ.

1.1.Ensure the graphics card operates normally

Please ensure that the NVIDIA driver, CUDA, and cuDNN are all installed successfully and that the versions are correct.

(Tutorial for installing drivers, CUDA, and cuDNN:Vinci Robotics Team Linux Introductory Tutorial)

# 检查显卡驱动
nvidia-smi

# 验证CUDA是否安装成功
nvcc -V

# 检查cuDNN版本命令(仅仅只是查了头文件)
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2

If the following appears, then you have the NVIDIA driver, CUDA, and cuDNN.

1.2.Install dependencies

# Debian系系统
sudo apt install -y libcurl4 build-essential pkg-config cmake-gui \
    libopenblas-dev libeigen3-dev libtbb-dev \
    libavcodec-dev libavformat-dev \
    libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev \
    libswscale-dev libgtk-3-dev libpng-dev libjpeg-dev \
    libcanberra-gtk-module libcanberra-gtk3-module libv4l-dev python3-dev python3-numpy

# RHEL红帽系系统
bash -c 'sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'
sudo dnf install -y curl gcc gcc-c++ make cmake cmake-gui \
    openblas-devel eigen3-devel tbb-devel \
    ffmpeg-libs ffmpeg-devel \
    gstreamer1-plugins-base-devel gstreamer1-devel \
    gtk3-devel libpng-devel libjpeg-devel \
    libc=aanberra-gtk3 libcanberra-devel v4l-utils v4l2loopback openexr-devel python3-dev python3-numpy

The table below describes these dependencies.

1.3.Download OpenCV source code

https://github.com/opencv/opencv

https://github.com/opencv/opencv_contrib

# 创建文件夹存放源码
mkdir -p ooppccvv
cd ooppccvv

# 解压源码
unzip ./opencv-4.11.0.zip
unzip ./opencv_contrib-4.11.0.zip

Confirm that the opencv directory and the opencv-contrib directory are located within the same parent directory, and confirm that both directories contain a modules subdirectory:(Generally, no need to confirm; as long as you type the commands I provided above exactly, there will be no problem.)

1.4.CMake build

1.4.1.Preparation

# 创建build文件夹用于装CMake生成的内容:
cd opencv-4.11.0
mkdir -p build && cd build

OpenCV is compiled using CMake and Makefile. There are many compilation options; for details, see (you can also skip this, though some CMake compilation options differ between versions):

https://docs.opencv.org/4.10.0/db/d05/tutorial_config_reference.html

The items struck through below are parameters that no longer exist in OpenCV4.11.0 but may still be valid in other versions of OpenCV; please useCMake-LAH(not recommended) command orCMake-gui(recommended) to view.

1. Query the GPU Compute Capability (CUDA_ARCH_BIN parameter):

https://developer.nvidia.com/cuda-gpus#collapseOne

After entering the website,

GeForce represents NVIDIA's gaming series of graphics cards, commonly including the GTX 1080, RTX 3080, RTX 4080, and others.

Jetson refers to a series of industrial PC graphics cards.

I have a 3060 Laptop (notebook mobile GPU, so look for the 3060 under Notebook in the right column).

If you are using a 3060 (desktop, look for the 3060 in the left column)

From the diagram, my GPU Compute Capability is 8.6, so the CUDA_ARCH_BIN parameter for my CMake is 8.6.

CUDA_ARCH_PTX is the maximum value of BIN. I set only one BIN, so the maximum value is 8.6. (Only when you want to replace the graphics card for the computer do you need to set multiple values for BIN, and you need to include all the values of the graphics cards you want to use in BIN, while PTX only needs the maximum value of BIN.)

2. For Python3 path detection, please use it in the terminal and check whether all results are generated. After ensuring the printed results meet expectations, proceed with the following CMake generation operation. (No need to remember the paths.)

Of course, you can also copy the path output by the command; this path is the value of the parameter.

# Python3 C++接口库的路径
python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))"

# Python3 矩阵库头文件的路径
python3 -c "import numpy; print(numpy.get_include())"

# OpenCV 的 Python3 包安装的路径。
python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"

# Python3 头文件的路径
python3 -c "import sysconfig; print(sysconfig.get_path('include'))"

1.4.2.CMake compilation **(choose one of the two methods)**

Because computer configurations differ, each computer's hardware, software (dependency packages), etc. are all different, so what I can run successfully may not necessarily run successfully for you right away.

Generally, it is difficult to have favorable weather; if there are problems, promptly search Baidu, Google, and the OpenCV forum for answers.

Forum:https://forum.opencv.org/

1.4.2.1.CMake terminal command method (not recommended; the GUI method is more recommended, as this terminal method is prone to strange issues)

cmake .. -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=/usr/local \
        -DBUILD_SHARED_LIBS=ON \
        -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.11.0/modules \
        -DOPENCV_ENABLE_NONFREE=ON \
        -DBUILD_TESTS=ON \
        -DBUILD_PERF_TESTS=ON \
        -DOPENCV_GENERATE_PKGCONFIG=ON \
        -DWITH_GTK=ON \
        -DWITH_CUDA=ON \
        -DENABLE_FAST_MATH=ON \
        -DCUDA_FAST_MATH=ON \
        -DWITH_CUBLAS=ON \
        -DCUDA_ARCH_BIN="8.6" \
        -DCUDA_ARCH_PTX="8.6" \
        -DCUDA_HOST_COMPILER=/usr/bin/gcc-13 \
        -DWITH_CUDNN=ON \
        -DOPENCV_DNN_CUDA=ON \
        -DWITH_IPP=ON \
        -DWITH_TBB=ON \
        -DWITH_OPENMP=ON \
        -DWITH_PTHREADS_PF=ON \
        -DOPENCV_PYTHON3_VERSION=3.12 \
        -DPYTHON3_EXECUTABLE=/usr/bin/python3 \
        -DPYTHON3_LIBRARY=$(python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))") \
        -DPYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
        -DPYTHON3_PACKAGES_PATH=$(python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))") \
        -DPYTHON3_INCLUDE_DIR=$(python3 -c "import sysconfig; print(sysconfig.get_path('include'))") \
        -DWITH_OPENGL=ON

You can add, remove, or modify these options as needed according to the previous instructions. If the cmake command fails, it is often due to missing dependencies or incorrectly configured generation options; check based on the prompt information. If it executes successfully, you can proceed to compilation (see the Makefiles compilation section):

1.4.2.2.CMake-GUI method (recommended; it is troublesome, but there are fewer problems and it is easier to locate problems)

1. Open CMake-GUI:

Configure these two paths.

Click Configure in the bottom left corner and select Makefiles

2. Configuration parameters

Configure each parameter one by one according to the table above. After filling in all options, click Configure when the configuration is complete.

If you encounter issues with configuration parameters, see the FAQ section below.

You can add, remove, or modify these options yourself as needed, following the previous instructions. If the cmake command fails, it is usually due to missing dependencies or incorrect generation options in the configuration; check based on the prompt information. If it succeeds, you can proceed to compilation (Makefiles compilation):

1.4.2.3.FAQ

####### No CUDA option Sometimes CMake-GUI only displays certain parameters after compilation (for example, CUDA-related options are only displayed after WITH_CUDA has been compiled).

So you need to first check WITH_CUDA, then click config in the bottom left corner. Only then will CUDA-related options appear, and then configure those options.

####### The module's path differs from the command-line approach It should be noted that this relative path is different from the parameter configured by directly typing the CMake command; here it is../opencv_contrib-4.11.0/modules.

####### OPENCV_PYTHON3_VERSION参数的类型错了 在cmake-gui中不知道为何OPENCV_PYTHON3_VERSION参数的类型成了布尔型。

It needs to be manually changed to string-type data.

The above image is where the problem lies—it is actually a boolean value here.

First, delete this option.

Then re-add this option.

# 查看python3版本
python3 --version

Enter 3.12 below; the subsequent minor version number does not need to be filled in.

1.5.Makefiles compilation

The meaning of the -j parameter after it is to use all CPUs for compilation. If CUDA is enabled, the generation process will be relatively slow, so please wait patiently. If errors occur during compilation, they are often due to version compatibility issues among the compiler, system libraries, and third-party libraries, and troubleshooting is relatively difficult.

Please execute the following command in the build directory.

# 内存小于16GB
make all

# 内存等于16GB
make all -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
make all -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
make all -j16

all-core compilation

As shown in the figure, this is what a truly successful compilation looks like, with no errors.

After generation is complete, run the following command to install:

$(grep -c ^processor /proc/cpuinfo)The variable is the number of CPU threads. (This allows the CPU to compile with full multithreading.)

# 内存小于16GB
sudo make install

# 内存等于16GB
sudo make install -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
sudo make install -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
sudo make install -j16

If there are no errors, the installation is successful.

1.6.Configure OpenCV environment variable ENV

1. You may need to configure the lib path:

vim ~/.bashrc

Add the content below at the very bottom

# 设置 LD_LIBRARY_PATH
export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"

2. Others require no configuration and work out of the box (you can configure pkg-config).

Check whether the installation was successful:

opencv_version

1.7.Test OpenCV_CUDA (CMake program example)

1. CMake is a must-learn; please refer to the documentation below to study:CMake C/C++ compilation environment configuration

2. As shown in the figure, the test is complete:

3. Below is the example project I have uploaded to GitHub for everyone to download: (The CMake template in the example project below is not the latest and may not be as feature-complete as the new version, nor as thoughtfully designed in all aspects. If you want to find the latest template, seeCMake C/C++ compilation environment setup)

https://github.com/tungchiahui/opencv_cuda_test

# 克隆源码
git clone https://github.com/tungchiahui/opencv_cuda_test.git

# cd进工程
cd opencv_cuda_test

# cd进build目录
cd build

# 进行cmake编译
cmake ..

# 进行make编译+进行make install安装(最大线程)
make install -j$(grep -c ^processor /proc/cpuinfo)

# 给予脚本执行权限
sudo chmod a+x ../script/setup_vinci_emis.bash
sudo chmod a+x ../script/vinci_emis
sudo chmod a+x ../install/.setup.bash

# 执行环境脚本
source ../script/setup_vinci_emis.bash

# 执行demo1二进制程序
../script/vinci_emis run demo1

OrClick directlyRun，Start DebuggingOrRun Without DebuggingEither is acceptable.(launch.json and task.json have both been fully configured)(If you find a bug, call me promptly. Before calling me, please check whether the latest CMake template has already fixed the bug. If it has not been fixed, then call me)

Testing complete

1.8.Frequently Asked Questions

1.8.1.cmake -jx compilation encounters operator!= or weight-related issues

  这个问题大多发生在ubuntu20.04或者cuda12.2上，编译时添加contrib库，对应ros版本为noetic，opencv版本为4.8.0。

  github上的讨论：

https://github.com/ros-perception/vision_opencv/tree/kinetic

According to the solution in GitHub, modify the corresponding code in the .hpp file where the error occurs (if there are only some WARNINGs, then modification is not necessary):

114 opencv/modules/dnn/src/cuda4dnn/primitives/normalize_bbox.hpp 中 if (weight != 1.0)Change to if (weight != static_cast<T>(1.0))

124 opencv/modules/dnn/src/cuda4dnn/primitives/region.hpp 中if (nms_iou_threshold > 0)

Change to if (nms_iou_threshold > static_cast<T>(0))

Compile again

make -j16

1.8.2.The issue of conflict between ROS's built-in OpenCV and a self-built opencv_cuda

Reason: ROS comes with cv_bridge which automatically links to ROS's OpenCV, and ROS's bundled OpenCV does not have CUDA acceleration, hence the error.

Ubuntu allows multiple versions of OpenCV to coexist; directly uninstalling OpenCV is not recommended, as it may cause related environments to become abnormal.

Solution: additionally configure a version of cv_bridge to link with OpenCV

1.8.2.1.ROS1

Solution:

在https://github.com/ros-perception/vision_opencv/tree/kinetic中下载对应版本的cv_bridge

先对cv_bridge中的CmakeList.txt进行修改，OpencvDIR对应自己的opencv安装路径，并将修改包名：

project(cv_bridge_480)#修改为你的包名，加个版本号就可以
set(OpenCV_DIR "/home/liu/opencv/opencv-4.8.0")
find_package(OpenCV 4.8.0 REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
  CONFIG
)

Modify the package name in package.xml

  <name>cv_bridge_480</name>

At this point, compile cv_bridge as a ROS package by copying the entire package into the src directory of your workspace for compilation.

cp -rf ./cv_bridge ~/Yolo_Tensorrt_Demo/demo01_test/src
catkin_make

Then it can be used as a functional package. In your original package's CMakeLists, add cv_bridge

find_package(cv_bridge_480)

package.xml:

<build_depend>cv_bridge_480</build_depend>
<build_export_depend>cv_bridge_480</build_export_depend>
<exec_depend>cv_bridge_480</exec_depend>

At this point, the custom cv_bridge package is configured.

If you still want to use code completion in vscode, add the path all the way to the include of the cv_bridge package. Here I have another workspace, but it's the same.

"includePath": [
    "/home/liu/cv_bridge_ws/src/cv_bridge/include",
    "/home/liu/cv_bridge_ws/src/cv_bridge"

]

1.8.2.2.ROS2

SeeROS2 Robot Operating System TutorialChapter CV_Bridge.

1.8.3.The problem of OpenCV compilation running out of memory

Solution: Use multi-core compilation, generally using make -j16 (here 16 is the number of CPU threads, which can be adjusted according to actual conditions) to solve the problem. The more CPU threads, the faster the compilation speed.

1.8.3.1.In Windows:

Mingw：

mingw64-make -j16

It is worth noting that mingw does not support opencv_contrib compilation on Windows, which will be indicated at the very beginning of configure.

Vs：

https://blog.csdn.net/hollyholly5/article/details/68062513

1.8.3.2.In Linux:

make -j$(grep -c ^processor /proc/cpuinfo)

2.Windows

See detailsInferencing YOLOv8 models using OpenCV (C++)