OpenCV_CUDA environment setup

Linux

It is more recommended to deploy on Linux. Some deep learning tasks run significantly much faster on Linux than on Windows.

If you don't have spare disk space for Linux, consider WSL2 (Windows Subsystem for Linux 2). Although there is a slight performance loss, it is still much faster than Windows.

WSL2 Installation Tutorial Vinci Robot Team Linux Getting Started Tutorial

Physical Machine Linux > WSL2 > > Windows

Regarding cv_bridge: It's best to compile OpenCV before installing ROS. This way, when ROS is installed, cv_bridge will automatically point to the already installed OpenCV, and ROS won't install a separate version of OpenCV. As a result, the find_package command will locate only the cv_bridge and OpenCV present on your computer, ensuring a pollution-free system environment. For recovery methods, please refer to the troubleshooting section.

Ensure the graphics card is functioning properly.

Please ensure that the NVIDIA driver, CUDA, and cuDNN are all installed successfully and with the correct versions.

(Driver, CUDA, and cuDNN Installation Tutorial: Vinci Robotics Team Linux Beginner Tutorial)

# 检查显卡驱动
nvidia-smi

# 验证CUDA是否安装成功
nvcc -V

# 检查cuDNN版本命令(仅仅只是查了头文件)
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2

If you see an image like the one below, it means you have the NVIDIA driver, CUDA, and cuDNN installed.

Install dependencies

# Debian系系统
sudo apt install -y libcurl4 build-essential pkg-config cmake-gui \
    libopenblas-dev libeigen3-dev libtbb-dev \
    libavcodec-dev libavformat-dev \
    libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev \
    libswscale-dev libgtk-3-dev libpng-dev libjpeg-dev \
    libcanberra-gtk-module libcanberra-gtk3-module libv4l-dev python3-dev python3-numpy

# RHEL红帽系系统
bash -c 'sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'
sudo dnf install -y curl gcc gcc-c++ make cmake cmake-gui \
    openblas-devel eigen3-devel tbb-devel \
    ffmpeg-libs ffmpeg-devel \
    gstreamer1-plugins-base-devel gstreamer1-devel \
    gtk3-devel libpng-devel libjpeg-devel \
    libc=aanberra-gtk3 libcanberra-devel v4l-utils v4l2loopback openexr-devel python3-dev python3-numpy

The table below explains these dependencies.

Download the OpenCV source code

https://github.com/opencv/opencv

https://github.com/opencv/opencv_contrib

# 创建文件夹存放源码
mkdir -p ooppccvv
cd ooppccvv

# 解压源码
unzip ./opencv-4.11.0.zip
unzip ./opencv_contrib-4.11.0.zip

Verify that the opencv directory and the opencv-contrib directory are located within the same parent directory, and confirm that both directories contain a modules subdirectory: (Generally, no need to verify — as long as you follow the commands I provided above, it will be fine)

CMake compilation

Preparation Work

# 创建build文件夹用于装CMake生成的内容:
cd opencv-4.11.0
mkdir -p build && cd build

OpenCV is compiled using CMake and Makefile, with numerous compilation options. For details, see (you can skip this, though CMake compilation options may vary across different versions):

https://docs.opencv.org/4.10.0/db/d05/tutorial_config_reference.html

The items crossed out below are parameters that no longer exist in OpenCV 4.11.0, but may still be valid in other versions of OpenCV. Please check on your own using the CMake-LAH (not recommended) command or CMake-gui (recommended).

1. Querying GPU Compute Capability (for the CUDA_ARCH_BIN parameter):

https://developer.nvidia.com/cuda-gpus#collapseOne

After entering the website,

GeForce represents NVIDIA's gaming series graphics cards, common examples include the GTX 1080, RTX 3080, RTX 4080, and others.

Jetson represents the industrial computer series GPU.

I have a 3060 Laptop (a mobile laptop GPU, so look for the 3060 under the Notebook column on the right).

If you have a 3060 (desktop version, look for the 3060 in the left column)

Based on the diagram, my GPU Compute Capability is 8.6, so my CMake CUDA_ARCH_BIN parameter is set to 8.6.

The highest value for CUDA_ARCH_PTX is indeed the maximum among the BIN values. Since you've set only one BIN value (8.6), that becomes the highest value. (Only when you need to support multiple GPUs on a computer do you set multiple values for BIN—ensuring they cover all target GPUs—while PTX only requires the highest value from BIN.)

2. To check the Python3 path query, please use it in the terminal and verify that results are generated. Ensure the printed output meets expectations before proceeding with the following CMake generation steps. (No need to memorize the path.)

Of course, you can also copy the path returned by the command, as this path is the value of the parameter.

# Python3 C++接口库的路径
python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))"

# Python3 矩阵库头文件的路径
python3 -c "import numpy; print(numpy.get_include())"

# OpenCV 的 Python3 包安装的路径。
python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"

# Python3 头文件的路径
python3 -c "import sysconfig; print(sysconfig.get_path('include'))"

CMake Compilation (Choose one of the two methods)

Because computer configurations vary, each machine has different hardware, software (dependency packages), and so on. So just because it runs on my computer doesn't mean it will run successfully on yours right away.

It's rarely smooth sailing, so if you run into issues, promptly search for answers on Baidu, Google, or the OpenCV forum.

Forum:https://forum.opencv.org/

CMake terminal command method (not recommended; the GUI method is preferred, as the terminal approach is prone to strange issues)

cmake .. -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=/usr/local \
        -DBUILD_SHARED_LIBS=ON \
        -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.11.0/modules \
        -DOPENCV_ENABLE_NONFREE=ON \
        -DBUILD_TESTS=ON \
        -DBUILD_PERF_TESTS=ON \
        -DOPENCV_GENERATE_PKGCONFIG=ON \
        -DWITH_GTK=ON \
        -DWITH_CUDA=ON \
        -DENABLE_FAST_MATH=ON \
        -DCUDA_FAST_MATH=ON \
        -DWITH_CUBLAS=ON \
        -DCUDA_ARCH_BIN="8.6" \
        -DCUDA_ARCH_PTX="8.6" \
        -DCUDA_HOST_COMPILER=/usr/bin/gcc-13 \
        -DWITH_CUDNN=ON \
        -DOPENCV_DNN_CUDA=ON \
        -DWITH_IPP=ON \
        -DWITH_TBB=ON \
        -DWITH_OPENMP=ON \
        -DWITH_PTHREADS_PF=ON \
        -DOPENCV_PYTHON3_VERSION=3.12 \
        -DPYTHON3_EXECUTABLE=/usr/bin/python3 \
        -DPYTHON3_LIBRARY=$(python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))") \
        -DPYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
        -DPYTHON3_PACKAGES_PATH=$(python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))") \
        -DPYTHON3_INCLUDE_DIR=$(python3 -c "import sysconfig; print(sysconfig.get_path('include'))") \
        -DWITH_OPENGL=ON

You can add, remove, or modify these options as needed, following the previous instructions. If the cmake command reports an error, it is often due to missing dependencies or incorrect generator options — check the error messages to troubleshoot. If it runs successfully, you can proceed with the build (see the Makefiles build section):

CMake-GUI method (recommended, though more cumbersome, but with fewer issues and easier troubleshooting)

1. Open CMake-GUI:

Configure these two paths.

Click the configuration option in the lower left corner and select Makefiles.

2. Configuration parameters

Configure each parameter one by one according to the table above. After filling in all the options, click Configure to complete the setup.

If you encounter issues while configuring parameters, please refer to the Common Problems section below.

You can add, remove, or modify these options as needed, following the previous instructions. If the cmake command reports an error, it is often due to missing dependencies or incorrect generator options — check the error messages to troubleshoot. If it runs successfully, you can proceed to compile (using Makefiles):

Frequently Asked Questions

####### No CUDA Option Sometimes CMake-GUI will only display certain parameters after compilation, such as CUDA-related options appearing only if compiled with WITH_CUDA enabled.

So you need to first check the box for WITH_CUDA, then click on config at the bottom left, so that the options related to CUDA will appear, and then configure those options.

The path of the module is different from the command line method. It is important to note that this relative path is different from the parameter configured by directly typing the CMake command; here it is ../opencv_contrib-4.11.0/modules.

####### The type of parameter OPENCV_PYTHON3_VERSION is incorrect In the cmake-gui, for some reason, the type of the OPENCV_PYTHON3_VERSION parameter has become a boolean.

Need to manually change to string data type.

The image above shows the issue — it's actually a boolean value here.

First, delete that option.

Add this option again.

# 查看python3版本
python3 --version

Fill in 3.12 below; no need to fill in the minor version number after it.

Makefiles Compilation

The meaning of the -j parameter is to use all CPUs for compilation. If CUDA is enabled, the build process will be slower, so please be patient. If errors occur during compilation, they are often due to version compatibility issues with the compiler, system libraries, or third-party libraries, and troubleshooting them can be quite challenging.

Please run the following command in the build directory.

# 内存小于16GB
make all

# 内存等于16GB
make all -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
make all -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
make all -j16

Full-core compilation

As shown in the figure, that's a truly successful compilation with no errors.

After generation is complete, run the following command to install:

$(grep -c ^processor /proc/cpuinfo) variable is the number of CPU threads. (This allows the CPU to compile with full multi-threading.)

# 内存小于16GB
sudo make install

# 内存等于16GB
sudo make install -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
sudo make install -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
sudo make install -j16

If there are no errors, the installation was successful.

Configure OpenCV environment variable ENV

1. You may need to configure the lib path:

vim ~/.bashrc

Add the following content at the very bottom.

# 设置 LD_LIBRARY_PATH
export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"

2. No other configuration needed—works right out of the box (you can configure pkg-config if desired).

Check if the installation was successful:

opencv_version

Test OpenCV_CUDA (CMake Code Example)

1. CMake is a must-learn. Please refer to the following document: CMake C/C++ Compilation Environment Setup
2. As shown in the figure, the test is complete:

3. Below is the example project I have uploaded to GitHub for everyone to download: (The CMake template in the example project below is not the latest version and may not be as feature-rich or as well-designed in all aspects as the new version. If you want the latest template, please see CMake C/C++ Compilation Environment Setup)

https://github.com/tungchiahui/opencv_cuda_test

# 克隆源码
git clone https://github.com/tungchiahui/opencv_cuda_test.git

# cd进工程
cd opencv_cuda_test

# cd进build目录
cd build

# 进行cmake编译
cmake ..

# 进行make编译+进行make install安装(最大线程)
make install -j$(grep -c ^processor /proc/cpuinfo)

# 给予脚本执行权限
sudo chmod a+x ../script/setup_vinci_emis.bash
sudo chmod a+x ../script/vinci_emis
sudo chmod a+x ../install/.setup.bash

# 执行环境脚本
source ../script/setup_vinci_emis.bash

# 执行demo1二进制程序
../script/vinci_emis run demo1

Or simply click Run, Start Debugging, or Run Without Debugging. (The launch.json and task.json are already fully configured.) (If you find a bug, call me promptly. Before calling me, please check whether the latest CMake template has already fixed the bug. If it hasn't been fixed, then call me.)

Testing complete.

Frequently Asked Questions

When compiling with cmake -jx, encountering issues related to operator!= or weight.

This issue mostly occurs on Ubuntu 20.04 or with CUDA 12.2. When compiling, add the contrib library. The corresponding ROS version is Noetic, and the OpenCV version is 4.8.0.

GitHub discussion:

https://github.com/ros-perception/vision_opencv/tree/kinetic

Based on the solution in GitHub, modify the corresponding code in the error-related .hpp file (if it's just some WARNINGs, there's no need to modify):

114 opencv/modules/dnn/src/cuda4dnn/primitives/normalize_bbox.hpp 中 if (weight != 1.0) changed to if (weight != static_cast<T>(1.0))

124 opencv/modules/dnn/src/cuda4dnn/primitives/region.hpp 中if (nms_iou_threshold > 0)

Change to if (nms_iou_threshold > static_cast<T>(0))

Compile again

make -j16

The conflict between ROS's bundled OpenCV and a custom-built OpenCV with CUDA support.

The reason is that ROS's built-in cv_bridge automatically links to ROS's own OpenCV, which lacks CUDA acceleration, resulting in the error.

Ubuntu allows multiple versions of OpenCV to coexist. It is not recommended to uninstall OpenCV directly, as this may cause environment-related issues.

Solution: Configure an additional version of cv_bridge for OpenCV linking.

ROS1

Solution:

In https://github.com/ros-perception/vision_opencv/tree/kinetic中下载对应版本的cv_bridge

First, modify the CMakeLists.txt file in cv_bridge, set OpenCV_DIR to your own OpenCV installation path, and then change the package name:

project(cv_bridge_480)#修改为你的包名，加个版本号就可以
set(OpenCV_DIR "/home/liu/opencv/opencv-4.8.0")
find_package(OpenCV 4.8.0 REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
  CONFIG
)

Modify the package name in package.xml

  <name>cv_bridge_480</name>

Now, compile cv_bridge as a ROS package. Copy the entire package into the src directory of your workspace and compile it.

cp -rf ./cv_bridge ~/Yolo_Tensorrt_Demo/demo01_test/src
catkin_make

Then you can use cv_bridge as a package by adding it to your original package's CMakeLists.

find_package(cv_bridge_480)

package.xml:

<build_depend>cv_bridge_480</build_depend>
<build_export_depend>cv_bridge_480</build_export_depend>
<exec_depend>cv_bridge_480</exec_depend>

The custom cv_bridge package has now been configured.

If you also want code completion in VSCode, simply add the path all the way up to the include directory of the cv_bridge package. Mine is in a different workspace, but it's the same principle.

"includePath": [
    "/home/liu/cv_bridge_ws/src/cv_bridge/include",
    "/home/liu/cv_bridge_ws/src/cv_bridge"

]

ROS2

For details, see the CV_Bridge section in the Robot Operating System 2 Tutorial.

The issue of OpenCV compilation running out of memory.

Solution: Use multi-core compilation. Typically, running make -j16 (where 16 is the number of CPU threads, adjustable based on your situation) will resolve this. The more CPU threads, the faster the compilation speed.

In Windows:

Mingw：

mingw64-make -j16

It's worth noting that MinGW doesn't support compiling OpenCV_contrib on Windows, and this will be flagged at the very start of the configure process.

Vs：

https://blog.csdn.net/hollyholly5/article/details/68062513

In Linux:

make -j$(grep -c ^processor /proc/cpuinfo)

Windows

See Using OpenCV to Infer Yolov8 Model (C++)