OpenCV CUDA Environment Setup

Linux

It is more recommended to deploy on Linux. Some deep learning tasks run significantly much faster on Linux than on Windows.

If you don't have spare disk space for Linux, consider WSL2 (Windows Subsystem for Linux 2). Although there is a slight performance loss, it is still much faster than Windows.

WSL2 Installation Tutorial Vinci Robot Team Linux Getting Started Tutorial

Physical Machine Linux > WSL2 > > Windows

Regarding cv_bridge: it is best to compile OpenCV before installing ROS. That way, when ROS is installed, cv_bridge will automatically point to the already installed OpenCV, and ROS will not install another version of OpenCV. This allows the find_package command to locate the only cv_bridge and OpenCV on the computer, keeping the system environment clean. For remediation steps, please refer to the FAQ.

Ensure the graphics card is functioning properly.

Please ensure that the NVIDIA driver, CUDA, and cuDNN are all installed successfully and with the correct versions.

(Driver, CUDA, and cuDNN Installation Tutorial: Vinci Robotics Team Linux Beginner Tutorial)

# 检查显卡驱动
nvidia-smi

# 验证CUDA是否安装成功
nvcc -V

# 检查cuDNN版本命令(仅仅只是查了头文件)
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2

If you see an image like the one below, it means you have the NVIDIA driver, CUDA, and cuDNN installed.

Install dependencies

# Debian系系统
sudo apt install -y libcurl4 build-essential pkg-config cmake-gui \
    libopenblas-dev libeigen3-dev libtbb-dev \
    libavcodec-dev libavformat-dev \
    libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev \
    libswscale-dev libgtk-3-dev libpng-dev libjpeg-dev \
    libcanberra-gtk-module libcanberra-gtk3-module libv4l-dev python3-dev python3-numpy

# RHEL红帽系系统
bash -c 'sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'
sudo dnf install -y curl gcc gcc-c++ make cmake cmake-gui \
    openblas-devel eigen3-devel tbb-devel \
    ffmpeg-libs ffmpeg-devel \
    gstreamer1-plugins-base-devel gstreamer1-devel \
    gtk3-devel libpng-devel libjpeg-devel \
    libc=aanberra-gtk3 libcanberra-devel v4l-utils v4l2loopback openexr-devel python3-dev python3-numpy

The table below explains these dependencies.

Download the OpenCV source code

https://github.com/opencv/opencv

https://github.com/opencv/opencv\_contrib

# 创建文件夹存放源码
mkdir -p ooppccvv
cd ooppccvv

# 解压源码
unzip ./opencv-4.11.0.zip
unzip ./opencv_contrib-4.11.0.zip

Verify that the opencv directory and the opencv-contrib directory are located within the same parent directory, and confirm that both directories contain a modules subdirectory: (Generally, no need to verify — as long as you follow the commands I provided above, it will be fine)

CMake compilation

Preparation Work

# 创建build文件夹用于装CMake生成的内容:
cd opencv-4.11.0
mkdir -p build && cd build

OpenCV is compiled using CMake and Makefile, with numerous compilation options. For details, see (you can skip this, though CMake compilation options may vary across different versions):

https://docs.opencv.org/4.10.0/db/d05/tutorial\_config\_reference.html

The items crossed out below are parameters that no longer exist in OpenCV 4.11.0, but may still be valid in other versions of OpenCV. Please check on your own using the CMake-LAH (not recommended) command or CMake-gui (recommended).

1. Query GPU Compute Capability (CUDA_ARCH_BIN parameter):

https://developer.nvidia.com/cuda-gpus#collapseOne

After entering the website,

GeForce represents NVIDIA's gaming series graphics cards, common examples include the GTX 1080, RTX 3080, RTX 4080, and others.

Jetson represents the industrial computer series GPU.

I have a 3060 Laptop (a mobile laptop GPU, so look for the 3060 under the Notebook column on the right).

If you have a 3060 (desktop version, look for the 3060 in the left column)

From the diagram, my GPU Compute Capability is 8.6, so my CMake CUDA_ARCH_BIN parameter is set to 8.6.

CUDA_ARCH_PTX is the highest value of BIN. I only set one BIN, so the highest value is 8.6. (Only if you are replacing your computer's graphics card should you set multiple values for BIN, in which case you need to include all the values for the graphics cards you intend to use in BIN, while PTX only needs the highest value of BIN.)

2. To check the Python3 path query, please use it in the terminal and verify that results are generated. Ensure the printed output meets expectations before proceeding with the following CMake generation steps. (No need to memorize the path.)

Of course, you can also copy the path returned by the command, as this path is the value of the parameter.

# Python3 C++接口库的路径
python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))"

# Python3 矩阵库头文件的路径
python3 -c "import numpy; print(numpy.get_include())"

# OpenCV 的 Python3 包安装的路径。
python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"

# Python3 头文件的路径
python3 -c "import sysconfig; print(sysconfig.get_path('include'))"

CMake Compilation (Choose one of the two methods)

Because computer configurations vary, each machine has different hardware, software (dependency packages), and so on. So just because it runs on my computer doesn't mean it will run successfully on yours right away.

It's rarely smooth sailing, so if you run into issues, promptly search for answers on Baidu, Google, or the OpenCV forum.

Forum:https://forum.opencv.org/

CMake terminal command method (not recommended; the GUI method is preferred, as the terminal approach is prone to strange issues)

cmake .. -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=/usr/local \
        -DBUILD_SHARED_LIBS=ON \
        -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.11.0/modules \
        -DOPENCV_ENABLE_NONFREE=ON \
        -DBUILD_TESTS=ON \
        -DBUILD_PERF_TESTS=ON \
        -DOPENCV_GENERATE_PKGCONFIG=ON \
        -DWITH_GTK=ON \
        -DWITH_CUDA=ON \
        -DENABLE_FAST_MATH=ON \
        -DCUDA_FAST_MATH=ON \
        -DWITH_CUBLAS=ON \
        -DCUDA_ARCH_BIN="8.6" \
        -DCUDA_ARCH_PTX="8.6" \
        -DCUDA_HOST_COMPILER=/usr/bin/gcc-13 \
        -DWITH_CUDNN=ON \
        -DOPENCV_DNN_CUDA=ON \
        -DWITH_IPP=ON \
        -DWITH_TBB=ON \
        -DWITH_OPENMP=ON \
        -DWITH_PTHREADS_PF=ON \
        -DOPENCV_PYTHON3_VERSION=3.12 \
        -DPYTHON3_EXECUTABLE=/usr/bin/python3 \
        -DPYTHON3_LIBRARY=$(python3 -c "import sysconfig; from os.path import join; print(join(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')))") \
        -DPYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
        -DPYTHON3_PACKAGES_PATH=$(python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))") \
        -DPYTHON3_INCLUDE_DIR=$(python3 -c "import sysconfig; print(sysconfig.get_path('include'))") \
        -DWITH_OPENGL=ON

You can add, remove, or modify these options as needed, following the previous instructions. If the cmake command reports an error, it is often due to missing dependencies or incorrect generator options — check the error messages to troubleshoot. If it runs successfully, you can proceed with the build (see the Makefiles build section):

CMake-GUI method (recommended, though more cumbersome, but with fewer issues and easier troubleshooting)

1. Open CMake-GUI:

Configure these two paths.

Click the configuration option in the lower left corner and select Makefiles.

2. Configuration parameters

Configure each parameter one by one according to the table above. After filling in all the options, click Configure to complete the setup.

If you encounter issues while configuring parameters, please refer to the Common Problems section below.

You can add, remove, or modify these options as needed, following the previous instructions. If the cmake command reports an error, it is often due to missing dependencies or incorrect generator options — check the error messages to troubleshoot. If it runs successfully, you can proceed to compile (using Makefiles):

Frequently Asked Questions

####### No CUDA Option Sometimes CMake-GUI only displays certain parameters after compilation (for example, CUDA-related options only appear after compiling WITH_CUDA).

So you need to first check WITH_CUDA, then click config in the lower-left corner. This will make the CUDA-related options appear, and then you can configure those options.

The path of the module is different from the command line method. It is important to note that this relative path is different from the parameter configured by directly typing the CMake command; here it is ../opencv_contrib-4.11.0/modules.

The type of the OPENCV_PYTHON3_VERSION parameter is incorrect. In cmake-gui, for some reason the type of the OPENCV_PYTHON3_VERSION parameter became boolean.

Need to manually change to string data type.

The image above shows the issue — it's actually a boolean value here.

First, delete that option.

Add this option again.

# 查看python3版本
python3 --version

Fill in 3.12 below; no need to fill in the minor version number after it.

Makefiles Compilation

The meaning of the -j parameter is to use all CPUs for compilation. If CUDA is enabled, the build process will be slower, so please be patient. If errors occur during compilation, they are often due to version compatibility issues with the compiler, system libraries, or third-party libraries, and troubleshooting them can be quite challenging.

Please run the following command in the build directory.

# 内存小于16GB
make all

# 内存等于16GB
make all -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
make all -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
make all -j16

Full-core compilation

As shown in the figure, that's a truly successful compilation with no errors.

After generation is complete, run the following command to install:

$(grep -c ^processor /proc/cpuinfo) variable is the number of CPU threads. (This allows the CPU to compile with full multi-threading.)

# 内存小于16GB
sudo make install

# 内存等于16GB
sudo make install -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 ))

# 内存大于32GB
sudo make install -j$(grep -c ^processor /proc/cpuinfo)

# 或者自行规定线程数量（比如16线程）
sudo make install -j16

If there are no errors, the installation was successful.

Configure OpenCV environment variable ENV

1. You may need to configure the lib path:

vim ~/.bashrc

Add the following content at the very bottom.

# 设置 LD_LIBRARY_PATH
export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"

2. No other configuration needed—works right out of the box (you can configure pkg-config if desired).

Check if the installation was successful:

opencv_version

Test OpenCV_CUDA (CMake Program Example)

1. CMake is a must-learn. Please refer to the following document: CMake C/C++ Compilation Environment Setup
2. As shown in the figure, the test is complete:

3. Below is the example project I have uploaded to GitHub for everyone to download: (The CMake template in the example project below is not the latest version and may not be as feature-rich or as well-designed in all aspects as the new version. If you want the latest template, please see CMake C/C++ Compilation Environment Setup)

https://github.com/tungchiahui/opencv\_cuda\_test

# 克隆源码
git clone https://github.com/tungchiahui/opencv_cuda_test.git

# cd进工程
cd opencv_cuda_test

# cd进build目录
cd build

# 进行cmake编译
cmake ..

# 进行make编译+进行make install安装(最大线程)
make install -j$(grep -c ^processor /proc/cpuinfo)

# 给予脚本执行权限
sudo chmod a+x ../script/setup_vinci_emis.bash
sudo chmod a+x ../script/vinci_emis
sudo chmod a+x ../install/.setup.bash

# 执行环境脚本
source ../script/setup_vinci_emis.bash

# 执行demo1二进制程序
../script/vinci_emis run demo1

Or simply click Run, Start Debugging, or Run Without Debugging. (The launch.json and task.json are already fully configured.) (If you find a bug, call me promptly. Before calling me, please check whether the latest CMake template has already fixed the bug. If it hasn't been fixed, then call me.)

Testing complete.

Frequently Asked Questions

When compiling with cmake -jx, encountering issues related to operator!= or weight.

This issue mostly occurs on Ubuntu 20.04 or with CUDA 12.2. When compiling, add the contrib library. The corresponding ROS version is Noetic, and the OpenCV version is 4.8.0.

GitHub discussion:

https://github.com/ros-perception/vision\_opencv/tree/kinetic

Based on the solution in GitHub, modify the corresponding code in the error-related .hpp file (if it's just some WARNINGs, there's no need to modify):

114 opencv/modules/dnn/src/cuda4dnn/primitives/normalize_bbox.hpp 中 if (weight != 1.0) changed to if (weight != static_cast<T>(1.0))

124 opencv/modules/dnn/src/cuda4dnn/primitives/region.hpp 中if (nms_iou_threshold > 0)

Change to if (nms_iou_threshold > static_cast<T>(0))

Compile again

make -j16

Conflict between the original ROS OpenCV and a custom-built OpenCV_CUDA.

Reason: The cv_bridge that comes with ROS automatically links to ROS's OpenCV, and ROS's OpenCV does not have CUDA acceleration, hence the error.

Ubuntu allows multiple versions of OpenCV to coexist. It is not recommended to uninstall OpenCV directly, as this may cause environment-related issues.

Solution: Configure an additional version of cv_bridge for OpenCV linking.

ROS1

Solution:

In https://github.com/ros-perception/vision\_opencv/tree/kinetic中下载对应版本的cv\_bridge

First, modify the CMakeList.txt in cv_bridge, set OpenCV_DIR to your own OpenCV installation path, and rename the package.

project(cv_bridge_480)#修改为你的包名，加个版本号就可以
set(OpenCV_DIR "/home/liu/opencv/opencv-4.8.0")
find_package(OpenCV 4.8.0 REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
  CONFIG
)

Modify the package name in package.xml

  <name>cv_bridge_480</name>

Now compile cv_bridge as a ROS package. Copy the entire package into the src folder of your workspace and build it.

cp -rf ./cv_bridge ~/Yolo_Tensorrt_Demo/demo01_test/src
catkin_make

Then you can use cv_bridge as a functional package. Add the following to the CMakeLists.txt of your existing package:

find_package(cv_bridge_480)

package.xml:

<build_depend>cv_bridge_480</build_depend>
<build_export_depend>cv_bridge_480</build_export_depend>
<exec_depend>cv_bridge_480</exec_depend>

At this point, the custom cv_bridge package is configured.

If you also want to use code completion in VS Code, just add the path all the way to the include directory of the cv_bridge package. In my case, it's a different workspace, but it works the same way.

"includePath": [
    "/home/liu/cv_bridge_ws/src/cv_bridge/include",
    "/home/liu/cv_bridge_ws/src/cv_bridge"

]

ROS2

See the CV_Bridge section in the ROS2 Robot Operating System Tutorial.

The issue of OpenCV compilation running out of memory.

Solution: Use multi-core compilation. Typically, running make -j16 (where 16 is the number of CPU threads, adjustable based on your situation) will resolve this. The more CPU threads, the faster the compilation speed.

In Windows:

Mingw：

mingw64-make -j16

It is worth noting that MinGW does not support compiling opencv_contrib on Windows, and this will be indicated at the very beginning of the configure step.

Vs：

https://blog.csdn.net/hollyholly5/article/details/68062513

In Linux:

make -j$(grep -c ^processor /proc/cpuinfo)

Windows

See Using OpenCV to Infer Yolov8 Model (C++)