JetPack 5.1 comes with OpenCV 4.5 installed.
However, many developers, including myself, prefer Python virtual environments using Anaconda. New OpenCV can be installed by creating a new virtual environment in Anaconda.
The advantage of this method is that existing OpenCV 4.5 can be used outside the virtual environment.
In this article, I will explain how to install the fosine version of OpenCV in anaconda virtual environment while maintaining the existing OpenCV 4.5. As of March 2023, the latest version of OpenCV is 4.7. However, it is not difficult for you to update the existing OpenCV 4.5 to the newly built OpenCV 4.7. I will explain this method as well.

Build the latest version of OpenCV in Anaconda

Prerequisites

You should install JetPack 5.1 in advance. And anaconda should also be installed.

Creating a Virtual Environment for OpenCV Installation

First, create a Python virtual environment. I will build and install a new OpenCV in this virtual environment.

# Then Create the env again
spypiggy@spypiggy-NX:~ $ conda create -n yolov8 python=3.8.16
spypiggy@spypiggy-NX:~ $ conda activate yolov8

Then execute the following script. This script stores the required package installation and the process of downloading, compiling, and installing OpenCV 4.7 source code in one file.

You can download this script at my repo(https://github.com/raspberry-pi-maker/NVIDIA-Jetson/tree/master/useful_scripts)

# part of the script to be modified
# 1. opencv version to install. The current version is 4.7.0
# 2. python version of anaconda virtual environment. The current version is 3.8
# 3. CUDA_ARCH_BIN. The Current value is "7.2" for Xavier NX


echo "OpenCV 4.7 Installation script for Jetson Xavier NX (Jetpack 5.1)"

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <Install Folder>"
    exit
fi
folder="$1"

#You don't have to remove pre-installed OpenCV 4.5
#echo "** Remove Old OpenCV first"
#sudo apt-get purge *libopencv*

echo "** Install requirement"
sudo apt-get update
sudo apt-get install -y build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
sudo apt-get install -y libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
sudo apt-get install -y libopenblas-dev libatlas-base-dev libblas-dev 
sudo apt-get install -y libeigen3-dev libhdf5-dev libgflags-dev
sudo apt-get install -y libv4l-dev v4l-utils qv4l2 v4l2ucp
sudo apt-get update
conda install -y numpy

echo "** Download opencv-4.7.0"
cd $folder
wget https://github.com/opencv/opencv/archive/4.7.0.zip 
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.7.0.zip 

unzip 4.7.0.zip 
unzip opencv_contrib.zip 
cd opencv-4.7.0/

echo "** Building..."
mkdir release
cd release
cmake \
-D ENABLE_PRECOMPILED_HEADERS=0  \
-D CUDA_ARCH_BIN="7.2" \
-D CUDA_ARCH_PTX="" \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.7.0/modules \
-D WITH_OPENCL=0 \
-D WITH_CUDA=1 \
-D OPENCV_DNN_CUDA=1 \
-D EIGEN_INCLUDE_PATH=/usr/include/eigen3 \
-D WITH_EIGEN=1 \
-D WITH_CUBLAS=1 \
-D WITH_FFMPEG=1 \
-D WITH_GSTREAMER=1 \
-D WITH_V4L=1 \
-D WITH_LIBV4L=1 \
-D BUILD_opencv_python2=0 \
-D BUILD_opencv_python3=1 \
-D BUILD_TESTS=0 \
-D BUILD_PERF_TESTS=0 \
-D BUILD_EXAMPLES=0 \
-D OPENCV_GENERATE_PKGCONFIG=1 \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
-D PYTHON3_LIBRARY=$CONDA_PREFIX/lib/python3.8 \
-D PYTHON3_INCLUDE_DIR=$CONDA_PREFIX/include/python3.8 \
-D PYTHON3_EXECUTABLE=$CONDA_PREFIX/bin/python \
-D PYTHON3_PACKAGES_PATH=$CONDA_PREFIX/lib/python3.8/site-packages \
..

make -j6
sudo make install

echo "** Install opencv-4.7.0 successfully"
echo "** Bye :)"

<install_opencv4.7_NX.sh>

When using an anaconda virtual environment, you need to know the Python path used. I created a virtual environment named yolov8. Then there are execution Python and installation packages in the following directories. The virtual environment Python path is stored in $CONDA_PREFIX.

(base) spypiggy@spypiggy-NX:/usr/lib/aarch64-linux-gnu$ cd ~/src
(base) spypiggy@spypiggy-NX:~/src$ conda activate yolov8
(yolov8) spypiggy@spypiggy-NX:~$ echo $CONDA_PREFIX
/home/spypiggy/anaconda3/envs/yolov8

Python executable path: /home/spypiggy/anaconda3/env/yolov8/bin/python
Python packages path: /home/spypiggy/anaconda3/env/yolov8/lib/python3.8/site-packages

The part of the script above that you should pay attention to is the red text.

This is the part where you set the anaconda python related build environment variable while creating a makefile with the cmake command. These environment variables should specify Anaconda's virtual environment Python.

Now run the script for building OpenCV 4.7 for Anaconda Virtual Environment!

I copied install_opencv4.7_NX.sh to ~/src directory first. This build process will take more than an hour.

(yolov8) spypiggy@spypiggy-NX:~$ cd ~/src
(yolov8) spypiggy@spypiggy-NX:~/src$ bash install_opencv4.7_NX.sh

If it ended well without errors, it is anacorn as follows. The cv2 directory should be created in the site-packages directory of the virtual environment.

(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8/lib/python3.8/site-packages$ pwd
/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8/lib/python3.8/site-packages$ ls -ald cv2
drwxr-xr-x 7 root root 4096  3월 13 08:33 cv2

libffi patch

Now, let's test the cv2 package to see if it works well.

(yolov8) spypiggy@spypiggy-NX:~$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/__init__.py", line 153, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /lib/aarch64-linux-gnu/libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0
>>>

Something went wrong. After several googling, libffi is the cause of this error.I could see that it was related to so.

Look up libffi related files in the /home/spypiggy/anaconda3/envs/yolov8/lib/ directory. You can see that the libffi.7.so file and the libffi.so.7 file are linked to version 8. You'll feel a little strange.

(yolov8) spypiggy@spypiggy-NX:~/anaconda3$ ls -al envs/yolov8/lib/libffi*
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 envs/yolov8/lib/libffi.7.so -> libffi.so.8.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 envs/yolov8/lib/libffi.8.so -> libffi.so.8.1.0
-rw-rw-r-- 2 spypiggy spypiggy 114688 11월 22 23:17 envs/yolov8/lib/libffi.a
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 envs/yolov8/lib/libffi.so -> libffi.so.8.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 envs/yolov8/lib/libffi.so.7 -> libffi.so.8.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 envs/yolov8/lib/libffi.so.8 -> libffi.so.8.1.0
-rwxrwxr-x 2 spypiggy spypiggy 104400 11월 22 23:17 envs/yolov8/lib/libffi.so.8.1.0
(yolov8) spypiggy@spypiggy-NX:~/anaconda3

This time, let's look for these files in the /home/spypiggy/anaconda3/lib directory.

(yolov8) spypiggy@spypiggy-NX:~/anaconda3/lib$ ls -al  libffi*lrwxrwxrwx 1 spypiggy spypiggy    15  3월  9 20:54 libffi.so -> libffi.so.7.1.0*
lrwxrwxrwx 1 spypiggy spypiggy    15  3월  9 20:54 libffi.so.6 -> libffi.so.7.1.0*
lrwxrwxrwx 1 spypiggy spypiggy    15  3월  9 20:54 libffi.so.7 -> libffi.so.7.1.0*
-rwxrwxr-x 2 spypiggy spypiggy 50376  3월  7  2021 libffi.so.7.1.0*

The libffi version 7 exists in this directory, and the symbolic link is also linked to the version 7.

It is not known what causes the symbolic links to change, but restoring these links solves the problem.

Copy the /home/spypiggy/anaconda3/lib/libffi.so.7.1.0 file to the /home/spypiggy/anaconda3/envs/yolov8/lib/ directory, and then recreate the symbolic link.

lrwxrwxrwx 1 spypiggy spypiggy     15  3월 13 17:09 libffi.7.1.0 -> libffi.so.7.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 13 18:03 libffi.7.so -> libffi.so.7.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 libffi.8.so -> libffi.so.8.1.0
-rw-rw-r-- 2 spypiggy spypiggy 114688 11월 22 23:17 libffi.a
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 12 16:22 libffi.so -> libffi.so.8.1.0
-rw-r--r-- 1 spypiggy spypiggy  50376  3월 13 23:06 libffi.so.7.1.0
lrwxrwxrwx 1 spypiggy spypiggy     15  3월 13 22:50 libffi.so.8 -> libffi.so.8.1.0
-rw-rw-r-- 2 spypiggy spypiggy  35144  3월 13 17:02 libffi.so.8.1.0

The libffi issue may be related to the version of OpenCV being built. When a lower version than the latest version 4.7 is built, the above error may not occur.

Now let's check out the built cv2 again.

(yolov8) spypiggy@spypiggy-NX:~/src/yolov8$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.7.0'
>>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.7.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /home/spypiggy/src/opencv_contrib-4.7.0/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2023-03-12T16:05:37Z
    Host:                        Linux 5.10.104-tegra aarch64
    CMake:                       3.16.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 9.4.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/usr/lib/aarch64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cvv java julia matlab ovis python2 sfm ts viz
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           GTK2
    GTK+:                        YES (ver 2.24.32)
      GThread :                  YES (ver 2.64.6)
      GtkGlExt:                  NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.54.100)
      avformat:                  YES (58.29.100)
      avutil:                    YES (56.31.100)
      swscale:                   YES (5.5.100)
      avresample:                NO
    GStreamer:                   YES (1.16.3)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.7)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.19.1)

  NVIDIA CUDA:                   YES (ver 11.4, CUFFT CUBLAS)
    NVIDIA GPU arch:             72
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.6.0)

  Python 3:
    Interpreter:                 /home/spypiggy/anaconda3/envs/yolov8/bin/python (ver 3.8.16)
    Libraries:                   /home/spypiggy/anaconda3/envs/yolov8/lib/python3.8 (ver 3.8.16)
    numpy:                       /home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/numpy/core/include (ver 1.23.5)
    install path:                /home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/python-3.8

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /home/spypiggy/anaconda3/envs/yolov8
-----------------------------------------------------------------

You can see that it is neatly built. The red text shows that ffmpeg is available for video processing and NVidia CUDA GPU is available.

And finally, you can see that Python 3 is linked to the Anaconda virtual environment Python we wanted.

Wrapping up

Because Xavier NX can add NVME SSDs, it can configure enough storage space compared to Jetson Nano. It is advantageous to use the Anaconda virtual environment to fully utilize the storage secured using SSD. After creating multiple virtual environments on SSDs, you can install and use ML frameworks such as PyTorch, Tensorflow, and YOLO in the desired virtual environment.

However, OpenCV provided by Anaconda excludes ffmpeg from video processing. And CUDA acceleration is also not supported. To solve this problem, it is recommended to build an OpenCV that can be used in Python in an anaconda virtual environment.

Please refer to the above and build your own OpenCV for Anaconda optimized for Xavier NX.

NVIDIA Jetson and Raspberry Pi

2023년 3월 13일 월요일

Installing the Latest Version of OpenCV (ver 4.7) on Xavier NX Anaconda Enviromment(JetPack 5.1)

Build the latest version of OpenCV in Anaconda

Prerequisites

Creating a Virtual Environment for OpenCV Installation

libffi patch

Wrapping up

댓글 없음:

댓글 쓰기