JetPack 4.5 - OpenCV 4.1 vs. OpenCV 4.5

 Jetpack 4.5 is provided with OpenCV 4.1.1 installed. OpenCV 4.1.1 is a version provided from Jetpack 4.3, released in April 2020. The current (2021.02) OpenCV latest version is 4.5.1. Let's see how to upgrade OpenCV 4.1.1 to 4.5.1 in JetPack 4.5.. And we will see if there is a need to upgrade from OpenCV 4.1.1 to 4.5.1.

OpenCV 4.1.1

First, let's look at the information of OpenCV installed in JetPack 4.5.

root@spypiggy-nano:~# python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.1.1 =====================================
  Version control:               4.1.1-2-gd5a58aa75

  Platform:
    Timestamp:                   2019-12-13T17:25:11Z
    Host:                        Linux 4.9.140-tegra aarch64
    CMake:                       3.10.2
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++ Compiler:                /usr/bin/c++  (ver 7.4.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections
    Linker flags (Debug):        -Wl,--gc-sections
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python2 python3 stitching ts video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java js
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:
    GTK+:                        YES (ver 2.24.32)
      GThread :                  YES (ver 2.56.4)
      GtkGlExt:                  NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
    JPEG 2000:                   build (ver 1.900.1)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    FFMPEG:                      YES
      avcodec:                   YES (57.107.100)
      avformat:                  YES (57.83.100)
      avutil:                    YES (55.78.100)
      swscale:                   YES (4.8.100)
      avresample:                NO
    GStreamer:                   YES (1.14.5)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            TBB (ver 2017.0 interface 9107)

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.4)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.15)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.15+)
    numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python2.7/dist-packages/cv2/python-2.7

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.6.9)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python3.6/dist-packages/cv2/python-3.6

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr
-----------------------------------------------------------------

The above is summarized as follows.

Item	Description	Etc
Python support	both Python 2.7.15 and 3.6.9
C/C++ support	yes
Java support	No
world module	No	Ability to build multiple modules in opencv into one world file
Platform	Linux 4.9.140-tegra aarch64, CMake Ver:3.10.2
CMake version	Linux 4.9.140-tegra aarch64
GUI support	GTK+
Media	ZLib, JPEG, WEBP, PNG, TIFF, JPEG 2000, HDR, SUNRASTER, PXM, PFM
Video	FFMPEG, GStreamer, v4l/v4l2
Third Party	Eigen, Custom HAL, Protobuf

If you look carefully at the above, there is something strange. There is no information about CUDA support. All Jetson series are equipped with NVidia GPU, and JetPack supports CUDA library to use this GPU. However, there is no information on CUDA support in OpenCV.

Let's check whether CUDA is supported once again.

root@spypiggy-nano:~# python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.cuda.getCudaEnabledDeviceCount())
0

It can be confirmed once again that CUDA is not supported. So why didn't the NVidia JetPack team support CUDA from OpenCV?

The reason for this can be found in OpenCV 4.1.1 in Jetpack 4.3 is built without CUDA (GPU support).

If you look at dusty_nv's reply on this page, he says:

There are issues with the CUDA code in OpenCV (the CUDA code is actually in opencv_contrib) which result in tests not passing. You are free to build with it enabled if you like, but we are not able to support it.

That is, it was not applied because of an error that occurred during the build process. If it is possible to build without errors, you can build it yourself.

Is CUDA Support Required in OpenCV?

If you mainly use Tensorflow, PyTorch, or TensorRT, there is no reason to use CUDA in OpenCV. This is because these machine learning frameworks support CUDA. Perhaps you will simply use OpenCV to open images, resize them, and reformat images. OpenCV rarely requires CUDA to do this.

However, if you want to load a machine learning network model directly from OpenCV, input an inference image and get the result, you must use OpenCV dnn module that supports CUDA. In addition, the ability to directly load models of machine learning frameworks such as Tensorflow and PyTorch is supported from OpenCV 4.2. So you must upgrade OpenCV 4.1.1 to 4.2.X or to higher version.

And in OpenCV, images are objects of class Mat. If you are using C++, use the cv::Mat class, and if you are using Python, use the cv2.Mat class.

In addition, OpenCV provides additional cv::gpu::GpuMat and cv2.cuda_GpuMat classes that operate in CUDA memory so that objects of the Mat class can be processed quickly in CUDA. 

Therefore, if you want to use the cv::gpu::GpuMat and cv2.cuda_GpuMat classes, you need to upgrade to OpenCV which also supports CUDA.

A brief introduction to OpenCV CUDA.

The following is a test after upgrading OpenCV to 4.5.1 on Jetson Nano's JetPack 4.5. OpenCV 4.1.1 provided by Jetpack 4.5 does not support CUDA, so OpenCV has been upgraded to 4.5.1 for performance comparison. The upgrade method will be explained later.

OpenCV's CUDA module is still being improved. Therefore, the functions of the existing CPU mode are not 100% usable in CUDA. Since some functions are not implemented, in some cases you may need to use functions that run on the CPU.

A description of the development background of OpenCV CUDA is available at https://opencv.org/platforms/cuda/. This page provides a table comparing the performance of Intel i5 CPU and Tesla C2050 CUDA. In Primitive Image Processing, it can be seen that the performance difference is up to 30 times. However, there is not a difference in performance in all operations using OpenCV. There are many performance differences in image processing filters that require repetitive numerical calculations. There are cases in which there is little difference in performance in tasks such as changing the image format, or if CUDA is used, the performance is inferior.

<Performance comparison>

How to use CUDA in OpenCV requires the following three steps.

1. upload: Move data to CUDA memory
2. processing: Use cuda function to proceed with the desired operation
3. download: Move the finished data to be used again in CPU

Let's see how to use CUDA in OpenCV with the following simple example code. In this example, the upload process is performed for CUDA processing, but the last download process is omitted. If you want to save the processed video frame result, you can save it through the download process.

import cv2
import time
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--gpu', type=int, default = 0)
opt = parser.parse_args()

video = './see.mp4'
vsize = (1024,768)
process_time = 0
num_frames = 0
def cuda_process():
    global process_time, num_frames
    count = 1
    vod = cv2.VideoCapture(video)
    num_frames = vod.get(cv2.CAP_PROP_FRAME_COUNT)
    
    ret, frame = vod.read()
    # set scale of resized image
    gpu_frame = cv2.cuda_GpuMat()
    while ret:
        s = time.time()
        gpu_frame.upload(frame)
        resized = cv2.cuda.resize(gpu_frame, vsize)
        luv = cv2.cuda.cvtColor(resized, cv2.COLOR_BGR2LUV)
        hsv = cv2.cuda.cvtColor(resized, cv2.COLOR_BGR2HSV)
        gray = cv2.cuda.cvtColor(resized, cv2.COLOR_BGR2GRAY)
        process_time += (time.time() - s)
        ret, frame = vod.read()
        #print('frame :%d'%count)
        count += 1
    vod.release()
   
def cpu_process():    
    global process_time, num_frames
    count = 1
    vod = cv2.VideoCapture(video)
    num_frames = vod.get(cv2.CAP_PROP_FRAME_COUNT)
    ret, frame = vod.read()
    while ret:
        s = time.time()
        resized = cv2.resize(frame, vsize)
        luv = cv2.cvtColor(resized, cv2.COLOR_BGR2LUV)
        hsv = cv2.cvtColor(resized, cv2.COLOR_BGR2HSV)
        gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
        process_time += (time.time() - s)
        ret, frame = vod.read()
        #print('frame :%d'%count)
        count += 1
    vod.release()
 

if __name__ == "__main__":
    start = time.time()
    if opt.gpu:
        cuda_process()
    else:
        cpu_process()
    end = time.time()
    print('processing time:%6.2f'%(process_time))
    print('total frame[%d] time[%6.2f]'%(num_frames, end - start))

<compare.py>

Let's run the example.

root@spypiggy-nano:/usr/local/src/study# python3 compare.py --gpu=0
processing time: 12.90
total frame[365] time[ 30.55]
root@spypiggy-nano:/usr/local/src/study# python3 compare.py --gpu=1
processing time: 15.61
total frame[365] time[ 33.11]

Contrary to your expectations, it took a little longer to process if you used CUDA. Perhaps if you added the download process to save the video frame, the time difference could have been a bit more. Even though CUDA was used like this, the reason it took more time is probably because the functions we used such as resize and cvtColor did not show a big difference in performance between CUDA and non-CUDA. Rather, the time difference would have occurred because the upload process was added to the CUDA memory.

However, there are some functions that can benefit greatly when using CUDA. The following is the second example. This example uses the goodFeaturesToTrack filter. This filter is a filter that finds the coordinates of an edge in an image.

<image with goodFeaturesToTrack filter applied>

import cv2
import time
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--gpu', type=int, default = 0)
opt = parser.parse_args()
scale = 0.5
video = './see.mp4'
vsize = (1024,768)
process_time = 0
num_frames = 0
def cuda_process():
    global process_time, num_frames
    count = 1
    vod = cv2.VideoCapture(video)
    num_frames = vod.get(cv2.CAP_PROP_FRAME_COUNT)
    ret, frame = vod.read()
    # set scale of resized image
    gpu_frame = cv2.cuda_GpuMat()
    detector = cv2.cuda.createCannyEdgeDetector(100, 200)
    trackdetector = cv2.cuda.createGoodFeaturesToTrackDetector(cv2.CV_8UC1)
    while ret:
        gpu_frame.upload(frame)
        gpu_resized = cv2.cuda.resize(gpu_frame, vsize)
        gpu_gray = cv2.cuda.cvtColor(gpu_resized, cv2.COLOR_BGR2GRAY)
        s = time.time()
        #canny = detector.detect(gpu_gray)
        gpu_coners = trackdetector.detect(gpu_gray)
        process_time += (time.time() - s)
        #cpu_canny = canny.download()
        cpu_coners = gpu_coners.download()
        '''
        cv2.imshow("original", gpu_resized.download())
        cv2.imshow("canny", cpu_canny)
        k = cv2.waitKey(1)
        if k == 27:
            break        
        '''    
        ret, frame = vod.read()
        #print('frame :%d'%count)
        count += 1
    vod.release()
   
def cpu_process():    
    global process_time, num_frames
    count = 1
    vod = cv2.VideoCapture(video)
    num_frames = vod.get(cv2.CAP_PROP_FRAME_COUNT)
    ret, frame = vod.read()
    while ret:
        resized = cv2.resize(frame, vsize)
        gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
        s = time.time()
        #canny = cv2.Canny(gray, 100,200)
        corners = cv2.goodFeaturesToTrack(gray,25,0.01,10)
        process_time += (time.time() - s)
        '''
        cv2.imshow("original", resized)
        cv2.imshow("canny", canny)
        k = cv2.waitKey(1)
        if k == 27:
            break        
        '''    
        ret, frame = vod.read()
        #print('frame :%d'%count)
        count += 1
    vod.release()
 

if __name__ == "__main__":
    start = time.time()
    if opt.gpu:
        cuda_process()
    else:
        cpu_process()
    end = time.time()
    cv2.destroyAllWindows()
    print('processing time:%6.2f'%(process_time))
    print('total frame[%d] time[%6.2f]'%(num_frames, end - start))
 

<compare2.py>

Let's run the second example.

root@spypiggy-nano:/usr/local/src/study# python3 compare2.py --gpu=0
processing time: 17.78
total frame[365] time[ 41.17]
root@spypiggy-nano:/usr/local/src/study# python3 compare2.py --gpu=1
processing time:  8.92
total frame[365] time[ 34.45]

Unlike the previous example, you can see that there is a significant performance improvement. In filter processing time, processing time has improved a lot, but it can be seen that some performance loss occurs because upload and download process between CUDA memory and CPU memory is required in total processing time. As you can see from the examples above, using CUDA does not necessarily improve the processing speed. Depending on the OpenCV function you use and the image size, the results will vary a lot.

Installing OpenCV 4.5.1

At this point, I will install the latest version of OpenCV, 4.5.1 which supports CUDA, into JetPack 4.5.

Increase swap memory(Jetson Nano Users)

If you are using Xavier NX, there is no need to increase the swap memory. When you build a large software packages like openCV, you may experience an out of memory phenomenon. Increasing the swap file size can prevent this malfunction.

git clone https://github.com/JetsonHacksNano/installSwapfile cd installSwapfile ./installSwapfile.sh

Above script file will increase 6GB swap files. You can change the swap file size by modifying the scripts. If you want to uninstall the swap setting, open the fstab file and delete the swap file line and reboot. 

 The description from now on can also be applied to Jetson Nano, Xavier NX and TX2 using JetPack 4.5.

 One thing to note is -D CUDA_ARCH_BIN="7.2", which specifies CUDA Architecture in the process of creating Makefile using cmake. Nano, TX2, and Xavier have different values. These values are described in JetsonNano-Useful tips before using Nano. 

Compute capability (version)	Micro-architecture	GPUs	Tegra,Jetson
3.2	Kepler	GK20A	Tegra K1, Jetson TK1
5.3	Maxwell	GM20B	Tegra X1, Jetson TX1, Jetson Nano
6.2	Pascal	GP10B	Tegra X2, Jetson TX2
7.2	Volta	GV10B	Tegra Xavier, Jetson Xavier NX, Jetson AGX Xavier

<table from JetsonNano-Useful tips before using Nano>

 

 The following is a script file that can be easily installed if you use Jetson Nano.

#!/bin/bash
#
# Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

# spyjetson(https://spyjetson.blogspot.com/) modified

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <Install Folder>"
    exit
fi
folder="$1"
user="nvidia"
passwd="nvidia"

echo "** Remove OpenCV4.1 first"
sudo sudo apt-get purge *libopencv*

echo "** Install requirement"
sudo apt-get update
sudo apt-get install -y build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
sudo apt-get install -y python2.7-dev python3-dev python-dev python-numpy python3-numpy
sudo apt-get install -y libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
sudo apt-get install -y libv4l-dev v4l-utils qv4l2 v4l2ucp
sudo apt-get install -y curl
sudo apt-get update

echo "** Download opencv-4.5.1"
cd $folder
curl -L https://github.com/opencv/opencv/archive/4.5.1.zip -o opencv-4.5.1.zip
curl -L https://github.com/opencv/opencv_contrib/archive/4.5.1.zip -o opencv_contrib-4.5.1.zip
unzip opencv-4.5.1.zip 
unzip opencv_contrib-4.5.1.zip 
cd opencv-4.5.1/

echo "** Building..."
mkdir release
cd release/
cmake -D WITH_CUDA=ON -D ENABLE_PRECOMPILED_HEADERS=OFF  -D CUDA_ARCH_BIN="5.3" -D CUDA_ARCH_PTX="" -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.5.1/modules -D WITH_GSTREAMER=ON -D WITH_LIBV4L=ON -D BUILD_opencv_python2=ON -D BUILD_opencv_python3=ON -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_EXAMPLES=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j4
sudo make install

echo "** Install opencv-4.5.1 successfully"
echo "** Bye :)"

<opencv4.5_jetsonnano.sh>

I put the script file on my github. And since the script that changed CUDA_ARCH_BIN value for Jetson Nano and TX2 is also uploaded, you can use the script suitable for the Jetson series you are using.

Now run the script.

root@spypiggy-nano:/usr/local/src# ./opencv4.5_jetsonnano.sh ./

The script file needs a parameter for source code download directory. The script file takes a lot of time to build OpenCV 4.5.1. If you are not the root user, you may be asked to enter a password to obtain sudo privileges in the middle.

root@spypiggy-nano:/usr/local/src# python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.5.1 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /usr/local/src/opencv_contrib-4.5.1/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2021-02-08T10:22:39Z
    Host:                        Linux 4.9.201-tegra aarch64
    CMake:                       3.10.2
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 7.5.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppicom nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda/lib64 -L/usr/lib/aarch64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cvv hdf java julia matlab ovis sfm ts viz
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:
    GTK+:                        YES (ver 2.24.32)
      GThread :                  YES (ver 2.56.4)
      GtkGlExt:                  NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        libjpeg-turbo (ver 2.0.6-62)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 2.3.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (57.107.100)
      avformat:                  YES (57.83.100)
      avutil:                    YES (55.78.100)
      swscale:                   YES (4.8.100)
      avresample:                NO
    GStreamer:                   YES (1.14.5)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.4)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 10.2, CUFFT CUBLAS)
    NVIDIA GPU arch:             53
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.0.0)

  OpenCL:                        YES (no extra features)
    Include path:                /usr/local/src/opencv-4.5.1/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.17)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.17)
    numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python2.7/dist-packages/cv2/python-2.7

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.6.9)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
    install path:                lib/python3.6/dist-packages/cv2/python-3.6

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

You can see that the newly built OpenCV 4.5.1 supports CUDA.

Wrapping Up

 It is not difficult to upgrade OpenCV to OpenCV 4.5.1 on JetPack 4.5. If you need to use CUDA or use the dnn module in OpenCV, it is recommended to upgrade to 4.5.1. However, if you mainly use machine learning frameworks such as Tensorflow, PyTorch and use OpenCV for simple purposes for image input/output, it is sufficient to use version 4.1.1 without upgrading.

Xavier NX - NVIDIA AI IOT - Fast debugging trt_pose(Human Pose estimation using TensorRT) using VSCode

 This article assumes that you have read the following two blogs. So, if you haven't seen the two blogs below, I recommend reading them in advance.

• Xavier NX - Remote Python Development using VSCode
• Xavier NX - NVIDIA AI IOT - Human Pose estimation using TensorRT

Although this article explains what works on Xavier NX, if you use JetPack 4.4 or higher, it will work on TX2, Nano as well. 

In "Xavier NX-Remote Python Development using VSCode" I introduced you how to debug Xavier NX's python code on a remote computer.

It is also possible to debug by installing VSCode directly on Xavier NX or Nano. However, I do most of the work using the Jetson series remotely using ssh on my WIndows laptop. While this way of working is familiar to me, debugging Python is inconvenient. Simple debugging can be done using the print function. However, the variable data of the Python code related to AI needs to check the value of one variable, not just the value of a single variable, but a large multidimensional data moment by moment. Because of this, there is a limit to using the print function.

In this article, I will look at how to remotely debug the Python code introduced in "Xavier NX-NVIDIA AI IOT-Human Pose estimation using TensorRT" directly using VSCode.

Run VSCode and connect to Xavier NX

The following figure is the screen described in "Xavier NX-Remote Python Development using VSCode". After reading the contents of the article "Xavier NX-Remote Python Development using VSCode", please follow this screen.

<Screen connected to remote Xavier NX using VSCode>

Let's run the example of "Xavier NX-NVIDIA AI IOT-Human Pose estimation using TensorRT" in the VSCode terminal to see if it works.

The "Unable to init server" message is a GUI related warning caused by using OpenCV and PIL. I can ignore this warning because I didn't put the code to print the image file to the screen in the example code. If you look at the picture above, you can see that it works normally after warning.

Debug trt_pose detect_image2.py 

Now let's debug the TensorRT Pose Estimation example.

First, open the detect_image2.py file, then click the Debug button on the left side of VSCode. Then the "create a launch.json file" screen appears as shown in the following figure.

<creating launch.json file>

Click "create a launch.json file" to display the Debug Configuration option. Here you select the option to debug the currently open Python file.

The following launch.json file will appear.

Since our Python code uses runtime parameters, we add an args entry to the launch.json file. And it's a good idea to specify the working directory where the code will run. The reason is that you open the human_pose.json file in the detect_image2.py file, and if you don't use an absolute path in your Python code, you need to specify the working directory to avoid an error.

<launch.json>

Now is the time to finally debug.

Set a breakpoint in the code you want to examine. To set it up, simply click next to the line number. Then a red dot appears.

<set breakpoint>

Once you have set the breakpoint, press the green triangle button to run the code.

If everything is OK, code execution will stop at that line as shown in the following figure. And the current values of variables are shown in real time on the left pane.

You can debug while moving in units of lines or functions by using the debugging button at the top of the VSCode screen. This is a great feature that can reduce your debugging time!

If you move the mouse cursor to the variable you want to investigate while proceeding the code line by line, you can check the value of the current variable in real time.

<Real-time variable value check using mouse cursor>

Measuring the confidence of trt_pose keypoints

Recently, there was someone who asked me if I could check the accuracy of the keypoints detected using trt_pose. Let's check if we can find the confidence value of the keypoint using the debugging function introduced in this article.

Since the confidence value will vary from keypoint to keypoint, if confidence value exists it will have to exist with the keypoint coordinates.

<keypoint value check>

Keypoint values are taken from the peaks variable. Examining these peaks values reveals that there are only two values. This value is a coordinate value indicating the location of the key point. Therefore, it can be seen that the confidence value of the keypoint is not currently provided by trt_pose.

Providing keypoint extraction threshold in trt_pose

The confidence value of the keypoint is unknown, but the threshold value can be provided when extracting the keypoints.

<ParseObjects check>

Examining parse_objects shows the values of cmap_threshold and link_threshold. These values are the threshold values used when extracting key points. Adjusting these values makes the keypoint extraction slightly different.

The default value of these thresholds is 0.1. You can easily see it by looking at the source code (https://github.com/NVIDIA-AI-IOT/trt_pose/blob/master/trt_pose/parse_objects.py).

Change these thresholds to 0.3, then test. The way to change the threshold is to provide this value as a parameter in the code that creates parse_objects.

Let's change the code as above and compare the created image files.

                          <threshold 0.1>                                                          <threshold 0.3>

At the threshold of 0.3, it can be seen that the keypoint is detected a little more strictly.

Wrapping up

I used VSCode to debug the Jetson series' trt_pose example on a remote computer. The method introduced in this article can be useful for all Python development such as tensorflow, pytorch, opencv, etc.

Xavier NX - Remote Python Development using VSCode

 What kind of development tools do you use?
Do you connect the keyboard and monitor directly to Jetson series, or do you develop the solution from a remote computer?

I develop it through remote access on my Windows 10 laptop unless it's a special occasion.
In addition, I use development tools sometimes cli editors such as vi, nano, etc., but use Notepad++ or VSCode more frequently.
These GUI development tools provide an expansion pack for editing and managing files on remote computers. In particular, VSCode, Microsoft's free editing framework, allows you to modify source code as well as remote GUI debugging. I think many people felt the need for debugging while developing machine learning tools such as OpenCV, Tensorflow, PyTroch, and TensorRT. I also often felt that I needed the ability to test and debug while editing the source code on a remote computer. I think VSCode is probably the best development tool for this purpose.

Install VSCode on your laptop

VSCode can be installed on Windows, Mac, and Linux. It can even be installed in Jetson Nano. I use Windows 10, but Mac and Linux users will be able to use it the same way.

• Laptop with VSCode (Windows or Mac)
  

Install VSCode Extensions on your laptop

Install Python Extensions on your laptop

Run VSCode first, then press the Extensions search button.

 

 <Run VSCode and click the Extensions button>

 

 Then search for Python and install it. Python extension is essential for developing Python in VSCode.

 

 <Install Python Extension>

Install Remote Extensions on your laptop

Then install 'Remote Development Extension'  to edit the source code of the remote Jetson series. When Microsoft first introduced the Remote Development Extension, it supported only the ssh key exchange method for remote access. I once posted on October 17, 2019 under the title of Remote Python Debugging with VSCode. This article explains how to connect to a remote computer using the ssh key exchange method. This method is still valid. However, not only ssh key exchange but also id/ password authentication is possible now.

Tips : In October 17, 2019 writings, VSCode Insider was used. VSCode Insider is an experimental version of the latest features. We used VSCode Insider because it was only a short time after the Remote Development Extensions came out. However, the Remote Development Extensions are now fully stabilized and can be used in normal VSCodes, not in the Insider version.

Search for and install Remote Development. This extension installs Remote -WSL, Remote-Contains, and Remote-SSH.

 <Install Remote Development Extension>

Setup Remote Jetson SSH connection

On the October 17, 2019 blog, the only way to access a remote server using ssh is to use an ssh key. However, this method has several drawbacks.

 If the remote server information is changed, a new ssh key must be created. I mainly work on the Raspberry Pi and NVidia Jetson series. These single board computers use SD cards as storage space. Therefore, the SD card image is frequently replaced. When the SD card is replaced, the previously created ssh keys are no longer valid. Therefore, id/password authentication is often convenient.

Since VSCode is now able to access ssh in the id/password method, this disadvantage is eliminated.  Now, the config file for ssh connection in VSCode was created as follows.

First, boot the Jetson Xavier to make it accessible.

 Old Method (ssh key exchange)

Run VSCode and press Ctrl + Shift + P to search for the extension command. Then select Remote-SSH: Connect to Host ... Then select C: \ Users \ Users.ssh \ config. And add the following line: Modify the IP, User, and Private Key routes as appropriate. Refer to https://tipspiggy.blogspot.com/2019/10/remote-python-debugging-with-vscode.html for key generation method.

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host JetsonXavierNX
    HostName 192.168.11.96
    User spypiggy
    IdentityFile ~/.ssh/jumpbox

For instructions on how to create a Jumpbox file, see https://tipspiggy.blogspot.com/2019/10/remote-python-debugging-with-vscode.html.

 

New Method (id/password)

Run VSCode and press Ctrl + Shift + P to search for the extension command. Then select Remote-SSH: Connect to Host ... Then select C: \ Users \ Users.ssh \ config. And add the following line: Modify the IP, User. You no longer need to put an IdentityFile entry.

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host JetsonXavierNX
    HostName 192.168.11.96
    User spypiggy

 

Now let's try new method.

press Ctrl + Shift + P to search for the "Remote-SSH:Connect to host ..." command. Then select your host that you just configured. 

 Then you will probably be prompted for the user's password. If you enter the correct password, you can connect to the remote host.

<Remote Jetson connected successfully>

 If you press the Explorer button and then the Open Folder button, you can specify the working directory of the remote Jetson Xavier. 

 

Install extensions on the remote host

 Remote debugging requires that you install an extension that enables remote debugging on the remote computer . Press the Extensions button or Ctrl + Shift + P and search Python again. Earlier we installed the Python module. This time we see a new "Install in SSH: JetsonXavierNX" button. This means installing the VSCode Python extension on the remote host Xavier NX. Clicking this button installs a remote extension for Python debugging on the Remote Xavier NX.

Tips : Installation of Python extensions on remote Xavier NX requires a Remote-SSH connection first..

 

Install Python Extensions on remote Xavier NX

If you have never installed the VSCode Python extension on a remote Xavier NX, the following screen will appear:

<Python extension installation on the remote host>

 

 If you select the Python extension in the VSCode window connected to the remote host, options for the remote host appear. Click "Install in SSH: JetsonXavierNX" to install the Python extension on the remote host.

 At the end of the installation, you can verify that the Python extension is installed on the remote Xavier NX as follows:

  <VSCode Extensions installed on the Remote Xavier NX>

 After installation, reload the VSCode workspace. You are now ready for Python remote debugging.

 Perhaps you will be asked to select the Python version to be used by the remote host as follows:

Do not use Python 2. I am using Python 3 virtual environment in Xavier NX in ~/python directory. Therefore, you will choose the last ~/ python/bin.

<Xavier NX python version you can select>

Then select New Terminal from the terminal menu.

When the terminal opens, you can see that the Python virtual environment is automatically run.

Code completion with IntelliSense

Most users who are familiar with IDE environments such as MS Visual Studio love to use code completion features. Code auto-completion not only speeds up development, but also greatly improves productivity by dramatically reducing the amount of typographical errors. Although the Linux development environment has some of these code autocompletion features, unlike the IDE on Windows and Mac, they often have limited functionality or require a paid product. In particular, remote development environments rarely provide this functionality. Using a remote control tool such as VNC is far from the remote development environment described in this article. 

 

After installing the remote Python debugging module, VSCode can now write Python code that works on the remote computer using autocomplete. It is a new and powerful feature that you will never experience using the VI editor on a SSH terminal. 

 

To use the code autocomplete feature in OpenCV, ...

 I'm using Python virtual environment on Xavier NX. Packages such as Tensorflow installed in a virtual environment do not have problems using autocomplete feature. But OpenCV has some problems.
JetPack 4.4 is provided with OpenCV 4.1 as standard. The location where this package is installed is as follows.

(python) spypiggy@XavierNX:~/src$ python
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
cv>>> cv2.__file__
'/usr/lib/python3.6/dist-packages/cv2/python-3.6/cv2.cpython-36m-aarch64-linux-gnu.so' 

>>> import tensorflow as tf
2020-09-02 10:30:14.846743: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
>>> tf.__file__
'/home/spypiggy/python/lib/python3.6/site-packages/tensorflow/__init__.py' 

Since Tensorflow was installed in a virtual environment, packages exist in /home/spypyggy/python/lib/python3.6/site-packages.
However, the OpenCV provided by JetPack4.4 has an installation location of /usr/lib/python3.6/dist-packages/cv2/python-3.6/.

To apply autocomplete to packages installed outside of a virtual environment, some of the configuration files in VSCode need to be modified. The .vscode directory is created in the remote host's connection directory, which is accessed using Remote-SSH in VSCode, as illustrated. And the settings.json file is automatically created.

<settings.json file in the .vscode directory>

To enable IntelliSense for packages that are installed in other, non-standard locations, add those locations to the python.autoComplete.extraPaths collection in the settings file (the default collection is empty).  Modify json file as follows. As you can see from the json file, Python.autoComplete.extraPaths can add multiple paths.

{
    "python.pythonPath": "/home/spypiggy/python/bin/python",
    "python.autoComplete.extraPaths": [
        "/usr/lib/python3.6/dist-packages/cv2/python-3.6"]
}

If you run VSCode again and test it, you can see that the automatic code completion function works in OpenCV.

<Extensions of code completion functions using python.autoComplete.extraPaths application>

 Details can be found in the Editing Python in Visual Studio Code.

Tips : The above is the case when Python virtual environment is used. If you do not use a virtual environment and use /usr/bin/python3 as the default Python interpreter, you do not need to modify the settings.json file described earlier. However, there will be times when it will be useful to know the autocomplete features of VSCode through settings.json file modifications.

 Remote Debugging

You can now set breakpoints by hovering your mouse next to the left number in the Python source code. Create a breakpoint at the desired location. Then press the debugger button and then the start debugger button. When asked what to debug, select Python File. This will automatically run the Python file in the current window. The program automatically stops at the breakpoint you specify. At this point, you can check the variable values ​​with the mouse and monitor the variable values ​​step by step using the left window.

<Debugging OpenCV image processing with VSCode>

 Wrapping Up

The Python debugging function provided by VSCode is fantastic. Especially on your laptop, you can connect the remote Jetson series to do Python debugging using a GUI.
In particular, it can dramatically improve productivity because it can check the value of objects at any time while executing machine learning Python code.
VSCode's remote Python code autocomplete feature and GUI debugging are available on both Jetson Nano, TX2, and Xavier NX with the JetPack series installed.

 

 

 

Xavier NX - OpenCV 4.5

 Xavier NX only works with JetPack 4.4 or later. And JetPack 4.4 is provided with OpenCV 4.1.1 installed. In most cases, there is no problem with using OpenCV 4.1.1. However, from OpenCV 4.2+, Super Resolution function is provided as C/C++ API. And from 4.3, it provides Python API. And finally, in 4.4, CUDA GPU acceleration is available. Therefore, it is recommended to use version 4.4 or higher to fully use the Super Resolution function provided by OpenCV. Therefore, to use OpenCV's SuperResolution, you need to delete the OpenCV 4.1.1 version of JetPack 4.4 and install 4.4 newly.

And from OpenCV 4.2, the dnn module started supporting Nvidia GPUs. In previous versions, only the CPU was available. The Jetson series has NVidia GPUs, so if you are using OpenCV's dnn, it is recommended to upgrade to version 4.2 or higher. OpenCV version 4.2 or higher uses NVidia GPU to speed up the inference.

If you are not interested in OpenCV's SuperResolution feature or not using OpenCV dnn module, you do not necessarily need to upgrade to OpenCV 4.4 or 4.5.

Increase swap memory

When you build a large software packages like OpenCV on Jetson Nano, you may experience an out of memory phenomenon. Increasing the swap file size can prevent this malfunction. If you are using Jetson Xavier NX, you don't have to do this.

git clone https://github.com/JetsonHacksNano/installSwapfile cd installSwapfile ./installSwapfile.sh

Above script file will increase 6GB swap files. You can change the swap file size by modifying the scripts. If you want to uninstall the swap setting, open the fstab file and delete the swap file line and reboot. 

Installing OpenCV 4.5

 The description from now on can also be applied to Jetson Nano and Jetson TX2 using JetPack 4.5.

 One thing to note is -D CUDA_ARCH_BIN="7.2", which specifies CUDA Architecture in the process of creating Makefile using cmake. Nano, TX2, and Xavier have different values. These values are described in JetsonNano-Useful tips before using Nano. 

Compute capability (version)	Micro-architecture	GPUs	Tegra,Jetson
3.2	Kepler	GK20A	Tegra K1, Jetson TK1
5.3	Maxwell	GM20B	Tegra X1, Jetson TX1, Jetson Nano
6.2	Pascal	GP10B	Tegra X2, Jetson TX2
7.2	Volta	GV10B	Tegra Xavier, Jetson Xavier NX, Jetson AGX Xavier

<table from JetsonNano-Useful tips before using Nano>

 

 The following is a script file that can be easily installed.

 

 

#!/bin/bash
#
# Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
#

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <Install Folder>"
    exit
fi
folder="$1"
user="nvidia"
passwd="nvidia"

echo "** Remove OpenCV4.1 first"
sudo apt-get purge *libopencv*

echo "** Install requirement"
sudo apt-get update
sudo apt-get install -y build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
sudo apt-get install -y python2.7-dev python3.6-dev python-dev python-numpy python3-numpy
sudo apt-get install -y libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
sudo apt-get install -y libv4l-dev v4l-utils qv4l2 v4l2ucp
sudo apt-get install -y curl
sudo apt-get update

echo "** Download opencv-4.5.1"
cd $folder
curl -L https://github.com/opencv/opencv/archive/4.5.1.zip -o opencv-4.5.1.zip
curl -L https://github.com/opencv/opencv_contrib/archive/4.5.1.zip -o opencv_contrib-4.5.1.zip
unzip opencv-4.5.1.zip 
unzip opencv_contrib-4.5.1.zip 
cd opencv-4.5.1/

echo "** Building..."
mkdir release
cd release/
cmake -D WITH_CUDA=ON -D ENABLE_PRECOMPILED_HEADERS=OFF  -D CUDA_ARCH_BIN="7.2" -D CUDA_ARCH_PTX="" -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.4.0/modules -D WITH_GSTREAMER=ON -D WITH_LIBV4L=ON -D BUILD_opencv_python2=ON -D BUILD_opencv_python3=ON -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_EXAMPLES=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j6
sudo make install

echo "** Install opencv-4.5.1 successfully"
echo "** Bye :)"

 <opencv4.5_xavier_nx.sh>

I put the script file on my github. And since the script that changed CUDA_ARCH_BIN value for Jetson Nano and TX2 is also uploaded, you can use the script suitable for the Jetson series you are using.

Now run the script

spypiggy@XavierNX:~/src$ ./opencv4.5_xavier_nx.sh /home/spypiggy/src/

The script file needs a parameter for source code download directory. The script file takes a lot of time to build OpenCV 4.5. If you are not the root user, you may be asked to enter a password to obtain sudo privileges in the middle.

spypiggy@XavierNX:~/src$ python3
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> from cv2 import dnn_superres
>>> cv2.__version__
'4.5.1'

If the installation is finished without error, you can check OpenCV 4.5 as follows. It can be seen that dnn_superres for Super Resolution is also normally imported.

Wrapping Up

 It is not difficult to upgrade OpenCV to OpenCV 4.5 on JetPack 4.5. Upgrade is possible only with the script file introduced earlier. If you want to use the new features of Ver 4.5 or need to use CUDA, you can upgrade OpenCV to 4.5.

To install OpenCV 4.5, refer to "JetPack 4.5-OpenCV 4.1 vs. OpenCV 4.5".