Xavier NX - YOLOv8 Object Detection (JetPack 5.1)

 

In the last two posts, I explained the process of installing Jetpack 5.1 on Xavier NX and installing YOLOv8.

Prerequsites

• Xavier NX (JetPack 5.1) - Effective Development environment using Anaconda
• Xavier NX - Installing PyTorch, YOLOv8 in Anaconda Virtual Environment (JetPack 5.1)

YOLOv8 is still being updated. Ultralytics, which released VOLOv8, is continuously releasing updated VOLOv8 on github. Therefore, it is recommended that you also update it from time to time.

The update method is simple. Since we installed the ultralytics package using pip in the anaconda virtual environment, you can also use the pip command to update.

pip install ultrslytics --upgrade

YOLOv8 Object Detection Models

VOLOv8 provides pre-trained models. These models were trained using the COCO dataset.

<YOLOv8 pre-trained models>

As shown in the table above, the YOLOv8n model is the lightest model and the YOLOv8x model is the heaviest. The lighter the model, the less memory is used and the processing speed is faster. But the accuracy is poor.

You can collect training data and create your own custom models, but in this article I will use pre-trained models.

COCO Dataset

COCO is a large-scale object detection, segmentation, and captioning dataset. COCO has several features:

• Object segmentation
• Recognition in context
• Superpixel stuff segmentation
• 330K images (>200K labeled)
• 1.5 million object instances
• 80 object categories
• 91 stuff categories
• 5 captions per image
• 250,000 people with keypoints

Many machine learning models are trained using the COCO dataset for performance evaluation. The pre-trained models of YOLOv8 are also models trained using the COCO dataset.

Object Detection of the COCO model is classified into a total of 80.

<COCO model 80 labels>

The YOLOv8 pre-trained model can also find 80 objects according to this classification.

Labeling values can be checked in the coco.yaml file.

YOLOv8 Object Detection

YOLOv8 provides two methods of object detection. The first method is to use the cli command and the second method is to use the Python API.

The document about YOLOv8 object recognition is https://docs.ultralytics.com/tasks/detection/.

CLI command

This is a method using the yolo command provided by VOLOv8. You can use the yolo command to perform the same functions as the Python API, such as learning, validation, and prediction. 

yolo detect predict model=yolov8n.pt source="https://ultralytics.com/images/bus.jpg"  # predict with official model
yolo detect predict model=path/to/best.pt source="https://ultralytics.com/images/bus.jpg"  # predict with custom model

The basic CLI command usage is as follows. We will use the above line because I will be using an pre-trained model.

If you test using ssh without X11 forwarding, it is better to save the result because it is difficult to check the image directly.

(base) spypiggy@spypiggy-NX:~$ conda activate yolov8
(yolov8) spypiggy@spypiggy-NX:~$ yolo detect predict model=yolov8n.pt source="https://ultralytics.com/images/bus.jpg" save=True show=False
Ultralytics YOLOv8.0.51 🚀 Python-3.8.16 torch-1.14.0a0+44dac51c.nv23.02 CUDA:0 (Xavier, 6857MiB)
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs

Downloading https://ultralytics.com/images/bus.jpg to bus.jpg...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 476k/476k [00:00<00:00, 3.94MB/s]
image 1/1 /home/spypiggy/bus.jpg: 640x480 4 persons, 1 bus, 1 stop sign, 86.2ms
Speed: 2.4ms preprocess, 86.2ms inference, 23.2ms postprocess per image at shape (1, 3, 640, 640)
Results saved to runs/detect/predict

You can see that the resulting image is stored in ./runs/detect/predict/bus.jpg. If you open the file, you can see the image like this.

<YOLOv8 detected image - /runs/detect/predict/bus.jpg>

If you test in the GUI environment of Xavier NX, you can directly check the result by changing the show option to True.

Python API

Here's how to use the Python API. Using the Python API, various application programs can be developed. 

The following is the simplest example using the Python API. Presumably all the important predictions are stored in results.

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n.pt")  # load an official model

# Predict with the model
results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image

The results stores as many results as the number of input inferences. Therefore, if only one image is input, it is a list structure with one result value. And the element of the list type is 'ultralytics.yolo.engine.results.Results'.

The following code outputs box coordinates, object classification, confidence, etc. from the detection result.

from ultralytics import YOLO

# Load a model
model = YOLO("yolov8n.pt")  # load an official model
# Predict with the model
results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image

for result in results:
    for box in result.boxes.data:
        print("x1:%f y1:%f  x2[%f] y2[%f] Conf[%f] Label[%f]"%(box[0], box[1], box[2], box[3], box[4], box[5]))

<sample_detect.py>

If you run the code, you can see the box coordinates, confidence, and label index.

(yolov8) spypiggy@spypiggy-NX:~/src/yolov8$ python sample_detect.py

Found https://ultralytics.com/images/bus.jpg locally at bus.jpg
image 1/1 /home/spypiggy/src/yolov8/bus.jpg: 640x480 4 persons, 1 bus, 1 stop sign, 85.5ms
Speed: 2.3ms preprocess, 85.5ms inference, 9.1ms postprocess per image at shape (1, 3, 640, 640)
x1:17.000000 y1:231.000000  x2[801.000000] y2[769.000000] Conf[0.870380] Label[5.000000]
x1:49.000000 y1:399.000000  x2[244.000000] y2[903.000000] Conf[0.868917] Label[0.000000]
x1:670.000000 y1:380.000000  x2[810.000000] y2[875.000000] Conf[0.852670] Label[0.000000]
x1:221.000000 y1:406.000000  x2[345.000000] y2[857.000000] Conf[0.818634] Label[0.000000]
x1:0.000000 y1:255.000000  x2[32.000000] y2[325.000000] Conf[0.347606] Label[11.000000]
x1:0.000000 y1:551.000000  x2[67.000000] y2[874.000000] Conf[0.281894] Label[0.000000]

The most important data is boxes.data, where coordinates, confidence, and label information are all stored.

Now let's draw these values to the image like the yolo CLI command. 

The ultralytics.yolo.engine.results.Results object contains original image information in orig_img. Since this value is a numpy array type, it can be used directly as a Mat type in OpenCV.

from ultralytics import YOLO
import cv2

colors = [(255,0 , 0), (0,255,0), (0,0,255)]
font = cv2.FONT_HERSHEY_SIMPLEX   
def draw(img, boxes):
    index = 0
    for box in boxes.data:
        p1 =  (int(box[0].item()), int(box[1].item()))
        p2 =  (int(box[2].item()), int(box[3].item()))
        img = cv2.rectangle(img, p1, p2, colors[index % len(colors)], 3)
        text = label_map[int(box[5].item())] + " %4.2f"%(box[4].item()) 
        cv2.putText(img, text, (p1[0], p1[1] - 10), font, fontScale = 1, color = colors[index % len(colors)], thickness = 2)
        index += 1
    cv2.imwrite("./result.jpg", img)    
    # cv2.imshow("draw", img)
    # cv2.waitKey(0)
    # cv2.destroyAllWindows()


# Load a model
model = YOLO("yolov8n.pt")  # load an official model
label_map = model.names
# Predict with the model
results = model("https://ultralytics.com/images/bus.jpg")  # predict on an image

count = len(results)

for result in results:
    draw(result.orig_img, result.boxes)

<sample_detect2.py>

Let's run and check the output result.jpg.

<result.jpg>

Finally, VOLOv8 using Python also got the same result!

Python API and YOLOv8 inference type

In the previous examples, the result was obtained by directly inserting the image file into the YOLO model.

But since I use OpenCV a lot, most of the time I want to open an image using OpenCV and then pass the Mat object to the YOLO model. 

from ultralytics import YOLO
import cv2

colors = [(255,0 , 0), (0,255,0), (0,0,255)]
font = cv2.FONT_HERSHEY_SIMPLEX   
def draw(img, boxes):
    index = 0
    for box in boxes.data:
        p1 =  (int(box[0].item()), int(box[1].item()))
        p2 =  (int(box[2].item()), int(box[3].item()))
        img = cv2.rectangle(img, p1, p2, colors[index % len(colors)], 3)
        text = label_map[int(box[5].item())] + " %4.2f"%(box[4].item()) 
        cv2.putText(img, text, (p1[0], p1[1] - 10), font, fontScale = 1, color = colors[index % len(colors)], thickness = 2)
        index += 1
    cv2.imwrite("./result2.jpg", img)    
    # cv2.imshow("draw", img)
    # cv2.waitKey(0)
    # cv2.destroyAllWindows()


# Load a model
model = YOLO("yolov8n.pt")  # load an official model
label_map = model.names

img = cv2.imread("./bus.jpg", cv2.IMREAD_COLOR)
results = model(img)  # predict on an OpenCV mat object

for result in results:
    draw(result.orig_img, result.boxes)

<simple_detect3.py>

As you can see from the code above, you can pass an OpenCV mat object as an input parameter instead of a file name, and the result is the same as passing a file name.

The types of input sources that can be received in YOLOv8 are as follows. You can see that it provides various input sources such as Python PIL, OpenCV, and numpy as well as file names.

<YOLOv8 inference source type>

Running YOLOv8 models using torchvision

In the above document, one of the input sources is a torch tensor. However, as of March 2023, this feature does not appear to have been implemented.

The ultralytics Githib issue page has the following article:

<Github issue page>

Therefore, if you use torchvision, you must convert the tensor type image to PIL or np.array format until YOLOV8 properly supports tensor type source. In the example below, I open an image file using torchvision and then convert the image tensor to PIL format and feed it to the model.

import torch
import torchvision as tv
from ultralytics import YOLO
import cv2
import torchvision.transforms as T

colors = [(255,0 , 0), (0,255,0), (0,0,255)]
font = cv2.FONT_HERSHEY_SIMPLEX   
def draw(img, boxes):
    index = 0
    for box in boxes.data:
        p1 =  (int(box[0].item()), int(box[1].item()))
        p2 =  (int(box[2].item()), int(box[3].item()))
        img = cv2.rectangle(img, p1, p2, colors[index % len(colors)], 3)
        text = label_map[int(box[5].item())] + " %4.2f"%(box[4].item()) 
        cv2.putText(img, text, (p1[0], p1[1] - 10), font, fontScale = 1, color = colors[index % len(colors)], thickness = 2)
        index += 1
    cv2.imwrite("./result3.jpg", img)    


# Load a model
model = YOLO("yolov8n.pt")  # load an official model
label_map = model.names


img = tv.io.read_image("./bus.jpg")
img = T.ToPILImage()(img)
results = model(img)  # predict on an image

for result in results:
    draw(result.orig_img, result.boxes)

<simple_detect4.py>

If you open and check the result3.jpg file, you can see that an image like result2.jpg has been created.

Running YOLOv8 models directly from OpenCV

In the 2021 article "Running OpenPose models directly from OpenCV", it was explained that since OpenCV 4.2, various network models can be used directly in OpenCV. 

I installed OpenCV using Anaconda. You should first check whether the version of OpenCV you are using can use the dnn function to directly load the network model. Unfortunately, the OpenCV of Anaconda we installed does not support the dnn function.

You can check it using OpenCV's cv2.getBuildInformation() function.

  ......

OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cvv datasets dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      barcode dnn_objdetect dnn_superres mcc text wechat_qrcode
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev dnn java julia matlab ovis python2 sfm ts viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

......

<cv2.getBuildInformation() output of anaconda opencv>

And this is output of Xavier NX built in OpenCV's output.

  ......

 OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python2 python3 stitching ts video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         NO

......

<cv2.getBuildInformation() output of Jetson built in opencv>

It can be seen that OpenCV's dnn cannot be used in the anaconda environment we are currently using.

Wrapping up

In the previous article, we learned about installing YOLOv8 on Xavier NX, and in this article, we learned how to use the CLI commands and Python API provided by YOLOv8.

In the next article, we will look at YOLOv8 processing speed comparison and video processing in Xavier NX.

You can downlaod the source code at my Github.

Xavier NX - Installing OpenCV 4.6, PyTorch, YOLOv8 in Anaconda Virtual Environment (JetPack 5.1)

 As of March 2023, the most recent version of YOLO is v8. VOLOv8 is published by Ultralytics, the creators of v5. In this article, we will learn how to use the latest YOLOv8 on Xavier NX with JetPack 5.1 installed.

Tips: In this article, I used my username as spypiggy. Therefore, change the user name of spypyggy to the user name you use, such as /home/spypyggy/anaconda3.

First of all, let's look at the software required to use YOLOv8. Refer to the requirement.txt file on the YOLOv8 homepage.

# Base ----------------------------------------
matplotlib>=3.2.2
numpy>=1.18.5
opencv-python>=4.6.0
Pillow>=7.1.2
PyYAML>=5.3.1
requests>=2.23.0
scipy>=1.4.1
torch>=1.7.0
torchvision>=0.8.1
tqdm>=4.64.0

The modules you need to pay attention to are opencv and pytorch. Earlier, we already created a virtual environment in Anaconda with python3.8 installed. We will continue to work on this virtual environment.

Prerequsites

 

Install OpenCV 4.6

JetPack 5.1 comes with OpenCV 4.5 installed. However, since we will be using an anaconda virtual environment, we need to install OpenCV newly. And as you can see from the contents of the requirement.txt file, OpenCV version 4.6 or higher must be used to install YOLOv8.

To install a package in an anaconda environment, you can use the "conda install" command or the python package management command "pip install". 

Caution: However, OpenCV 4.5, which is already installed in JetPack 5.1, and Anaconda's OpenCV 4.6, which we want to install, have differences as well as differences in version. This difference becomes a problem when doing video or camera processing later. I will explain this part again later. If you want to use OpenCV for video playback or recording as well as image processing, please do not install OpenCV from Anaconda, go to "Build the latest version of OpenCV "later.

Packages that can be installed using the conda install command can be found with the conda search command. Let's find OpenCV with this command. If you can't find it, install it with "pip install" command. In general, it is recommended to install with the conda command in an anaconda environment. The reason is that the Python package installed with the conda install command is well-optimized for the anaconda environment, so it is known that better performance can be achieved.

(yolov8) spypiggy@spypiggy-NX:~$ conda install opencv=4.6.0
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/spypiggy/anaconda3/envs/yolov8

  added / updated specs:
    - opencv=4.6.0


The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-aarch64::blas-1.0-openblas
  bzip2              pkgs/main/linux-aarch64::bzip2-1.0.8-hfd63f10_2
  cairo              pkgs/main/linux-aarch64::cairo-1.16.0-h537eab0_3
  dbus               pkgs/main/linux-aarch64::dbus-1.13.18-h821dc26_0
  eigen              pkgs/main/linux-aarch64::eigen-3.3.7-h59a28a9_1
  expat              pkgs/main/linux-aarch64::expat-2.4.9-h419075a_0
  ffmpeg             pkgs/main/linux-aarch64::ffmpeg-4.2.2-hdfaaa67_0
  fontconfig         pkgs/main/linux-aarch64::fontconfig-2.14.1-haa5834d_1
  freetype           pkgs/main/linux-aarch64::freetype-2.12.1-h6df46f4_0
  giflib             pkgs/main/linux-aarch64::giflib-5.2.1-h998d150_3
  glib               pkgs/main/linux-aarch64::glib-2.69.1-h94b7715_2
  gmp                pkgs/main/linux-aarch64::gmp-6.2.1-h22f4aa5_3
  gnutls             pkgs/main/linux-aarch64::gnutls-3.6.15-hc6589d6_0
  graphite2          pkgs/main/linux-aarch64::graphite2-1.3.14-h22f4aa5_1
  gst-plugins-base   pkgs/main/linux-aarch64::gst-plugins-base-1.14.1-h419075a_1
  gstreamer          pkgs/main/linux-aarch64::gstreamer-1.14.1-h998d150_1
  harfbuzz           pkgs/main/linux-aarch64::harfbuzz-4.3.0-h085e3a5_0
  hdf5               pkgs/main/linux-aarch64::hdf5-1.10.6-h8b20701_1
  icu                pkgs/main/linux-aarch64::icu-68.1-h22f4aa5_0
  jpeg               pkgs/main/linux-aarch64::jpeg-9e-h998d150_1
  krb5               pkgs/main/linux-aarch64::krb5-1.19.4-ha2725d6_0
  lame               pkgs/main/linux-aarch64::lame-3.100-hfd63f10_0
  lerc               pkgs/main/linux-aarch64::lerc-3.0-h22f4aa5_0
  libclang           pkgs/main/linux-aarch64::libclang-10.0.1-default_h6b8c85e_2
  libdeflate         pkgs/main/linux-aarch64::libdeflate-1.17-h998d150_0
  libedit            pkgs/main/linux-aarch64::libedit-3.1.20221030-h998d150_0
  libevent           pkgs/main/linux-aarch64::libevent-2.1.12-ha9ffb65_0
  libgfortran-ng     pkgs/main/linux-aarch64::libgfortran-ng-11.2.0-h6e398d7_1
  libgfortran5       pkgs/main/linux-aarch64::libgfortran5-11.2.0-h1234567_1
  libidn2            pkgs/main/linux-aarch64::libidn2-2.3.1-h2f4d8fa_0
  libllvm10          pkgs/main/linux-aarch64::libllvm10-10.0.1-h6c8bc22_6
  libopenblas        pkgs/main/linux-aarch64::libopenblas-0.3.21-hc2e42e2_0
  libopus            pkgs/main/linux-aarch64::libopus-1.3.1-h2f4d8fa_0
  libpng             pkgs/main/linux-aarch64::libpng-1.6.39-h998d150_0
  libpq              pkgs/main/linux-aarch64::libpq-12.9-h140f9b7_3
  libtasn1           pkgs/main/linux-aarch64::libtasn1-4.16.0-hfd63f10_0
  libtiff            pkgs/main/linux-aarch64::libtiff-4.5.0-h419075a_2
  libunistring       pkgs/main/linux-aarch64::libunistring-0.9.10-h2f4d8fa_0
  libuuid            pkgs/main/linux-aarch64::libuuid-1.41.5-h998d150_0
  libvpx             pkgs/main/linux-aarch64::libvpx-1.8.2-h7c1a80f_0
  libwebp            pkgs/main/linux-aarch64::libwebp-1.2.4-he1bfee4_1
  libwebp-base       pkgs/main/linux-aarch64::libwebp-base-1.2.4-h998d150_1
  libxcb             pkgs/main/linux-aarch64::libxcb-1.15-h2f4d8fa_0
  libxkbcommon       pkgs/main/linux-aarch64::libxkbcommon-1.0.1-h1897131_0
  libxml2            pkgs/main/linux-aarch64::libxml2-2.9.14-he30c317_0
  libxslt            pkgs/main/linux-aarch64::libxslt-1.1.35-hd0e857b_0
  lz4-c              pkgs/main/linux-aarch64::lz4-c-1.9.4-h419075a_0
  nettle             pkgs/main/linux-aarch64::nettle-3.7.3-h82288b7_1
  nspr               pkgs/main/linux-aarch64::nspr-4.33-h22f4aa5_0
  nss                pkgs/main/linux-aarch64::nss-3.74-hcaefab4_0
  numpy              pkgs/main/linux-aarch64::numpy-1.23.5-py38h8708280_0
  numpy-base         pkgs/main/linux-aarch64::numpy-base-1.23.5-py38h4a83355_0
  opencv             pkgs/main/linux-aarch64::opencv-4.6.0-py38he2e48ef_3
  openh264           pkgs/main/linux-aarch64::openh264-1.8.0-h22f4aa5_0
  openjpeg           pkgs/main/linux-aarch64::openjpeg-2.4.0-hf3eb033_0
  pcre               pkgs/main/linux-aarch64::pcre-8.45-h22f4aa5_0
  pixman             pkgs/main/linux-aarch64::pixman-0.40.0-h2f4d8fa_1
  qt-main            pkgs/main/linux-aarch64::qt-main-5.15.2-h1cb44d8_7
  qt-webengine       pkgs/main/linux-aarch64::qt-webengine-5.15.9-ha607213_4
  qtwebkit           pkgs/main/linux-aarch64::qtwebkit-5.212-h2b8f10b_4
  x264               pkgs/main/linux-aarch64::x264-1!152.20180806-h2f4d8fa_0
  zstd               pkgs/main/linux-aarch64::zstd-1.5.2-hfcb3217_0


Proceed ([y]/n)? y

Since the conda install command automatically installs related packages required for the package installation process, errors do not occur except in special cases.

After the installation is complete, you can check:

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.6.0'
>>>

JetPack 5.1 built in OpenCV 4.5 Vs. Anaconda OpenCV 4.6

Now the Xavier NX I use has two OpenCVs installed.

One is OpenCV 4,5 installed by default in JetPack 5.1, and the other is OpenCV 4.6 installed in Anaconda virtual environments. Let's take a look at the difference between these two.

The best way to verify the properties of OpenCV is to use the cv2.getBuildInformation() function. This function shows various options at the time of OpenCV package build.

This is the output of the built-in version. Please pay attention to the red marked part.

>>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.5.4 =====================================
  Version control:               4.5.4-8-g3e4c170df4

  Platform:
    Timestamp:                   2022-01-18T10:01:01Z
    Host:                        Linux 5.10.65-tegra aarch64
    CMake:                       3.16.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 9.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python2 python3 stitching ts video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           GTK2
    GTK+:                        YES (ver 2.24.32)
      GThread :                  YES (ver 2.64.6)
      GtkGlExt:                  NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.1.0)
    JPEG 2000:                   build (ver 2.4.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    FFMPEG:                      YES
      avcodec:                   YES (58.54.100)
      avformat:                  YES (58.29.100)
      avutil:                    YES (56.31.100)
      swscale:                   YES (5.5.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.16.2)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            TBB (ver 2020.1 interface 11101)

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.7)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.5.1)

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.18)
    numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.16.5)
    install path:                lib/python2.7/dist-packages/cv2/python-2.7

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.8.10)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.8.so (ver 3.8.10)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.17.4)
    install path:                lib/python3.8/dist-packages/cv2/python-3.8

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr
-----------------------------------------------------------------

<output of Jetpack 5.1 pre-installed OpenCV 4.5>

This is the output of the Anaconda OpenCV version 4.6. A meaningless placehold garbage string is being output, but I'm not sure what's causing this. But the important part is the part marked in red. 

Compared to OpenCV 4.5 installed with JetPack 5.1, the package is built without support for ffmpeg. And the GStreamer version is also a little lower. 

>>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.6.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /croot/opencv-suite_1676452041368/work/opencv_contrib-4.6.0/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2023-02-15T09:08:53Z
    Host:                        Linux 5.10.162-141.675.amzn2.aarch64 aarch64
    CMake:                       3.22.1
    CMake generator:             Ninja
    CMake build tool:            /croot/opencv-suite_1676452041368/_build_env/bin/ninja
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /croot/opencv-suite_1676452041368/_build_env/bin/aarch64-conda-linux-gnu-c++  (ver 11.2.0)
    C++ flags (Release):         -fvisibility-inlines-hidden -std=c++11 -fmessage-length=0 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=/croot/opencv-suite_1676452041368/work=/usr/local/src/conda/opencv-suite-4.6.0 -fdebug-prefix-map=/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac=/usr/local/src/conda-prefix -D__STDC_CONSTANT_MACROS   -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fvisibility-inlines-hidden -std=c++11 -fmessage-length=0 -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=/croot/opencv-suite_1676452041368/work=/usr/local/src/conda/opencv-suite-4.6.0 -fdebug-prefix-map=/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac=/usr/local/src/conda-prefix -D__STDC_CONSTANT_MACROS   -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fopenmp -g  -DDEBUG -D_DEBUG
    C Compiler:                  /croot/opencv-suite_1676452041368/_build_env/bin/aarch64-conda-linux-gnu-cc
    C flags (Release):           -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=/croot/opencv-suite_1676452041368/work=/usr/local/src/conda/opencv-suite-4.6.0 -fdebug-prefix-map=/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac=/usr/local/src/conda-prefix   -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -isystem /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=/croot/opencv-suite_1676452041368/work=/usr/local/src/conda/opencv-suite-4.6.0 -fdebug-prefix-map=/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac=/usr/local/src/conda-prefix   -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fopenmp -g  -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -Wl,-rpath-link,/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -L/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib  -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined
    Linker flags (Debug):        -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -Wl,-rpath-link,/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -L/croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib  -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core cvv datasets dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      barcode dnn_objdetect dnn_superres mcc text wechat_qrcode
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev dnn java julia matlab ovis python2 sfm ts viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           QT5
    QT:                          YES (ver 5.15.2 )
      QT OpenGL support:         NO

  Media I/O:
    ZLib:                        /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/libz.so (ver 1.2.13)
    JPEG:                        /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/libjpeg.so (ver 90)
    PNG:                         /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/libpng.so (ver 1.6.37)
    TIFF:                        /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/libtiff.so (ver 42 / 4.2.0)
    JPEG 2000:                   OpenJPEG (ver 2.3.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    GStreamer:                   YES (1.14.1)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            OpenMP

  Trace:                         YES (built-in)

  Other third-party libraries:
    Eigen:                       YES (ver 3.3.7)
    Custom HAL:                  YES (carotene (ver 0.0.1))

  Python 3:
    Interpreter:                 /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/bin/python3 (ver 3.8.15)
    Libraries:                   /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/libpython3.8.so (ver 3.8.15)
    numpy:                       /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.8/site-packages/numpy/core/include (ver 1.16.6)
    install path:                lib/python3.8/site-packages/cv2/python-3.8

  Python (for build):            /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/bin/python

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /croot/opencv-suite_1676452041368/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
-----------------------------------------------------------------

<output of Anaconda installed OpenCV 4.6>

As a result of my testing, there is no problem opening and working with normal images, but problems occur when opening and working with videos. Since GStreamer is supported, you might work with video files by creating a GStreamer pipeline.

>>> import cv2
>>> cap = cv2.VideoCapture("./WUzgd7C1pWA.mp4")
[ WARN:0@41.141] global /croot/opencv-suite_1676452041368/work/modules/videoio/src/cap_gstreamer.cpp (2386) handleMessage OpenCV | GStreamer warning: your GStreamer installation is missing a required plugin
[ WARN:0@41.142] global /croot/opencv-suite_1676452041368/work/modules/videoio/src/cap_gstreamer.cpp (2401) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module uridecodebin0 reported: Your GStreamer installation is missing a plug-in.
[ WARN:0@41.145] global /croot/opencv-suite_1676452041368/work/modules/videoio/src/cap_gstreamer.cpp (1356) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@41.146] global /croot/opencv-suite_1676452041368/work/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
>>> print(cap.read())
(False, None)

<Video File Open Error in Anaconda OpenCV>

So if you have to use OpenCV to process video files in an anaconda environment, there are two possible ways.

The first method is to create and use GStreamer's pipeline in OpenCV. And the second is to build and install the OpenCV package that supports ffmpeg.

Build the latest version of OpenCV for Anaconda

For how to install the latest version of OpenCV in Xavier NX, Jetpack 5.1, refer to "Installing the Latest Version of OpenCV on Xavier NX".

Install Pytorch

There are two ways to use PyTorch with JetPack 5.1. The first is to use a Docker image. The second is download and install PyTorch for Jetson Xavier NX. In an anaconda environment, I will use the second method.

Caution: Do not follow the installation instructions on the PyTorch homepage. Since NVidia provides a PyTorch package adapted for the Jetson series of GPUs, you must use PyTorch provided by NVidia.

NVIDIA PyTorch Docker Image

How to download and use PyTorch docker images on the Jetson series is well documented on the NVIDIA L4T PyTorch page.

<NVIDIA L4T PyTorch homepage>

Using docker images with Jetpack 5.1 will work up to PyTorch 2.0.

Download and install PyTorch

Previously, to install PyTorch on the Jetson series, refer to https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048. But now the official page to install PyTorch from the Jetson series has moved to https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html.

<Official page to install PyTorch from Jetson series>

Caution: Since we use anaconda virtual environment, refer to the contents of the above page, but do not follow the same installation process as above.

PyTorch Download

If you go to the https://developer.download.nvidia.cn/compute/redist/jp/v51/pytorch/ page, you can find two PyTorch packages as follows. Download the latest version.

Instead of downloading the PyTorch whl file to your local computer, you can also install the remote whl file directly using the pip command. But I will safely download and install it.

First install requred packages.

sudo apt-get -y update
sudo apt-get -y install bc lld-8 gettext-base gfortran-8 iputils-ping \
libbz2-dev libc++-dev libcgal-dev libfreetype6-dev  \
libhdf5-dev libjpeg-dev liblzma-dev libncurses5-dev libncursesw5-dev \
libpng-dev libreadline-dev libssl-dev libsqlite3-dev libxml2-dev \
libxslt-dev locales moreutils python-openssl rsync scons libopenblas-dev

Caution : The official NVidia PyTorch installation page, https://developer.download.nvidia.cn/compute/redist/jp/v51/pytorch/, says to install libffi-dev together, but in the Anaconda environment, libffi-3.4.2 version is already installed. Therefore, there is no problem with the build. Rather, installing a new libffi-dev causes a version collision problem. Therefore, if PyTorch is installed in an anaconda environment, this package is not installed.

Then download and install PyTorch.

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ wget https://developer.download.nvidia.cn/compute/redist/jp/v51/pytorch/torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl
--2023-03-09 23:47:41--  https://developer.download.nvidia.cn/compute/redist/jp/v51/pytorch/torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl
Resolving developer.download.nvidia.cn (developer.download.nvidia.cn)... 129.227.6.166, 129.227.6.167
Connecting to developer.download.nvidia.cn (developer.download.nvidia.cn)|129.227.6.166|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 573441346 (547M) [application/octet-stream]
Saving to: ‘torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl’

torch-1.14.0a0+44dac51c.nv23.0 100%[=================================================>] 546.88M  11.7MB/s    in 49s

2023-03-09 23:48:31 (11.1 MB/s) - ‘torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl’ saved [573441346/573441346]

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ pip install --no-cache torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl
Processing ./torch-1.14.0a0+44dac51c.nv23.02-cp38-cp38-linux_aarch64.whl
Collecting networkx
  Downloading networkx-3.0-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 11.2 MB/s eta 0:00:00
Collecting sympy
  Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 12.3 MB/s eta 0:00:00
Collecting typing-extensions
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting mpmath>=0.19
  Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 17.3 MB/s eta 0:00:00
Installing collected packages: mpmath, typing-extensions, sympy, networkx, torch
Successfully installed mpmath-1.3.0 networkx-3.0 sympy-1.11.1 torch-1.14.0a0+44dac51c.nv23.2 typing-extensions-4.5.0

Let's verify that we have successfully installed PyTorch.

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'1.14.0a0+44dac51c.nv23.02'
>>> torch.cuda.is_available()
True

Build Torchvision

Now it is time to install Torchvision. Torchvision doesn't seem to be provided separately by NVidia. You can download the source code and build it yourself.

You need ffmpeg to process the video on Torchvision. Install ffmepg 4.2 in advance.

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ sudo apt-get install ffmpeg

Download the source code and build torchvision

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ wget https://github.com/pytorch/vision/archive/v0.14.0.tar.gz
(yolov8) spypiggy@spypiggy-NX:~/Downloads$ tar -xvzf v0.14.0.tar.gz
(yolov8) spypiggy@spypiggy-NX:~/Downloads$ cd vision-0.14.0
#This takes very long time, have a coffee time
(yolov8) spypiggy@spypiggy-NX:~/Downloads/vision-0.14.0$ python setup.py install

Let's check whether the installation is correct. If you see the screen like this, the installation is successful. 

(yolov8) spypiggy@spypiggy-NX:~/Downloads/vision-0.14.0$ cd ..
(yolov8) spypiggy@spypiggy-NX:~/Downloads$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchvision
>>> torchvision.__version__
'0.14.0a0'
>>>

Caution: Do not test in the vision-0.14.0 directory. The reason is that this directory contains a subdirectory named torchvision, so Python's "import torchvision" statement doesn't work.

Install the rest of the packages

Now install the packages that were not installed from the requirement.txt file in the YOLOv8 directory. Use conda install whenever possible.

(yolov8) spypiggy@spypiggy-NX:~$ conda install matplotlib requests scipy tqdm
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/spypiggy/anaconda3/envs/yolov8

  added / updated specs:
    - matplotlib
    - requests
    - scipy
    - tqdm


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    brotlipy-0.7.0             |py38hfd63f10_1002         319 KB
    cffi-1.15.1                |   py38h998d150_3         270 KB
    contourpy-1.0.5            |   py38hb8fdbf2_0         200 KB
    cryptography-39.0.1        |   py38h3d58568_0         1.4 MB
    idna-3.4                   |   py38hd43f75c_0          93 KB
    importlib_resources-5.2.0  |     pyhd3eb1b0_1          21 KB
    kiwisolver-1.4.4           |   py38h419075a_0          77 KB
    matplotlib-3.7.0           |   py38hd43f75c_0           8 KB
    matplotlib-base-3.7.0      |   py38he2e48ef_0         6.6 MB
    packaging-22.0             |   py38hd43f75c_0          68 KB
    pillow-9.4.0               |   py38h419075a_0         716 KB
    pooch-1.4.0                |     pyhd3eb1b0_0          41 KB
    pyopenssl-23.0.0           |   py38hd43f75c_0          96 KB
    pyparsing-3.0.9            |   py38hd43f75c_0         148 KB
    pysocks-1.7.1              |   py38hd43f75c_0          28 KB
    requests-2.28.1            |   py38hd43f75c_0          93 KB
    scipy-1.10.0               |   py38h7caaa05_1        23.5 MB
    tornado-6.2                |   py38h998d150_0         598 KB
    tqdm-4.64.1                |   py38hd43f75c_0         126 KB
    urllib3-1.26.14            |   py38hd43f75c_0         192 KB
    zipp-3.11.0                |   py38hd43f75c_0          19 KB
    ------------------------------------------------------------
                                           Total:        34.6 MB

You are now ready to install YOLOv8. Finally, let's install YOLOV8.

Install YOLOv8

Since we have installed most of the software we need, we now install YOLOv8 with the pip command.

(yolov8) spypiggy@spypiggy-NX:~/Downloads$ pip install ultralytics

Note that the package name is ultralytics, not yolov8. But the moment I load YOLOv8 in python I get an error like this.

Fixing GLIBCXX_3.4.29 Problem

(yolov8) spypiggy@spypiggy-NX:~$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ultralytics import YOLO
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/__init__.py", line 5, in <module>
    from ultralytics.yolo.engine.model import YOLO
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/yolo/__init__.py", line 3, in <module>
    from . import v8
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/yolo/v8/__init__.py", line 3, in <module>
    from ultralytics.yolo.v8 import classify, detect, segment
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/yolo/v8/classify/__init__.py", line 3, in <module>
    from ultralytics.yolo.v8.classify.predict import ClassificationPredictor, predict
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/yolo/v8/classify/predict.py", line 5, in <module>
    from ultralytics.yolo.engine.predictor import BasePredictor
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/ultralytics/yolo/engine/predictor.py", line 34, in <module>
    import cv2
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/__init__.py", line 153, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /lib/aarch64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/spypiggy/anaconda3/envs/yolov8/lib/python3.8/site-packages/cv2/python-3.8/cv2.cpython-38-aarch64-linux-gnu.so)

If you look for libstdc++.so.6 in anaconda virtual environment, you can see that it is a symbolic link of libstdc++.so.6.0.29 file at /home/spypiggy/anaconda3/lib.

(base) spypiggy@spypiggy-NX:~/anaconda3/lib$ ll /home/spypiggy/anaconda3/lib/libstdc++*
lrwxrwxrwx 1 spypiggy spypiggy      19  3월  9 20:54 libstdc++.so -> libstdc++.so.6.0.29*
-rwxrwxr-x 3 spypiggy spypiggy 3934728  6월  1  2022 libstdc++.so.6.0.29*

And the /usr/lib/aarch64-linux-gnu directory has the following files.

(base) spypiggy@spypiggy-NX:/usr/lib$ ll aarch64-linux-gnu/libstd*
lrwxrwxrwx 1 root root      19  5월 29  2021 aarch64-linux-gnu/libstdc++.so.6 -> libstdc++.so.6.0.28
-rw-r--r-- 1 root root 1907992  5월 29  2021 aarch64-linux-gnu/libstdc++.so.6.0.28

There are two ways to solve this problem./usr/linux-linux-gnu directory link 64-gnu directory link. 

And a little safer way to change the LD_LIBRARY_PATH environment variable changes.

The first way

Replace the symbolic link in the /usr/lib/aarch64-linux-gnu directory with libstdc++.so.6.0.29 in the anaconda directory, or copy the libstdc++.so.6.0.29 file to the /usr/lib directory and remake the symbolic link like this:

(base) spypiggy@spypiggy-NX:~/anaconda3/lib$ sudo cp  /home/spypiggy/anaconda3/lib/libstdc++.so.6.0.29 /usr/lib/aarch64-linux-gnu/
(base) spypiggy@spypiggy-NX:~/anaconda3/lib$ cd /usr/lib/aarch64-linux-gnu
(base) spypiggy@spypiggy-NX:/usr/lib/aarch64-linux-gnu$ sudo rm -f libstdc++.so.6
(base) spypiggy@spypiggy-NX:/usr/lib/aarch64-linux-gnu$ sudo ln -s libstdc++.so.6.0.29 libstdc++.so.6
(base) spypiggy@spypiggy-NX:/usr/lib/aarch64-linux-gnu$ ll libstd*
lrwxrwxrwx 1 root root      19  3월 10 02:09 libstdc++.so.6 -> libstdc++.so.6.0.29*
-rw-r--r-- 1 root root 1907992  5월 29  2021 libstdc++.so.6.0.28
-rwxr-xr-x 1 root root 3934728  3월 10 02:07 libstdc++.so.6.0.29*

This operation carries some risks. If another program wants to use libstdc++.so.6 linked to libstdc++.so.6.0.28, it can be a problem. In this case, you can restore the symbolic link again.

A safer second way (recommended method)

If you want to avoid the risks described above, do not modify the symbolic link in the /usr/lib/aarch64-linux-gnu directory, but change the LD_LIBRARY_PATH environment variable.

Automatically execute shell scripts in the ./etc/conda/activate.d directory when the Anaconda virtual environment is activated. Conversely, when the virtual environment is deactivated, it automatically executes the shell script in the ./etc/conda/deactivate.d directory. If you create a script as follows, when the virtual environment is activated, you first find the lib path of the anaconda.

(yolov8) spypiggy@spypiggy-NX:~$ cd ~/anaconda3/envs/yolov8
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ mkdir -p ./etc/conda/activate.d
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ mkdir -p ./etc/conda/deactivate.d
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ touch ./etc/conda/activate.d/env_vars.sh
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ touch ./etc/conda/deactivate.d/env_vars.sh

(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ cat etc/conda/activate.d/env_vars.sh
export OLD_LD_LIBRARY_PATH=${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH=/home/spypiggy/anaconda3/lib:${LD_LIBRARY_PATH}
(yolov8) spypiggy@spypiggy-NX:~/anaconda3/envs/yolov8$ cat etc/conda/deactivate.d/env_vars.sh
export LD_LIBRARY_PATH=${OLD_LD_LIBRARY_PATH}
unset OLD_LD_LIBRARY_PATH

And of course, the symbolic links in the /usr/lib/aarch64-linux-gnu directory do not need to be changed. It is still as follows.

(base) spypiggy@spypiggy-NX:/usr/lib$ ll aarch64-linux-gnu/libstd*
lrwxrwxrwx 1 root root      19  5월 29  2021 aarch64-linux-gnu/libstdc++.so.6 -> libstdc++.so.6.0.28
-rw-r--r-- 1 root root 1907992  5월 29  2021 aarch64-linux-gnu/libstdc++.so.6.0.28

Now let's test again.

(yolov8) spypiggy@spypiggy-NX:~$ python
Python 3.8.16 (default, Mar  2 2023, 03:16:31)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ultralytics import YOLO
>>> model = YOLO("yolov8n.pt")
Downloading https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt to yolov8n.pt...
100%|█████████████████████████████████████████████████████████████████████████████████| 6.23M/6.23M [00:01<00:00, 3.59MB/s]
>>>

Finally I successfully loaded the YOLOv8 model.

Tips : Occasionally, a Permission denied error may occur during package installation. The reason for this is that most of the previous installation process works with the root account rather than the current user (in my case, spypyggy), and the owner of a specific file or directory is the root. If this error occurs, it changes the owner of the entire anaconda directory at once.

(yolov8) spypiggy@spypiggy-NX:~/$ sudo chown -R spypiggy:spypiggy  /home/spypiggy/anaconda3

Wrapping up

Since YOLOv8 is based on PyTorch, it is important to have PyTorch properly installed. It is not difficult to install packages such as PyTorch and OpenCV because it is very easy to install and manage packages in the Anaconda environment.

I will cover how to use VOLOv8 in Xavier NX while testing YOLOv8 in the next article.