last updated 2022.02.27 : update for Jetpack 4.6

In this article, I have included content that can help in installing and optimizing OS images on Jetson series such as Nano, TX2, Xavier NX, and Xavier.

Jetson Series Tips

JetPack Version information

Sometimes, JetPack version-specific information is required.

date	version	L4T	OpenCV	Cuda	cuDNN	TensorRT	DeepStream
2021/8	JetPack 4.6 Production Release	L4T 32.6.1	OpenCV 4.1.1	CUDA 10.2	cuDNN 8.2.1	TensorRT 8.0.1	DeepStream 6.0
2020/12	JetPack 4.5 Production Release	L4T 32.5.0	OpenCV 4.1.1	CUDA 10.2.89	cuDNN 8.0.0	TensorRT 7.1.3	DeepStream 5.0
2020/7	JetPack 4.4 Production Release	L4T 32.4.3	OpenCV 4.1.1	CUDA 10.2.89	cuDNN 8.0.0	TensorRT 7.1.3	DeepStream 5.0
2020/4	JetPack 4.4 Developer Preview	L4T 32.4.2	OpenCV 4.1.1	CUDA 10.2.89	cuDNN 8.0.0	TensorRT 7.1.0	DeepStream 5.0 DP
2019/12	JetPack 4.3	L4T 32.3.1	OpenCV 4.1.1	CUDA 10.0.326	cuDNN 7.6.3	TensorRT 6.0.1	DeepStream 4.0.2
2019/11	JetPack 4.2.3	L4T 32.2.3	OpenCV 3.3.1	CUDA 10.0.326	cuDNN 7.5.0.56	TensorRT 5.1.6	DeepStream DeepStream 4.0.1

Check L4T(Linux for Tegra) version

How to check the JetPack version is as follows. Find the L4T value in the table above and compare it with the JetPack version. Use the following command to find the L4T value. This command is available for all Jetson series products using JetPack as well as Jetson Nano.

In the figure above, if you combine the R32 (Release) value and the 4.2 (Revision) value, you can get the L4T 32.4.2 value. Therefore, the Jetson device that executed the above command is using JetPack 4.4 DP version.

Check CUDA version and capability version

First, go to the /usr/local/cuda/bin/ directory. Then search "cuda-install-samples-X.X.sh" file. X.X means the version of cuda. Then execute the script file.

Note the addition of. At the end. This means to copy the sample file to the current directory.

spypiggy@spypiggy-nano:~$ cd /usr/local/cuda/bin/
spypiggy@spypiggy-nano:/usr/local/cuda/bin$ ls -al *.sh
-rwxr-xr-x 1 root root 800 10월 30  2019 cuda-install-samples-10.2.sh
spypiggy@spypiggy-nano:/usr/local/cuda/bin$ sudo ./cuda-install-samples-10.2.sh .
Copying samples to ./NVIDIA_CUDA-10.2_Samples now...
Finished copying samples.

Now you can see that the sample directory has been created.

spypiggy@spypiggy-nano:/usr/local/cuda/bin$ ls -al
total 50028
drwxr-xr-x  4 root root     4096  2월  7 19:22 .
drwxr-xr-x 12 root root     4096  1월 20 05:37 ..
-rwxr-xr-x  1 root root    64120 10월 30  2019 bin2c
drwxr-xr-x  2 root root     4096  1월 20 05:27 crt
-rwxr-xr-x  1 root root  4214864 10월 30  2019 cudafe++
-rwxr-xr-x  1 root root  6171912 10월 30  2019 cuda-gdb
-rwxr-xr-x  1 root root   477336 10월 30  2019 cuda-gdbserver
-rwxr-xr-x  1 root root      800 10월 30  2019 cuda-install-samples-10.2.sh
-rwxr-xr-x  1 root root   261656 10월 30  2019 cuda-memcheck
-rwxr-xr-x  1 root root   187568 10월 30  2019 cuobjdump
-rwxr-xr-x  1 root root   175056 10월 30  2019 fatbinary
-rwxr-xr-x  1 root root   200792 10월 30  2019 nvcc
-rw-r--r--  1 root root      417 10월 30  2019 nvcc.profile
-rwxr-xr-x  1 root root 22672976 10월 30  2019 nvdisasm
drwxr-xr-x 11 root root     4096  2월  7 19:22 NVIDIA_CUDA-10.2_Samples
-rwxr-xr-x  1 root root  6928624 10월 30  2019 nvlink
-rwxr-xr-x  1 root root  2928328 10월 30  2019 nvprof
-rwxr-xr-x  1 root root    80736 10월 30  2019 nvprune
-rwxr-xr-x  1 root root  6817024 10월 30  2019 ptxas

Now build the deviceQuery sample .

spypiggy@spypiggy-nano:/usr/local/cuda/bin/NVIDIA_CUDA-10.2_Samples$ cd 1_Utilities/deviceQuery
spypiggy@spypiggy-nano:/usr/local/cuda/bin/NVIDIA_CUDA-10.2_Samples/1_Utilities/deviceQuery$ sudo make
/usr/local/cuda-10.2/bin/nvcc -ccbin g++ -I../../common/inc  -m64    -gencode arch=compute_30,code=sm_30 -gencode arch=compute_32,code=sm_32 -gencode arch=compute_53,code=sm_53 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_62,code=sm_62 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery.o -c deviceQuery.cpp
/usr/local/cuda-10.2/bin/nvcc -ccbin g++   -m64      -gencode arch=compute_30,code=sm_30 -gencode arch=compute_32,code=sm_32 -gencode arch=compute_53,code=sm_53 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_62,code=sm_62 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery deviceQuery.o 
mkdir -p ../../bin/aarch64/linux/release

Run deviceQuery program and you can get detailed information about CUDA.

spypiggy@spypiggy-nano:/usr/local/cuda/bin/NVIDIA_CUDA-10.2_Samples/1_Utilities/deviceQuery$ ./deviceQuery 
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "NVIDIA Tegra X1"
  CUDA Driver Version / Runtime Version    10.2 / 10.2
  CUDA Capability Major/Minor version number:    5.3
  Total amount of global memory:                 3964 MBytes (4156694528 bytes)
  ( 1) Multiprocessors, (128) CUDA Cores/MP:     128 CUDA Cores
  GPU Max Clock rate:                            922 MHz (0.92 GHz)
  Memory Clock rate:                             13 Mhz
  Memory Bus Width:                              64-bit
  L2 Cache Size:                                 262144 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
  Maximum Layered 1D Texture Size, (num) layers  1D=(16384), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(16384, 16384), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 32768
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 1 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            Yes
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Compute Preemption:            No
  Supports Cooperative Kernel Launch:            No
  Supports MultiDevice Co-op Kernel Launch:      No
  Device PCI Domain ID / Bus ID / location ID:   0 / 0 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.2, CUDA Runtime Version = 10.2, NumDevs = 1

Since the CUDA version is already determined according to the installed Jetpack version, you can also check the JetPack's official documentation.

CUDA capability can be seen as a version of GPU. As new products with new GPUs are released, this value will continue to increase. The following table is part of a Wiki page.

Check cuDNN version

You can check the version in the cudnn_version.h file. This file resides in /usr/include/.

spypiggy@spypiggy-nano:/usr/local/cuda/bin$ cat /usr/include/cudnn_version.h |grep CUDNN
#ifndef CUDNN_VERSION_H_
#define CUDNN_VERSION_H_
#define CUDNN_MAJOR 8
#define CUDNN_MINOR 0
#define CUDNN_PATCHLEVEL 0
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)
#endif /* CUDNN_VERSION_H */

jetson_stats

jetson_stats is a very powerful packages that provides monitoring and system information of the Jetson series.

root@JetsonNano:~# pip3 install -U jetson-stats

root@JetsonNano:~# systemctl restart jetson_stats.service

root@JetsonNano:~# systemctl list-units --type service --all

  UNIT                                     LOAD      ACTIVE   SUB     DESCRIPTION
  accounts-daemon.service                  loaded    active   running Accounts Service
  alsa-restore.service                     loaded    active   exited  Save/Restore Sound Card State
  alsa-state.service                       loaded    inactive dead    Manage Sound Card State (restore and store)
  anacron.service                          loaded    inactive dead    Run anacron jobs
  apparmor.service                         loaded    inactive dead    AppArmor initialization
  apport-autoreport.service                loaded    inactive dead    Process error reports when automatic reporting is enabled
  apport.service                           loaded    active   exited  LSB: automatic crash report generation
  apt-daily-upgrade.service                loaded    inactive dead    Daily apt upgrade and clean activities
  apt-daily.service                        loaded    inactive dead    Daily apt download activities
● auditd.service                           not-found inactive dead    auditd.service
  avahi-daemon.service                     loaded    active   running Avahi mDNS/DNS-SD Stack
  bluetooth.service                        loaded    active   running Bluetooth service
  bolt.service                             loaded    active   running Thunderbolt system service
● chronyd.service                          not-found inactive dead    chronyd.service
  colord.service                           loaded    active   running Manage, Install and Generate Color Profiles
● connman.service                          not-found inactive dead    connman.service
● console-screen.service                   not-found inactive dead    console-screen.service
  console-setup.service                    loaded    active   exited  Set console font and keymap
  containerd.service                       loaded    active   running containerd container runtime
  cron.service                             loaded    active   running Regular background program processing daemon
  dbus.service                             loaded    active   running D-Bus System Message Bus
  dns-clean.service                        loaded    inactive dead    Clean up any mess left by 0dns-up
  docker.service                           loaded    inactive dead    Docker Application Container Engine
  emergency.service                        loaded    inactive dead    Emergency Shell
● festival.service                         not-found inactive dead    festival.service
● firewalld.service                        not-found inactive dead    firewalld.service
  fstrim.service                           loaded    inactive dead    Discard unused blocks
  gdm.service                              loaded    active   running GNOME Display Manager
  getty-static.service                     loaded    inactive dead    getty on tty2-tty6 if dbus and logind are not available
  getty@tty1.service                       loaded    inactive dead    Getty on tty1
  getty@tty7.service                       loaded    inactive dead    Getty on tty7
  gpsd.service                             loaded    inactive dead    GPS (Global Positioning System) Daemon
  grub-common.service                      loaded    active   exited  LSB: Record successful boot for GRUB
  haveged.service                          loaded    active   running Entropy daemon using the HAVEGE algorithm
  jetson_stats.service                loaded   active  running jetson_stats service
● kbd.service                              not-found inactive dead    kbd.service
  kerneloops.service                       loaded    active   running Tool to automatically collect and submit kernel crash signatures
  keyboard-setup.service                   loaded    active   exited  Set the console keyboard layout

root@JetsonNano:~# jtop

jetson_release is a powerful tool that organizes and shows the information of the Jetson system.

root@spypiggy-nano:~# jetson_release
 - NVIDIA Jetson Nano (Developer Kit Version)
   * Jetpack 4.5 [L4T 32.5.0]
   * NV Power Mode: MAXN - Type: 0
   * jetson_stats.service: active
 - Libraries:
   * CUDA: 10.2.89
   * cuDNN: 8.0.0.180
   * TensorRT: 7.1.3.0
   * Visionworks: 1.6.0.501
   * OpenCV: 4.1.1 compiled CUDA: NO
   * VPI: ii libnvvpi1 1.0.12 arm64 NVIDIA Vision Programming Interface library
   * Vulkan: 1.2.70

You can get more information about jetson_stats at https://github.com/rbonghi/jetson_stats.

Jetson Nano, Xavier NX Image Download

There are two ways to install the Jetson Nano Image. First method is using the SDK Manager. If you want to use the SDK Manager, prepare the host PC(should be installed Ubuntu OS). Jetson TX2 or Jetson Xaver must be installed using SDK Manager. But There's a SD card image for Jetson Nano and Xavier NX. Just download the image zip file at NVIDIA Jetson Download page and use Etcher program to flash the image zip file. Etcher supports zip archives, so you don't need to unzip zip files.

Headless setup

I like headless setup when setting up a single board computers like Raspberry Pi, Odroid. The same is true of the Jetson Nano. After Jetpack 4.2.1, Jetson Nano supports headless setup using USB cable. The host computer can be Linux, Windos or Mac.
If you use a Ubuntu host, there's an excellent article by JetsonHack here.

Connect the Jetson Nano and Host Computer with USB cable

Insert the flashed sd card into the Jetson Nano.
Connect the Jetson Nano and Windows PC(WIndows OS is OK) with USB cable(which data transfer possible). I used a USB cable shipped with Jetson TX2.
Power on the Jetson Nano, then wait for Windows to recognize Nano.
Start a Windows PowerShell and check the COM port of Jetson Nano. You can use WIndows Device manager instead.
Run putty and open the serial port. In my case "COM5".
If successful, you can see the console screen of Jetson Nano. See the images below.
Next, follow the instructions on the screen.

Network configuration

If you connect a monitor and keyboard to Jetson Nano, there is no problem. GUI program will help you. But headless mode(remote ssh client), you must search the configuration file first.
Generally you search /etc/network directory. But Jetson Nano use network-manager service, so check the "/etc/NetworkManager/system-connections" directory instead. This is my wired connection information. If you need to modify the static IP address, modify this file and reboot the Jetson Nano.

root@JetsonNano:/etc/NetworkManager/system-connections# pwd
/etc/NetworkManager/system-connections
root@JetsonNano:/etc/NetworkManager/system-connections# cat Wired\ connection\ 1 

[connection]
id=Wired connection 1
uuid=c31d6e43-9ed7-4724-8bca-bcd61adfb2e2
type=802-3-ethernet

[802-3-ethernet]
mac=00:04:4B:E5:58:9D

[ipv4]
method=manual
dns=8.8.8.8;
address1=192.168.11.86/24,192.168.11.1

[ipv6]
method=auto
ip6-privacy=2

The Power of Power

When I first purchased a JetsonNano, I used a 2.2A ~ 2.5A micro-usb DC power supply that I used with Raspberry Pi.

After making a sd card image, I inserted the card to the Jetson Nano, connect usb keyboard, mouse, hdmi monitor.
Then I turn on the Nano. Everything is OK. I installed development tools, compile the package like openCV successfully.

Then I run a heavy machine learning program like 'tensorflow based person pose estimation'. In a few minutes, Jetson Nano suddenly turned off.

At first, I think it's symptoms of insufficient memory. Generally if running out of memory occurs, OS kills the process, no shutdown.
After some googling, I found the cause may be the power problem.
I replaced the micro-usb power to a 4A barrel jack DC adapter.

Be careful : you should place a jump on J48 to use a barrel jack DC adapter.
I used a dupont cable instead. And a sudden turning off disappeared.

Power modes of Jetson Nano

The Jetson Nano has two power profiles, called modes. Mode 0 is 10W, Mode 1 is 5W.

To set the mode to 5 Watt mode:

$ sudo nvpmodel -m 1

To set it back to 10 Watt mode:

$ sudo nvpmodel -m 0

You can query the current mode(Default mode is 0):

root@spypiggy-desktop:/# nvpmodel -q
NV Power Mode: MAXN
0

Power modes of Jetson Xavier NX

The Jetson Xavier supports new 20W modes from JetPack 4.6. Jetpack 4.6 users are recommended to use 20W mode. Video encoding and AI operation superior to 15W mode are possible.

If you want high-performance work, set the mode to 6 ~ 8:(for JetPack 4.6 users)

$ sudo nvpmodel -m 6

If you want to work in general, set the mode to 5:

$ sudo nvpmodel -m 5

You can query the current mode(Default mode is 5):

root@spypiggy-nx:/usr/local/src# nvpmodel -q
NV Fan Mode:quiet
NV Power Mode: MODE_15W_6CORE
2

Dynamic Voltage and Frequency Scaling(DVFS)

From https://en.wikipedia.org/wiki/Dynamic_voltage_scaling:
Dynamic voltage scaling is a power management technique in computer architecture, where the voltage used in a component is increased or decreased, depending upon circumstances. Dynamic voltage scaling to increase voltage is known as overvolting; dynamic voltage scaling to decrease voltage is known as undervolting. Undervolting is done in order to conserve power, particularly in laptops and other mobile devices,^[1] where energy comes from a battery and thus is limited, or in rare cases, to increase reliability. Overvolting is done in order to increase computer performance.

The jetson_clocks tool(/usr/bin/jetson_clocks) always sets this value for best performance. Power usage will probably increase slightly.
Run this tool to maximize the Jetson Nano performance:

root@spypiggy-desktop:/# jetson_clocks
root@spypiggy-desktop:/# jetson_clocks --show
SOC family:tegra210  Machine:NVIDIA Jetson Nano Developer Kit
Online CPUs: 0-3
CPU Cluster Switching: Disabled
cpu0: Online=1 Governor=schedutil MinFreq=1428000 MaxFreq=1428000 CurrentFreq=1428000 IdleStates: WF                                 I=0 c7=0
cpu1: Online=1 Governor=schedutil MinFreq=1428000 MaxFreq=1428000 CurrentFreq=1428000 IdleStates: WF                                 I=0 c7=0
cpu2: Online=1 Governor=schedutil MinFreq=1428000 MaxFreq=1428000 CurrentFreq=1428000 IdleStates: WF                                 I=0 c7=0
cpu3: Online=1 Governor=schedutil MinFreq=1428000 MaxFreq=1428000 CurrentFreq=1428000 IdleStates: WF                                 I=0 c7=0
GPU MinFreq=921600000 MaxFreq=921600000 CurrentFreq=921600000
EMC MinFreq=204000000 MaxFreq=1600000000 CurrentFreq=1600000000 FreqOverride=1
Fan: speed=255
NV Power Mode: MAXN

After running the jetson_clocks tool, MinFreq and MaxFreq are now equal.

There's a good explanation on this site.
https://www.jetsonhacks.com/2019/04/10/jetson-nano-use-more-power/

Use more memory by changing Ubuntu desktop to LXDE!

If you download Jetson Nano official images, check the memory.

You may find that about 1.5GB memory is in use. Jetson Nano's 4GB memory is not enough for AI related tasks.
Changing Ubuntu 18.04 desktop to LXDE desktop, you can make extra 1GB memory.

sudo apt remove --purge ubuntu-desktop
sudo apt install lxdm 
sudo apt remove --purge gdm3
sudo apt install lxde
sudo apt install --reinstall lxdm

Be careful : You should do above tasks on remote ssh client.

After reboot, your Jetson Nano's desktop would be changed.

Now you can see that the memory usage of Jetson Nano is under 500MB.

Desktop background image is Debian! My Jetson Nano OS is changed to Debian?
No. Only the background image is changed. Your Jetson Nano OS is still Ubuntu 18.04.

There's a good explanation on this site.
https://www.zaferarican.com/post/how-to-save-1gb-memory-on-jetson-nano-by-installing-lubuntu-desktop

Remove Docker

If you don't use a docker container, remove it. Docker daemons use some system resources.

apt-get remove docker docker-engine docker.io containerd runc

Delete LibreOffice

I don't think you need LibreOffice on the JetsonNano.
Removing the LibreOffice will make more storage space.

apt-get remove --purge libreoffice*
apt-get clean
apt-get autoremove

Increase Memory #1- zram

This method is available for all Jetson series.

The Jetson Nano's 2GB and 4GB memory is by no means a big deal. Sometimes it is necessary to build large packages such as TensorFlow and OpenCV using tools such as gcc and bazel. In this case, an error may occur due to insufficient memory. In this case, it is necessary to compensate for the lack of RAM by increasing virtual memory.

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk with on-the-fly disk compression. The block device created with zram can then be used for swap or as general-purpose RAM disk. The two most common uses for zram are for the storage of temporary files (/tmp) and as a swap device. Initially, zram had only the latter function, hence the original name "compcache" ("compressed cache").

There is a second way to use the files on the SD card as virtual memory, but there is a way to increase the RAM capacity by compressing the data in the actual RAM.

This method has the disadvantage of increasing CPU usage because the CPU is involved in data IO of RAM, but it provides much faster speed than the method using the swap file.

So personally, I prefer this method to using the swap file.

install zram tool

Since there is already a well-made installation script file, download it from git.

$ git clone https://github.com/StuartIanNaylor/zram-swap-config \
&& cd zram-swap-config
$ sudo ./install.sh

Then reboot the system.

zram settings

Edit the /etc/zram-swap-config.conf file. The description of the item values is as follows.

MEM_FACTOR = Percentage of available ram to allocate to all zram swap devices which is divided equally by swap_devices number
DRIVE_FACTOR = Virtual uncompressed zram drive size estimate approx alg compression ratio
COMP_ALG lz4 is faster than lzo but some distro's show compile and operational difference and in use lzo depending on binary may be faster. Compression rates list below are minimums and generally far bigger in operation but dependent on content.
SWAP_DEVICES = number of indivial drives sharing memeory provided by MEM_FACTOR each device support multiple streams 1 large drive is generally better
SWAP_PRI = swap_priority for each drive 75 is a high order preference and should be well above other swap drives
PAGE_CLUSTER default page cluster is 3 which caches fetches in batches of 8 and helps with HDD paging, with zram mem 0 forces single page fetches This can help reduce latency and increase performance
SWAPPINESS default swappiness is 60 but with increased performance of zram swap 80 garners overall performance gain without excessive load Because zram uses compression load is created and even if minimal at intense load periods such as boot any extra load is unwanted Unfortunately there is no dynamic load balancing of swappiness as with zram in general operation SWAPINESS=100 will garner performance benefit If the overall load is reasonable at high load it will cause load to accumulate. If you check my repo there will also be a simple dynamic load based SWAPPINESS governor that will get of the hurdle of a static compromise on swappiness

The following is the memory status of my Xavier NX 8GB model.

spypiggy@spypiggy-desktop:~/src/resizeSwapMemory$ free -m
              total        used        free      shared  buff/cache   available
Mem:           7765        2396        5146           2         222        5179
Swap:          3882        1654        2228

And the following is the memory after rebooting. (I removed swap file. Because I don't need it any more)

spypiggy@spypiggy-desktop:~$ sudo cat /proc/swaps
[sudo] password for spypiggy: 
Filename				Type		Size	Used	Priority
/dev/zram0                              partition	9551992	0	75


spypiggy@spypiggy-desktop:~$ free -m
              total        used        free      shared  buff/cache   available
Mem:           7773        4004        1564          29        2205        3540
Swap:          9328           0        9328

Increase Memory #2- swap file

When you build a large software packages like openCV, you may experience an out of memory phenomenon. Increasing the swap file size can prevent this malfunction.

git clone https://github.com/JetsonHacksNano/installSwapfile
cd installSwapfile
./installSwapfile.sh

Above script file will increase 6GB swap files. You can change the swap file size by modifying the scripts. If you want to uninstall the swap setting, open the fstab file and delete the swap file line and reboot.

sudo vi /etc/fstab

There's a good explanation on this site.
https://www.jetsonhacks.com/2019/04/14/jetson-nano-use-more-memory/

Be careful : Even though you increase the swap file size, cuda actually use physical memory only. So deep learning frameworks those use cuda may not get benefits.

Python Virtual Environment

When using machine learning frameworks such as Tensorflow, Yolo and Pytorch at the same time, there are many cases where a virtual environment is created not to affected by other packages due to version problems of installation packages.
On desktops using SDDs and HDDs, it is cumbersome to replace the storage device with the operating system installed. Therefore, it is effective to create and use multiple Python virtual environments on the desktop. But in the case of Jetson Nano, which uses a microSD card as a storage device, it is more advantageous to prepare and replace several micro SD cards than to create a virtual environment. Therefore, it is recommended to prepare several microSD cards to install Python packages for your purpose, and replace them whenever necessary.

Python2 or Python3

I strongly recommend you use python3. Most deep learning frameworks support Python 3. In the future, support for Python 2 will be discontinued. So Python3 is the correct choice.

Resizing SD Card

Suppose you have backed up a 32GB SD card image of a Jetson Nano or Xavier NX.

And if you restore the image using a 64GB SD card, you will probably only be able to use 32GB of storage space as follows.

spypiggy@spypiggy-nano:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   30G   21G  7.1G  75% /
none            1.8G     0  1.8G   0% /dev
tmpfs           2.0G  4.0K  2.0G   1% /dev/shm
tmpfs           2.0G   27M  2.0G   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           397M     0  397M   0% /run/user/1000

In the case of the Raspberry Pi, there is a function to increase the SD card image in the management tool raspi-conifg. However, there is no such management tool in the Jetson series. Therefore, you must manually adjust the storage space using the resize2fs command. It is convenient to use the following script file to easily increase the storage space.

#!/bin/bash
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#  * Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
#  * Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#  * Neither the name of NVIDIA CORPORATION nor the names of its
#    contributors may be used to endorse or promote products derived
#    from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# This is a script to resize partition and filesystem on the root partition
# This will consume all un-allocated sapce on SD card after boot.

set -e

function cleanup()
{
	# Delete nvresizefs.sh, nvresizefs.service and its symlink
	# rm "/etc/systemd/nvresizefs.sh"
	# rm "/etc/systemd/system/nvresizefs.service"
	# rm "/etc/systemd/system/multi-user.target.wants/nvresizefs.service"
        echo 'clean up'
}

if [ -e "/proc/device-tree/compatible" ]; then
	model="$(tr -d '\0' < /proc/device-tree/compatible)"
	if [[ "${model}" =~ "jetson-nano" ]]; then
		model="jetson-nano"
	fi
fi

if [ "${model}" != "jetson-nano" ]; then
        echo 'if you are using Xavier NX, just annotate below commands'
       cleanup
        exit 0
fi

# Move backup GPT header to end of disk
sgdisk --move-second-header /dev/mmcblk0

# Get root partition name i.e partition No. 1
partition_name="$(sgdisk -i 1 /dev/mmcblk0 | \
	grep "Partition name" | cut -d\' -f2)"

partition_type="$(sgdisk -i 1 /dev/mmcblk0 | \
	grep "Partition GUID code:" | cut -d' ' -f4)"

partition_uuid="$(sgdisk -i 1 /dev/mmcblk0 | \
	grep "Partition unique GUID:" | cut -d' ' -f4)"

# Get start sector of the root partition
start_sector="$(cat /sys/block/mmcblk0/mmcblk0p1/start)"

# Delete and re-create the root partition
# This will resize the root partition.
sgdisk -d 1 -n 1:"${start_sector}":0 -c 1:"${partition_name}" \
	-t 1:"${partition_type}" -u 1:"${partition_uuid}" /dev/mmcblk0

# Inform kernel and OS about change in partitin table and root
# partition size
partprobe /dev/mmcblk0

# Resize filesystem on root partition to consume all un-allocated
# space on disk
resize2fs /dev/mmcblk0p1

# Clean up
cleanup

<nvresizefs.sh>

Now let's use a script to change the 64GB to use.

Boot using the restored 64GB SD card and run the script as follows.

spypiggy@spypiggy-nano:/usr/local$ sudo ./nvresizefs.sh 
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Setting name!
partNum is 0
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
resize2fs 1.44.1 (24-Mar-2018)
Filesystem at /dev/mmcblk0p1 is mounted on /; on-line resizing required
old_desc_blocks = 2, new_desc_blocks = 4
The filesystem on /dev/mmcblk0p1 is now 15626747 (4k) blocks long.

spypiggy@spypiggy-nano:/usr/local$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   59G   21G   36G  38% /
none            1.8G     0  1.8G   0% /dev
tmpfs           2.0G  4.0K  2.0G   1% /dev/shm
tmpfs           2.0G   27M  2.0G   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           397M     0  397M   0% /run/user/1000

You can see that the storage space of the SD card has increased to 64GB. The above script file can be downloaded from my GitHub.

Reusing SD Card

While using an SD card for the Jetson Nano, it may be necessary to create a new JetPack image. For example, if you are using JetPack 4.3 and a new JetPack 4.4 comes out, you may want to erase the existing SD card image and install a new 4.4. However, the SD card used in Windows may not be recognized even if it is connected to a USB device.

For reference, the Raspbian image used in the Raspberry Pi consists of two partitions. One is the ExFAT file system that Windows recognizes and contains important configuration information. And another partition is the Linux partition where the Raspbian OS is stored. The SD card used in the Raspberry Pi is immediately recognized by Windows because the ExFAT partition exists on the SD card. However, the JetPack image does not have an ExFAT partition. Therefore, the JetPack SD card is not recognized by Windows.

In this case, you can check the JetPack SD card using Windows 'Disk Management' and reformat it.

If your Windows system does not detect the JetPack SD card, open the 'Disk Management ' and you can see the SD Card. SD card was not damaged! It still exists healthy.

Select "Disk 2"
Click the right mouse button and choose "Delete volumes..." menu to delete the volume.
If the existing volume erased, click the right mouse button and choose "New simple volumes..." menu to create a volume.
Follow the on-screen instructions to create a volume with ease. When the Disk Management asks you to format the partition, select "do not format".

Now you can see the SD Card on the Windows explorer and burn a new Jetpack image using SD Card flashing tools like Etcher.

NVIDIA Jetson and Raspberry Pi

2019년 9월 27일 금요일

Useful tips before using Jetson Series(Nano, TX2, Xavier NX, Xavier)