What kind of development tools do you use?
Do you connect the keyboard and monitor directly to Jetson series, or do you develop the solution from a remote computer?

I develop it through remote access on my Windows 10 laptop unless it's a special occasion.
In addition, I use development tools sometimes cli editors such as vi, nano, etc., but use Notepad++ or VSCode more frequently.
These GUI development tools provide an expansion pack for editing and managing files on remote computers. In particular, VSCode, Microsoft's free editing framework, allows you to modify source code as well as remote GUI debugging. I think many people felt the need for debugging while developing machine learning tools such as OpenCV, Tensorflow, PyTroch, and TensorRT. I also often felt that I needed the ability to test and debug while editing the source code on a remote computer. I think VSCode is probably the best development tool for this purpose.

Install VSCode on your laptop

VSCode can be installed on Windows, Mac, and Linux. It can even be installed in Jetson Nano. I use Windows 10, but Mac and Linux users will be able to use it the same way.

Laptop with VSCode (Windows or Mac)

Install VSCode Extensions on your laptop

Install Python Extensions on your laptop

Run VSCode first, then press the Extensions search button.

Then search for Python and install it. Python extension is essential for developing Python in VSCode.

Install Remote Extensions on your laptop

Then install 'Remote Development Extension' to edit the source code of the remote Jetson series. When Microsoft first introduced the Remote Development Extension, it supported only the ssh key exchange method for remote access. I once posted on October 17, 2019 under the title of Remote Python Debugging with VSCode. This article explains how to connect to a remote computer using the ssh key exchange method. This method is still valid. However, not only ssh key exchange but also id/ password authentication is possible now.

Tips : In October 17, 2019 writings, VSCode Insider was used. VSCode Insider is an experimental version of the latest features. We used VSCode Insider because it was only a short time after the Remote Development Extensions came out. However, the Remote Development Extensions are now fully stabilized and can be used in normal VSCodes, not in the Insider version.

Search for and install Remote Development. This extension installs Remote -WSL, Remote-Contains, and Remote-SSH.

Setup Remote Jetson SSH connection

On the October 17, 2019 blog, the only way to access a remote server using ssh is to use an ssh key. However, this method has several drawbacks.

If the remote server information is changed, a new ssh key must be created. I mainly work on the Raspberry Pi and NVidia Jetson series. These single board computers use SD cards as storage space. Therefore, the SD card image is frequently replaced. When the SD card is replaced, the previously created ssh keys are no longer valid. Therefore, id/password authentication is often convenient.

Since VSCode is now able to access ssh in the id/password method, this disadvantage is eliminated. Now, the config file for ssh connection in VSCode was created as follows.

First, boot the Jetson Xavier to make it accessible.

Old Method (ssh key exchange)

Run VSCode and press Ctrl + Shift + P to search for the extension command. Then select Remote-SSH: Connect to Host ... Then select C: \ Users \ Users.ssh \ config. And add the following line: Modify the IP, User, and Private Key routes as appropriate. Refer to https://tipspiggy.blogspot.com/2019/10/remote-python-debugging-with-vscode.html for key generation method.

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host JetsonXavierNX
    HostName 192.168.11.96
    User spypiggy
    IdentityFile ~/.ssh/jumpbox

For instructions on how to create a Jumpbox file, see https://tipspiggy.blogspot.com/2019/10/remote-python-debugging-with-vscode.html.

New Method (id/password)

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host JetsonXavierNX
    HostName 192.168.11.96
    User spypiggy

Now let's try new method.

press Ctrl + Shift + P to search for the "Remote-SSH:Connect to host ..." command. Then select your host that you just configured.

Then you will probably be prompted for the user's password. If you enter the correct password, you can connect to the remote host.

If you press the Explorer button and then the Open Folder button, you can specify the working directory of the remote Jetson Xavier.

Install extensions on the remote host

Remote debugging requires that you install an extension that enables remote debugging on the remote computer . Press the Extensions button or Ctrl + Shift + P and search Python again. Earlier we installed the Python module. This time we see a new "Install in SSH: JetsonXavierNX" button. This means installing the VSCode Python extension on the remote host Xavier NX. Clicking this button installs a remote extension for Python debugging on the Remote Xavier NX.

Tips : Installation of Python extensions on remote Xavier NX requires a Remote-SSH connection first..

Install Python Extensions on remote Xavier NX

If you have never installed the VSCode Python extension on a remote Xavier NX, the following screen will appear:

If you select the Python extension in the VSCode window connected to the remote host, options for the remote host appear. Click "Install in SSH: JetsonXavierNX" to install the Python extension on the remote host.

At the end of the installation, you can verify that the Python extension is installed on the remote Xavier NX as follows:

After installation, reload the VSCode workspace. You are now ready for Python remote debugging.

Perhaps you will be asked to select the Python version to be used by the remote host as follows:

Do not use Python 2. I am using Python 3 virtual environment in Xavier NX in ~/python directory. Therefore, you will choose the last ~/ python/bin.

Then select New Terminal from the terminal menu.

When the terminal opens, you can see that the Python virtual environment is automatically run.

Code completion with IntelliSense

Most users who are familiar with IDE environments such as MS Visual Studio love to use code completion features. Code auto-completion not only speeds up development, but also greatly improves productivity by dramatically reducing the amount of typographical errors. Although the Linux development environment has some of these code autocompletion features, unlike the IDE on Windows and Mac, they often have limited functionality or require a paid product. In particular, remote development environments rarely provide this functionality. Using a remote control tool such as VNC is far from the remote development environment described in this article.

After installing the remote Python debugging module, VSCode can now write Python code that works on the remote computer using autocomplete. It is a new and powerful feature that you will never experience using the VI editor on a SSH terminal.

To use the code autocomplete feature in OpenCV, ...

I'm using Python virtual environment on Xavier NX. Packages such as Tensorflow installed in a virtual environment do not have problems using autocomplete feature. But OpenCV has some problems.
JetPack 4.4 is provided with OpenCV 4.1 as standard. The location where this package is installed is as follows.

(python) spypiggy@XavierNX:~/src$ python
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
cv>>> cv2.__file__
'/usr/lib/python3.6/dist-packages/cv2/python-3.6/cv2.cpython-36m-aarch64-linux-gnu.so'

>>> import tensorflow as tf
2020-09-02 10:30:14.846743: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
>>> tf.__file__
'/home/spypiggy/python/lib/python3.6/site-packages/tensorflow/__init__.py'

Since Tensorflow was installed in a virtual environment, packages exist in /home/spypyggy/python/lib/python3.6/site-packages.
However, the OpenCV provided by JetPack4.4 has an installation location of /usr/lib/python3.6/dist-packages/cv2/python-3.6/.

To apply autocomplete to packages installed outside of a virtual environment, some of the configuration files in VSCode need to be modified. The .vscode directory is created in the remote host's connection directory, which is accessed using Remote-SSH in VSCode, as illustrated. And the settings.json file is automatically created.

<settings.json file in the .vscode directory>

To enable IntelliSense for packages that are installed in other, non-standard locations, add those locations to the python.autoComplete.extraPaths collection in the settings file (the default collection is empty). Modify json file as follows. As you can see from the json file, Python.autoComplete.extraPaths can add multiple paths.

{
    "python.pythonPath": "/home/spypiggy/python/bin/python",
    "python.autoComplete.extraPaths": [
        "/usr/lib/python3.6/dist-packages/cv2/python-3.6"]
}

If you run VSCode again and test it, you can see that the automatic code completion function works in OpenCV.

Details can be found in the Editing Python in Visual Studio Code.

Tips : The above is the case when Python virtual environment is used. If you do not use a virtual environment and use /usr/bin/python3 as the default Python interpreter, you do not need to modify the settings.json file described earlier. However, there will be times when it will be useful to know the autocomplete features of VSCode through settings.json file modifications.

Remote Debugging

You can now set breakpoints by hovering your mouse next to the left number in the Python source code. Create a breakpoint at the desired location. Then press the debugger button and then the start debugger button. When asked what to debug, select Python File. This will automatically run the Python file in the current window. The program automatically stops at the breakpoint you specify. At this point, you can check the variable values with the mouse and monitor the variable values step by step using the left window.

Wrapping Up

The Python debugging function provided by VSCode is fantastic. Especially on your laptop, you can connect the remote Jetson series to do Python debugging using a GUI.
In particular, it can dramatically improve productivity because it can check the value of objects at any time while executing machine learning Python code.
VSCode's remote Python code autocomplete feature and GUI debugging are available on both Jetson Nano, TX2, and Xavier NX with the JetPack series installed.

NVIDIA Jetson and Raspberry Pi

2020년 9월 3일 목요일

Xavier NX - Remote Python Development using VSCode