Raspberry Pi Camera V2 connection
Raspberry Pi cameras connect using the Camera Serial Interface (CSI).
The video from JetsonHack can help you a lot. Make sure to turn off the Jetson Nano before connecting. When connecting, pay attention to the direction of the ribbon cable.
<Raspberry Pi Camera V2 and Jetson Nano>
After connecting, boot your Jetson Nano. Then check if the camera is connected with the following command. This command can check both USB camera and CSI camera.
In the previous post, I explained in detail how to check the camera.
root@spypiggy-nano:~# ls -al /dev/video* crw-rw----+ 1 root video 81, 0 2월 20 01:33 /dev/video0 root@spypiggy-nano:~# v4l2-ctl --list-devices vi-output, imx219 6-0010 (platform:54080000.vi:0): /dev/video0
You can check the camera stats on the Raspberry Pi website:
Raspberry Camera supports V4L2 driver, so you can use v4l2-ctl command to check the camera. The image sensor of Raspberry Pi camera V2 is "SONY IMX219", and you can find this in the ablove command output. So the Raspberry Pi camera is working correctly.
Test the camera
gstreamer usage
Jetson Nano uses a gstreamer to output camera input to the screen. We will test the camera using the gst-launch-1.0 program, the gstreamer test tool.
gst-launch passes the result of the plug-in corresponding to elements to the next element via a link.
For example, in the figure, Element1 can be either a camera plug-in or a source video file, and Element2 can be a plug-in that changes the size of the frame received through Element1, and mirrors front and back. And finally, Element3 can be a plug-in that receives the frame changed by Element2 and prints it to the screen or saves it to a file.
Test the Raspberry Pi camera V2
Run this command, you can see the camera video.root@spypiggy-nano:~#gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12＇ ! nvegltransform ! nveglglessink -e
This command has the following meaning:
Adjust camera settingsvi readme.txt
Perhaps the camera you see is upside down or left, right changed. In this case, you can do this by inserting a plug-in that controls flip.The NVIDIA proprietary nvvidconv Gstreamer-1.0 plug-in also allows you to perform video rotation operations
flip-method
|
Property value
|
no flip
|
0
|
counterclockwise - 90 degrees
|
1
|
rotate - 180 degrees
|
2
|
clockwise - 90 degrees
|
3
|
Before passing to the plug-in responsible for the display, you can add a flip plug in as follows:
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12＇ ! nvvidconv flip-method=1 ! nvegltransform ! nveglglessink -e
The NVIDIA proprietary nvvidconv Gstreamer-1.0 plug-in allows you to convert between OSS (raw) video formats and NVIDIA video formats. The nvvidconv plug-in currently supports the format conversions described in this section
Therefore, make sure to go through the nvvidconv plugin before passing it to the display like this.
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12＇ ! nvvidconv flip-method=0 ! nvvidconv !nvegltransform ! nveglglessink -e
Supported resolutions
When you run the program above, the modes supported by the current camera are displayed on the screen as follows.GST_ARGUS: Available Sensor modes : GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
Using Raspberry Pi camera V2 with python OpenCV
We are familiar with camera processing using OpenCV. So let's see how to manipulate the Raspberry Pi camera in OpenCV.Jetson Nano's OpenCV also supports cameras using gstreamer. Therefore, once you understand the pipeline described above, you can easily manipulate the Raspberry Pi camera in OpenCV.
# MIT License # Copyright (c) 2019 JetsonHacks # See license # Using a CSI camera (such as the Raspberry Pi Version 2) connected to a # NVIDIA Jetson Nano Developer Kit using OpenCV # Drivers for the camera and OpenCV are included in the base image import cv2 # gstreamer_pipeline returns a GStreamer pipeline for capturing from the CSI camera # Defaults to 1280x720 @ 60fps # Flip the image by setting the flip_method (most common values: 0 and 2) # display_width and display_height determine the size of the window on the screen def gstreamer_pipeline( capture_width=1280, capture_height=720, display_width=1280, display_height=720, framerate=60, flip_method=0, ): return ( "nvarguscamerasrc ! " "video/x-raw(memory:NVMM), " "width=(int)%d, height=(int)%d, " "format=(string)NV12, framerate=(fraction)%d/1 ! " "nvvidconv flip-method=%d ! " "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! " "videoconvert ! " "video/x-raw, format=(string)BGR ! appsink" % ( capture_width, capture_height, framerate, flip_method, display_width, display_height, ) ) def show_camera(): # To flip the image, modify the flip_method parameter (0 and 2 are the most common) print(gstreamer_pipeline(flip_method=0)) cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER) if cap.isOpened(): window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE) # Window while cv2.getWindowProperty("CSI Camera", 0) >= 0: ret_val, img = cap.read() cv2.imshow("CSI Camera", img) # This also acts as keyCode = cv2.waitKey(30) & 0xFF # Stop the program on the ESC key if keyCode == 27: break cap.release() cv2.destroyAllWindows() else: print("Unable to open camera") if __name__ == "__main__": show_camera()
The OpenCV installed on Jetson Nano is built to work with gstreamer, so the code above runs fine.
Note :
- The use of cv2.CAP_GSTREAMER as the second parameter in the cv2.VideoCapture function is optional. The second parameter can be omitted.
- In the gstreamer pipline string, last video format is "BGR", because the OpenCV's default color map is BGR.
- Notice that the last pipeline is appsink, not the screen output(nvegltransform ! nveglglessink -e).
Using Raspberry Pi camera and Webcam simultaneously
First connect the USB webcam to the Jetson Nano, and check the cameras.root@spypiggy-nano:/usr/local/src/study/csi_cam# ls -al /dev/video* crw-rw----+ 1 root video 81, 0 2월 20 01:33 /dev/video0 crw-rw----+ 1 root video 81, 3 2월 21 21:49 /dev/video1 root@spypiggy-nano:/usr/local/src/study/csi_cam# v4l2-ctl --list-devices vi-output, imx219 6-0010 (platform:54080000.vi:0): /dev/video0 UVC Camera (046d:0825) (usb-70090000.xusb-2.1): /dev/video1
Notice that the USB webcam is video1. I will compare two cameras at the same time.
# MIT License # Copyright (c) 2019 JetsonHacks # See license # Using a CSI camera (such as the Raspberry Pi Version 2) connected to a # NVIDIA Jetson Nano Developer Kit using OpenCV # Drivers for the camera and OpenCV are included in the base image import cv2 # gstreamer_pipeline returns a GStreamer pipeline for capturing from the CSI camera # Defaults to 1280x720 @ 60fps # Flip the image by setting the flip_method (most common values: 0 and 2) # display_width and display_height determine the size of the window on the screen def gstreamer_pipeline( capture_width=1280, capture_height=720, display_width=640, display_height=480, framerate=60, flip_method=0, ): return ( "nvarguscamerasrc ! " "video/x-raw(memory:NVMM), " "width=(int)%d, height=(int)%d, " "format=(string)NV12, framerate=(fraction)%d/1 ! " "nvvidconv flip-method=%d ! " "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! " "videoconvert ! " "video/x-raw, format=(string)BGR ! appsink" % ( capture_width, capture_height, framerate, flip_method, display_width, display_height, ) ) camSet='nvarguscamerasrc ! video/x-raw(memory:NVMM), width=3264, height=2464, format=NV12, framerate=21/1 ! nvvidconv flip-method=0 ! video/x-raw, width=640, height=480, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink' def show_camera(): # To flip the image, modify the flip_method parameter (0 and 2 are the most common) print(gstreamer_pipeline(flip_method=0)) #cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER) csi_cap = cv2.VideoCapture(camSet) web_cap = cv2.VideoCapture(1) web_cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640) web_cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480) if csi_cap.isOpened() and web_cap.isOpened(): while True: ret_valw, imgw = web_cap.read() ret_val, img = csi_cap.read() cv2.imshow("Web Camera", imgw) cv2.imshow("CSI Camera", img) # This also acts as keyCode = cv2.waitKey(30) & 0xFF # Stop the program on the ESC key if keyCode == 27: break csi_cap.release() web_cap.release() cv2.destroyAllWindows() else: print("Unable to open camera") if __name__ == "__main__": show_camera()
<csi_web_cam.py>
Run the code, you can see two windows for each camera.
Be careful : When I tested an old Logitech C270 Webcam, the Raspberry Pi camera output is delayed. I can't find the reason, but when I changed the webcam to the newest one, the delay has disappeared and two cameras were output at about the same time.
