NVIDIA Jetson and Raspberry Pi: Camera - Search for supported resolutions of your webcam

There are many different types of webcams on the market. Therefore, knowing exactly what resolution your webcam supports will help you determine whether you can set the resolution you want. When implementing the edge-related Edge AI in the Jetson series, high resolution is not necessarily good. High resolution images not only require a lot of resources when importing images from the camera, but can also slow down processing in deep learning models. It is advisable to use the resolution recommended by your vision deep learning model. Therefore, you need to know if your webcam is capable of setting the resolution recommended by the deep learning model. This post will cover this.

Resolutions

The following table lists the usage share of display resolutions from several sources, as of May 2019. The numbers are not representative of computer users in general.

Standard	Aspect ratio	Width (px)	Height (px)	Steam^[11] (%)	StatCounter^[12] (%)
nHD	16:9	640	360	n/a	1.23
SVGA	4:3	800	600	n/a	0.64
XGA	4:3	1024	768	0.50	3.76
WXGA	16:9	1280	720	0.37	3.71
WXGA	16:10	1280	800	0.62	3.69
SXGA	5:4	1280	1024	1.78	3.52
HD	≈16:9	1360	768	1.70	1.69
HD	≈16:9	1366	768	11.39	24.55
WXGA+	16:10	1440	900	3.15	7.15
other	16:9	1536	864	n/a	6.08
HD+	16:9	1600	900	3.19	5.03
WSXGA+	16:10	1680	1050	2.32	2.56
FHD	16:9	1920	1080	64.50	19.70
WUXGA	16:10	1920	1200	0.85	1.16
QWXGA	16:9	2048	1152	n/a	0.42
other	≈21:9	2560	1080	1.02	n/a
QHD	16:9	2560	1440	4.71	1.82
other	≈21:9	3440	1440	0.59	n/a
4K UHD	16:9	3840	2160	1.61	n/a
Other				1.69	13.31

Find your Webcam

Most webcams released today support PnP, so they are automatically detected the moment you connect to the Jetson series. The following command will list the USB devices currently connected to the Jetson device.

root@spypiggy-desktop:~# lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 001 Device 004: ID 05a3:9230 ARC International
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I connected two USB webcams. One is Logitech C270, and the other is Chinese company Ailipu Technology's 2MP 1080p webcam. It is labeled ARC International in the above list. Two webcams are normally recognized as USB devices.

Linux exposes camera interfaces at /dev/video*. So check this directory.

root@spypiggy-desktop:~# ls -al /dev/video*
crw-rw---- 1 root video 81, 0  2월  6 21:49 /dev/video0
crw-rw---- 1 root video 81, 1  2월  6 21:50 /dev/video1

Yes, I can see 2 videos in that directory. These index numbers(0, 1) are very important later for OpenCV programming. Now you need to check whether index 0 is a Logitech webcam or another webcam. /dev/v4l/bu-id directory contains symbolic links that shows the information we need.

root@spypiggy-desktop:~# ls -al /dev/v4l/by-id
total 0
drwxr-xr-x 2 root root 80  2월  6 21:50 .
drwxr-xr-x 4 root root 80  2월  6 21:49 ..
lrwxrwxrwx 1 root root 12  2월  6 21:49 usb-046d_0825_6B9CA2D0-video-index0 -> ../../video0
lrwxrwxrwx 1 root root 12  2월  6 21:50 usb-HD_Camera_Manufacturer_USB_2.0_Camera-video-index0 -> ../../video1

You should now be able to find Index 0 as a Logitech webcam. So index 1 would be Ailipu Technology's webcam.

There is also a useful command called v4l2-ctl. To use this command, first install the v4l-utils.

root@spypiggy-nano:~# apt-get install v4l-utils
root@spypiggy-nano:~# lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 001 Device 003: ID 046d:c534 Logitech, Inc. Unifying Receiver
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
root@spypiggy-nano:~# v4l2-ctl --list-devices
UVC Camera (046d:0825) (usb-70090000.xusb-2.3):
        /dev/video0

Get list of supported resolutions of your Webcam

The resolution of the webcam can be obtained using various commands. Some of these commands are only available if certain packages are installed. You can use one of several methods.
If you pass bus and divice values with the -s option and -v(verbose), the lsusb command displays detailed information. These values have already been obtained above with the lsusb command. This command displays a lot of information. Several wWidth and wHeight values are displayed in the middle of the output value. These values list the resolutions supported by the webcam.

root@spypiggy-desktop:~# lsusb -s 1:3 -v

Bus 001 Device 003: ID 046d:0825 Logitech, Inc. Webcam C270
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x046d Logitech, Inc.
  idProduct          0x0825 Webcam C270
  bcdDevice            0.10
  iManufacturer           0
  iProduct                0
  iSerial                 2 6B9CA2D0
  bNumConfigurations      1

.................
.................

      VideoStreaming Interface Descriptor:
        bLength                            50
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         9
        bmCapabilities                   0x01
          Still image supported
        wWidth                        640
        wHeight                       360
        dwMinBitRate                 18432000
        dwMaxBitRate                110592000
        dwMaxVideoFrameBufferSize      460800
        dwDefaultFrameInterval         333333
        bFrameIntervalType                  6
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            400000
        dwFrameInterval( 2)            500000
        dwFrameInterval( 3)            666666
        dwFrameInterval( 4)           1000000
        dwFrameInterval( 5)           2000000

.................
.................

Use the egrep command together to select only the lines you need.

root@spypiggy-desktop:~# lsusb -s 1:4 -v |egrep "wWidth|wHeight"
        wWidth                           1920
        wHeight                          1080
        wWidth                           1280
        wHeight                           720
        wWidth                           1024
        wHeight                           768
        wWidth                            640
        wHeight                           480
        wWidth                            800
        wHeight                           600
        wWidth                           1280
        wHeight                          1024
        wWidth                            320
        wHeight                           240
        wWidth                           1920
        wHeight                          1080
        wWidth                           1280
        wHeight                           720
        wWidth                           1024
        wHeight                           768
        wWidth                            640
        wHeight                           480
        wWidth                            800
        wHeight                           600
        wWidth                           1280
        wHeight                          1024
        wWidth                            320
        wHeight                           240
        wWidth( 0)                       1920
        wHeight( 0)                      1080

If you use "bDescriptorSubtype" together, it's much easier to understand the output.

root@spypiggy-desktop:~# lsusb -s 1:4 -v |egrep "bDescriptorSubtype|wWidth|wHeight"
        bDescriptorSubtype      1 (HEADER)
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bDescriptorSubtype      6 (EXTENSION_UNIT)
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bDescriptorSubtype      5 (PROCESSING_UNIT)
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bDescriptorSubtype                  6 (FORMAT_MJPEG)
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                           1920
        wHeight                          1080
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                           1280
        wHeight                           720
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                           1024
        wHeight                           768
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                            640
        wHeight                           480
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                            800
        wHeight                           600
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                           1280
        wHeight                          1024
        bDescriptorSubtype                  7 (FRAME_MJPEG)
        wWidth                            320
        wHeight                           240
        bDescriptorSubtype                 13 (COLORFORMAT)
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                           1920
        wHeight                          1080
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                           1280
        wHeight                           720
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                           1024
        wHeight                           768
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                            640
        wHeight                           480
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                            800
        wHeight                           600
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                           1280
        wHeight                          1024
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        wWidth                            320
        wHeight                           240
        bDescriptorSubtype                  3 (STILL_IMAGE_FRAME)
        wWidth( 0)                       1920
        wHeight( 0)                      1080
        bDescriptorSubtype                 13 (COLORFORMAT)

You can also use a "v4l2-ctl" command .

root@spypiggy-nano:/usr/local/src/jetson-inference/python/examples# v4l2-ctl --list-devices
UVC Camera (046d:0825) (usb-70090000.xusb-2.3):
        /dev/video0

root@spypiggy-nano:/usr/local/src/jetson-inference/python/examples# v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.040s (25.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)
                        Interval: Discrete 0.067s (15.000 fps)
                        Interval: Discrete 0.100s (10.000 fps)
                        Interval: Discrete 0.200s (5.000 fps)
                Size: Discrete 160x120
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.040s (25.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)
                        Interval: Discrete 0.067s (15.000 fps)
                        Interval: Discrete 0.100s (10.000 fps)
                        Interval: Discrete 0.200s (5.000 fps)
                Size: Discrete 176x144
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.040s (25.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)
                        Interval: Discrete 0.067s (15.000 fps)
                        Interval: Discrete 0.100s (10.000 fps)
                        Interval: Discrete 0.200s (5.000 fps)
                Size: Discrete 320x176
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.040s (25.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)
                        Interval: Discrete 0.067s (15.000 fps)
                        Interval: Discrete 0.100s (10.000 fps)
                        Interval: Discrete 0.200s (5.000 fps)

                 ............

                 ............

Testing resolutions with OpenCV

Use OpenCV to check if the resolutions listed above are actually available.

VideoCapture function

To open a webcam or video stream, use the VideoCapture function. Depending on the parameters this function accepts, it works as follows:

VideoCapture ()

Default constructor. More...

VideoCapture (const String &filename)

Open video file or image file sequence or a capturing device or a IP video stream for video capturing. More...

VideoCapture (const String &filename, int apiPreference)

Open video file or a capturing device or a IP video stream for video capturing with API Preference. More...

VideoCapture (int index)

Open a camera for video capturing. More...

VideoCapture (int index, int apiPreference)

Opens a camera for video capturing. More...

Of these, the last two things to watch out for in order to use a webcam are:
In general, use the VideoCapture (int index) function, but you should also look at the following VideoCapture (int index, int apiPeference) function.

apiPreference parameter

This parameter is one of these values:

CAP_ANY Python: cv.CAP_ANY	Auto detect == 0.
CAP_VFW Python: cv.CAP_VFW	Video For Windows (platform native), == 200
CAP_V4L Python: cv.CAP_V4L	V4L/V4L2 capturing support via libv4l. == 200
CAP_V4L2 Python: cv.CAP_V4L2	Same as CAP_V4L. == 200
CAP_FIREWIRE Python: cv.CAP_FIREWIRE	IEEE 1394 drivers.
CAP_FIREWARE Python: cv.CAP_FIREWARE	Same as CAP_FIREWIRE.
CAP_IEEE1394 Python: cv.CAP_IEEE1394	Same as CAP_FIREWIRE.
CAP_DC1394 Python: cv.CAP_DC1394	Same as CAP_FIREWIRE.
CAP_CMU1394 Python: cv.CAP_CMU1394	Same as CAP_FIREWIRE.
CAP_QT Python: cv.CAP_QT	QuickTime.
CAP_UNICAP Python: cv.CAP_UNICAP	Unicap drivers.
CAP_DSHOW Python: cv.CAP_DSHOW	DirectShow (via videoInput) == 700
CAP_PVAPI Python: cv.CAP_PVAPI	PvAPI, Prosilica GigE SDK.
CAP_OPENNI Python: cv.CAP_OPENNI	OpenNI (for Kinect)
CAP_OPENNI_ASUS Python: cv.CAP_OPENNI_ASUS	OpenNI (for Asus Xtion)
CAP_ANDROID Python: cv.CAP_ANDROID	Android - not used.
CAP_XIAPI Python: cv.CAP_XIAPI	XIMEA Camera API.
CAP_AVFOUNDATION Python: cv.CAP_AVFOUNDATION	AVFoundation framework for iOS (OS X Lion will have the same API)
CAP_GIGANETIX Python: cv.CAP_GIGANETIX	Smartek Giganetix GigEVisionSDK.
CAP_MSMF Python: cv.CAP_MSMF	Microsoft Media Foundation (via videoInput) == 1400
CAP_WINRT Python: cv.CAP_WINRT	Microsoft Windows Runtime using Media Foundation.
CAP_INTELPERC Python: cv.CAP_INTELPERC	Intel Perceptual Computing SDK.
CAP_OPENNI2 Python: cv.CAP_OPENNI2	OpenNI2 (for Kinect)
CAP_OPENNI2_ASUS Python: cv.CAP_OPENNI2_ASUS	OpenNI2 (for Asus Xtion and Occipital Structure sensors)
CAP_GPHOTO2 Python: cv.CAP_GPHOTO2	gPhoto2 connection
CAP_GSTREAMER Python: cv.CAP_GSTREAMER	GStreamer. == 1800
CAP_FFMPEG Python: cv.CAP_FFMPEG	Open and record video file or stream using the FFMPEG library.
CAP_IMAGES Python: cv.CAP_IMAGES	OpenCV Image Sequence (e.g. img_%02d.jpg)
CAP_ARAVIS Python: cv.CAP_ARAVIS	Aravis SDK.
CAP_OPENCV_MJPEG Python: cv.CAP_OPENCV_MJPEG	Built-in OpenCV MotionJPEG codec.
CAP_INTEL_MFX Python: cv.CAP_INTEL_MFX	Intel MediaSDK.
CAP_XINE Python: cv.CAP_XINE	XINE engine (Linux)

Usually, if you omit this parameter value, the first CAP_ANY is used. In this case, select the appropriate parameter in OpenCV. In most cases it works fine without this parameter. Not all of these values can be used. You need to specify whether to include the CMake switch at OpenCV build time (eg : -DWITH_MSMF = ON -DWITH_VFW = ON ...)
In some cases, however, you may need to set this value yourself. Consider the following examples
I tested with the Logitech C270 webcam on Jetson Nano.

Jetson Nano (Jetpack 4.3) Example

import cv2
import sys, time
import argparse


parser = argparse.ArgumentParser(description='Webcam resolution Test')
parser.add_argument('--api', type=int, default=200, help='cv2.CAP_GSTREAMER=1800,  cv2.CAP_V4L2 = 200 ')
parser.add_argument('--width', type=int, default=1280)
parser.add_argument('--height', type=int, default=960)
args = parser.parse_args()

print('Webcam Test')
cap = cv2.VideoCapture(0,  args.api)
if (cap.isOpened() == False): 
    print("Unable to read camera feed")
    sys.exit(0)
ret = cap.set(cv2.CAP_PROP_FRAME_WIDTH, args.width)
print('apiReference[%d] WebCAM width  set :%d'%(args.api, ret))
ret = cap.set(cv2.CAP_PROP_FRAME_HEIGHT, args.height)
print('apiReference[%d] WebCAM height set :%d'%(args.api, ret))
ret, img = cap.read()
if ret == False:
    print('WebCAM Read Error')    
    sys.exit(0)
h, w, c = img.shape
print('Video Frame shape H:%d, W:%d, Channel:%d'%(h, w, c))


count = 1
while cap.isOpened():
    try:
        start = time.time()
        ret, img = cap.read()
        if ret == False:
            break
        count += 1
        cv2.waitKey(1)
        cv2.imshow('webcam', img)
        end = time.time()
        print('FPS : %f'%(1 / (end - start)))
        if count > 10:
            break
    except KeyboardInterrupt:
        print('Ctrl + C')
        break

print('Webcam Frame read End. Total Frames are : %d'%(count))
cv2.destroyAllWindows()
cap.release()

<webcam_capability.py>

I tested this program with cv2.CAP_GSTREAMER, cv2.CAP_V4L2 parameters. If I use cv2.CAP_GSTREAMER option, cap.set(cv2.CAP_PROP_FRAME_WIDTH, args.width) function fails, but the capture resolution has changed. If I use cv2.CAP_V4L2 option, cap.set(cv2.CAP_PROP_FRAME_WIDTH, args.width) function successes, and the capture resolution has also changed. This is a bit odd.

spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=1280 --height=960 --api=200
Webcam Test
apiReference[200] WebCAM width  set :1
apiReference[200] WebCAM height set :1
Video Frame shape H:960, W:1280, Channel:3
FPS : 5.295792
FPS : 14.249086
FPS : 7.164093
FPS : 7.888464
FPS : 6.887923
FPS : 7.774774
FPS : 7.695109
FPS : 7.614115
FPS : 7.541189
FPS : 7.332135
Webcam Frame read End. Total Frames are : 11
spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=1280 --height=960 --api=1800
Webcam Test
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
apiReference[1800] WebCAM width  set :0
apiReference[1800] WebCAM height set :0
Video Frame shape H:960, W:1280, Channel:3
FPS : 5.056150
FPS : 15.758937
FPS : 7.625869
FPS : 7.402074
FPS : 6.682095
FPS : 8.054770
FPS : 7.539793
FPS : 7.610565
FPS : 7.562755
FPS : 7.365483
Webcam Frame read End. Total Frames are : 11

This time, let's test using a resolution that is not supported.

spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=1288 --height=966 --api=200
Webcam Test
apiReference[200] WebCAM width  set :1
apiReference[200] WebCAM height set :1
Video Frame shape H:960, W:1280, Channel:3
FPS : 5.445239
FPS : 13.088222
FPS : 7.367735
FPS : 7.533780
FPS : 7.028237
FPS : 7.775639
FPS : 7.582731
FPS : 7.634516
FPS : 7.544106
FPS : 7.364164
Webcam Frame read End. Total Frames are : 11
spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=1288 --height=966 --api=1800
Webcam Test
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (515) startPipeline OpenCV | GStreamer warning: unable to start pipeline
apiReference[1800] WebCAM width  set :0
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1055) setProperty OpenCV | GStreamer warning: no pipeline
apiReference[1800] WebCAM height set :0
WebCAM Read Error
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

When using the cv2.CAP_V4L2 option, the cap.set (cv2.CAP_PROP_FRAME_WIDTH, args.width) function succeeds and sets the closest resolution even if you use a non-clearer resolution. However, when using the cv2.CAP_GSTREAMER option, an error occurs when using a resolution that is not supported. This is also a bit odd.

FPS Comparing

This time, use the cv2.CAP_V4L2 option to change the resolution and compare the speed of reading frames.

spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=1280 --height=960 --api=200
Webcam Test
apiReference[200] WebCAM width  set :1
apiReference[200] WebCAM height set :1
Video Frame shape H:960, W:1280, Channel:3
FPS : 5.474221
FPS : 13.026355
FPS : 7.411112
FPS : 7.641457
FPS : 6.995345
FPS : 7.628199
FPS : 7.724943
FPS : 7.595487
FPS : 7.562278
FPS : 7.386458
Webcam Frame read End. Total Frames are : 11
spypiggy@spypiggy-nano:/usr/local/src/study/webcam_test$ python3 webcam.py --width=640 --height=480 --api=200
Webcam Test
apiReference[200] WebCAM width  set :1
apiReference[200] WebCAM height set :1
Video Frame shape H:480, W:640, Channel:3
FPS : 8.379993
FPS : 88.221273
FPS : 14.520197
FPS : 14.759737
FPS : 13.776569
FPS : 14.644712
FPS : 15.116189
FPS : 14.696024
FPS : 14.573322
FPS : 14.823062
Webcam Frame read End. Total Frames are : 11

Reducing the capture resolution to half both horizontally and vertically increases the processing speed by about twice.

Wrapping up

We can see that the processing results are subtly different depending on the webcam's capture options. But I do not know the exact cause of this phenomenon. Knowing these subtle phenomena, you can adjust the webcam's resolution to suit your needs.

NVIDIA Jetson and Raspberry Pi

2020년 2월 7일 금요일

Camera - Search for supported resolutions of your webcam