In my previous post, "OpenCV camera control on Raspberry Pi BullsEye OS", I described the change of the Raspberry Pi's camera stack, which caused a lot of confusion. I also explained that OpenCV using GStreamer can be built and used successfully. OpenCV without GStreamer cannot be used with the new camera stack, libcamera. Therefore, camera control is not possible.
This article will take a look at which library is best to use for controlling the Raspberry Pi CSI camera from the point of view of a Python developer.
Choice 1. Restore to legacy mode
Legacy mode restoration is to restore the camera stack back to the Legacy stack as before BullsEye. Then there will be the following advantages.
- Existing camera control programs (raspistill, raspivid) can be used as they are.
- Python developers can use the existing Picamera package as is.
In other words, the developer can use the Raspberry Pi CSI camera without any changes as in the previous Buster version. The easiest and safest way. However, it is very likely that legacy mode restore will not continue to be supported. The legacy mode camera stack will probably disappear at some point in the future. Support for the current legacy mode restore feature is due to the sudden change of APIs. This is a way to reduce confusion for users and to guarantee the use of existing programs to some extent.
If you plan to use the current version without using the newly updated Raspberry Pi OS in the future, there is no problem, but if you plan to continuously improve the program in the new OS, this method is difficult to recommend.
Choice 2. Developing for the new camera stack(libcamera)
If you decide to use the new libcamera stack, let's figure out which python package to use. And let's look at the pros and cons of the package.
Picamera
Existing legacy camera stacks used the Picera package. However, the Piccamera package is no longer available in the libcamera stack.
import io import time import picamera import warnings warnings.filterwarnings('error', category=DeprecationWarning) with picamera.PiCamera() as camera: camera.resolution = (1280, 720) camera.framerate = 24 camera.start_preview() camera.preview.fullscreen = True camera.preview.alpha = 128 time.sleep(2) stream = io.BytesIO() camera.capture(stream, 'yuv', use_video_port=True)
<picamera peview example>
If you run the Python code above on the newly installed BullsEye (32-bit or 64-bit), an error occurs.
The reason this error message occurs is that as the camera stack is changed to libcamera in BullsEye, the picamera python package can no longer be supported, so it is no longer installed in the download image. If you change the camera stack to the legacy stack and then go back to libcamera, the error message may be different. In this case, the picamera package was installed while reverting to the legacy camera stack, but the error message is different because it is not available in the libcamera stack.
The conclusion is that the pycamera package is no longer available in the libcamera camera stack. So the Picamera package is no longer a consideration
Picamera2
Since the Picamera package was developed for the Legacy camera stack, a new Picamera package for the new libcamera camera stack is needed. The Raspberry Pi Foundation is currently developing this package under the name Picamera2.
As of March 2022, the preview version is available on Github(https://github.com/raspberrypi/picamera2).
The following is to save as a jpg file among the Picamera 2 preview version examples.
#!/usr/bin/python3 # Capture a JPEG while still running in the preview mode. When you # capture to a file, the return value is the metadata for that image. from picamera2.picamera2 import * import time picam2 = Picamera2() preview_config = picam2.preview_configuration(main={"size": (800, 600)}) picam2.configure(preview_config) picam2.start_preview(Preview.QTGL) picam2.start() time.sleep(2) metadata = picam2.capture_file("test.jpg") print(metadata) picam2.close()
<Picamera2 example>
Perhaps sooner or later, an official version of Picamera2 will be released, and the Raspberry Pi Foundation will recommend Python developers to use this package. But I wonder if I need to use Picamera2. This package is only available for Raspberry Pi CSI camera. We use the CSI camera on the Raspberry Pi as well, but we can also use the webcam. In particular, except for the CM (Computing Module), only one CSI camera can be used. Of course, if you use 3rd party products provided by Arducam, you can use multiple CSI cameras. But the easiest way is to use a CSI camera and webcam together. In this case, it is most convenient to control the CSI camera and webcam in one package.
Therefore I do not recommend the use of the picamera2 package. In fact, even before BullsEye, I seldom used the picamera package.
My conclusion is OpenCV.
OpenCV
If you properly use an image processing package such as Pillow together with OpenCV, you can have the following advantages.
- CSI cameras and webcams can be handled with the same API.
- Not only camera control, but also video control and photo control are possible.
- In addition to video and image recording, image format conversion and processing are possible.
- It has versatility that can be used not only on Raspberry Pi, but also on Linux, Windows, Mac OS and even smartphones.
- OpenCV is used for vision processing in most machine learning frameworks such as TensorFlow, Caffe, and PyTorch.
To use OpenCV with BullsEye's libcamera camera stack, you must change the build option to use GStreamer and then build a new one. Building OpenCV using GStreamer is described in detail in the previous article.
import cv2 import numpy as np import sys connstr = 'libcamerasrc ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! videoscale ! clockoverlay time-format="%D %H:%M:%S" ! appsink' cap = cv2.VideoCapture(connstr, cv2.CAP_GSTREAMER) if cap.isOpened() == False: print('camera open Failed') sys.exit(0) while True: succes, img = cap.read() if succes == False: print('camera read Failed') sys.exit(0) k = cv2.waitKey(1) if k == ord('q'): break cv2.imshow('Img',img) cap.release() cv2.destroyAllWindows()
<OpenCV example using GStreamer>
Migrate picamera source code for OpenCV
Let's implement some important functions of picamera to OpenCV.
Preview
Picamera implementation
from picamera import PiCamera from time import sleep f_res = None def set_resolution(ver, index): global f_res, camera res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)], "v2":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)] } f_res = res[ver][index] print("final resolution", f_res) camera.resolution = f_res camera = PiCamera() set_resolution("v1", 4) camera.start_preview(fullscreen=False,window=(0,0,f_res[0],f_res[1])) sleep(5) camera.stop_preview()
<picamera preview.py >
OpenCV implementation
import cv2 import numpy as np import sys import time connstr = None f_res = None def set_resolution(ver, index): global f_res, cap, connstr form = '''libcamerasrc ! video/x-raw, width={}, height={}, framerate=30/1 ! videoconvert ! videoscale ! appsink''' res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)], "v2":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)] } f_res = res[ver][index] connstr = form.format(f_res[0], f_res[1]) print("GStreamer PipeLine:", connstr) set_resolution("v1", 4) cap = cv2.VideoCapture(connstr, cv2.CAP_GSTREAMER) if cap.isOpened() == False: print('camera open Failed') sys.exit(0) start = time.time() while cap.isOpened(): _, img = cap.read() k = cv2.waitKey(1) if k == ord('q'): break cv2.imshow('Img',img) elapsed = time.time() - start if(elapsed > 5) : break cap.release() cv2.destroyAllWindows()
<opencv preview.py>
Take pictures
Picamera implementation
from picamera import PiCamera, Color from time import sleep f_res = None def set_resolution(ver, index): global f_res, camera res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)], "v2":[(3280,2464),(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)] } f_res = res[ver][index] print("final resolution", f_res) camera.resolution = f_res camera = PiCamera() set_resolution("v2", 0) # camera.annotate_background = Color('blue') # camera.annotate_foreground = Color('yellow') # camera.annotate_text = " Hello world " # camera.brightness = 70 camera.start_preview(fullscreen=False,window=(0,0,1280,960)) sleep(5) camera.capture('/home/pi/src/legacy/picamera_cap.jpg') camera.stop_preview()
<picamera saveimage.py >
OpenCV implementation
import cv2 import numpy as np import sys import time connstr = None f_res = None def set_resolution(ver, index): global f_res, cap, connstr form = '''libcamerasrc ! video/x-raw, width={}, height={}, framerate=30/1 ! videoconvert ! videoscale ! appsink''' res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)], "v2":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)] } f_res = res[ver][index] connstr = form.format(f_res[0], f_res[1]) print("GStreamer PipeLine:", connstr) set_resolution("v1", 4) cap = cv2.VideoCapture(connstr, cv2.CAP_GSTREAMER) if cap.isOpened() == False: print('camera open Failed') sys.exit(0) start = time.time() while cap.isOpened(): _, img = cap.read() k = cv2.waitKey(1) if k == ord('q'): break cv2.imshow('Img',img) elapsed = time.time() - start if(elapsed > 5) : cv2.imwrite('/home/pi/src/libcamera/picamera_cap.jpg', img) break cap.release() cv2.destroyAllWindows()
<opencv saveimage.py>
Save video
Picamera implementation
from picamera import PiCamera, PiCameraValueError from time import sleep f_res = None def set_resolution(ver, index): global f_res, camera res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(1024, 768),(800, 600),(640, 480)], "v2":[(3280,2464),(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(1024, 768),(800, 600),(640, 480)] } f_res = res[ver][index] print("final resolution", f_res) camera.resolution = f_res camera = PiCamera() set_resolution("v2", 3) camera.start_preview(fullscreen=False,window=(0,0,1280,960)) try: camera.start_recording('/home/pi/src/legacy/picamera.h264') sleep(5) camera.stop_recording() except PiCameraValueError as err: print("Picamera Err:", err) print("Please use another resolution") camera.stop_preview()
<picamera savevideo.py >
OpenCV implementation
import cv2 import numpy as np import sys import time connstr = None f_res = None def set_resolution(ver, index): global f_res, cap, connstr form = '''libcamerasrc ! video/x-raw, width={}, height={}, framerate=30/1 ! videoconvert ! videoscale ! appsink''' res = {"v1":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)], "v2":[(2592,1944),(1920, 1280),(1296, 972), (1296, 730),(640, 480)] } f_res = res[ver][index] connstr = form.format(f_res[0], f_res[1]) print("GStreamer PipeLine:", connstr) set_resolution("v1", 4) cap = cv2.VideoCapture(connstr, cv2.CAP_GSTREAMER) if cap.isOpened() == False: print('camera open Failed') sys.exit(0) fourcc = cv2.VideoWriter_fourcc('m', 'p', '4', 'v') out_video = cv2.VideoWriter('/home/pi/src/libcamera/picamera.mp4', fourcc, cap.get(cv2.CAP_PROP_FPS), f_res) start = time.time() while cap.isOpened(): _, img = cap.read() k = cv2.waitKey(1) if k == ord('q'): break cv2.imshow('Img',img) out_video.write(img) elapsed = time.time() - start if(elapsed > 5) : break cap.release() out_video.release() cv2.destroyAllWindows()
<opencv savevideo.py>
I have changed some frequently used functions in Picamera to OpenCV. Most camera control is possible with OpenCV using GStreamer.
Wrapping up
I'm not saying you shouldn't use picamera2. It is recommended to use OpenCV first for camera control. If you need to fine tune your camera , you may need picamera2. In other words, use picamera2 only when it is difficult to process with OpenCV. By the way, I haven't experienced anything like this yet.
OpenCV includes numerous functions such as camera control, image processing, and machine learning support. It provides rich features that cannot be compared with the picamera package, which can only control the CSI camera of the Raspberry Pi, and can be used on almost all platforms including Windows, Linux, Mac OS, and Android. In addition, various development languages such as Python, C++, and Java are supported. There are good reasons to consider using OpenCV over Picamera.
The source code can be downloaded from My Github.
댓글 없음:
댓글 쓰기