Hello everyone,
I ran into a problem problem of low frame capture efficiency in OpenCV.
**Hardware & Software.**
- Raspberry Pi 3 (1,2 GHz quad-core
ARM) with HDMI Display
- IP camera: LAN
connected, RTSP, H264 codec,
1280x720 resolution, 20 fps, 1 GOP,
2500 kB/s VBR bitrate (parameters can
be changed).
- OS Raspbian Stretch
- Python 3.5
- OpenCV 4.1
- Gstreamer 1.0
**Task.**
**Get videostream from IP camera, recognize images and display resulting video (with marks and messages).**
Important features: real-time processing, HD resolution (1280x720), high frame rate (>20 fps), continuous operation for several hours.
**My solution.**
General algorithm: source video stream -> decoding and frame grabbing -> work with frames in OpenCV -> assembling the processed frames into a video stream -> display video using a Raspberry Pi GPU
OpenCV output/display method - **imshow - does not work well even at low-resolution video**. The only library that allows to use a Raspberry Pi GPU to decode and display video is a Gstreamer.
I compiled Gstreamer modules (gstreamer1.0-plugins-bad, gstreamer1.0-omx) with OMX support and tested it:
gst-launch-1.0 rtspsrc location='rtsp://web_camera_ip' latency=400 ! queue ! rtph264depay ! h264parse ! omxh264dec ! glimagesink
It works great, **CPU usage is about 9%**.
Next I compiled OpenCV with Gstreamer, NEON, VFPV3 support.
I use the following code for testing:
import cv2
import numpy as np
src='rtsp://web_camera_ip'
stream_in = cv2.VideoCapture(src)
pipeline_out = "appsrc ! videoconvert ! video/x-raw, framerate=20/1, format=RGBA ! glimagesink sync=false"
fourcc = cv2.VideoWriter_fourcc(*'H264')
stream_out = cv2.VideoWriter(pipeline_out, cv2.CAP_GSTREAMER, fourcc, 20.0, (1280,720))
while True:
ret, frame = stream_out.read()
if ret:
stream_out.write(frame)
cv2.waitKey(1)
It also worked, but not so well as Gstreamer itself. **CPU usage is about 50%**, **without stream_out.write(frame) - 35%**. At **frame rate above 15**, there are lags and delays.
**How I tried to solve the problem.**
A. Use Gstreamer to decode video stream:
pipline_in='rtspsrc location=rtsp://web_camera_ip latency=400 ! queue ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! appsink'
stream_in = cv2.VideoCapture(pipline_in)
It even worsened the situation - the **CPU load increased by several percent**, the delay has become more.
B. I also tried to optimize the library using [method](https://www.pyimagesearch.com/2015/12/21/increasing-webcam-fps-with-python-and-opencv/) from PyImageSearch.com - threading using WebcamVideoStream from imutils library.
from threading import Thread
import cv2
import numpy as np
import imutils
src='rtsp://web_camera_ip'
stream_in = WebcamVideoStream(src).start()
pipeline_out = "appsrc ! videoconvert ! video/x-raw, framerate=20/1, format=RGBA ! glimagesink sync=false"
fourcc = cv2.VideoWriter_fourcc(*'H264')
stream_out = cv2.VideoWriter(pipeline_out, cv2.CAP_GSTREAMER, fourcc, 20.0, (1280,720))
while True:
frame = stream_in.read()
out.write(frame)
cv2.waitKey(1)
**CPU usage has increased to 70%**, the quality of the output video stream has not changed.
C. Сhanging the following parameters does not help: whaitKey(1-50), videostream bitrate (1000-5000 kB/s), videostream GOP (1-20).
**Questions.**
As I understand, VideoCaputre/Videowritter methods has a very low efficiency. Maybe it's not noticeable on PC, but it is critical for Raspberry Pi 3.
**Is it possible to increase the performance of the VideoCaputre (Videowritter)?
Is there an alternative way to capture frames from video to OpenCV?**
Thanks in advance for answers!
↧