README.rst
1.. zephyr:code-sample:: video-capture
2 :name: Video capture
3 :relevant-api: video_interface
4
5 Use the video API to retrieve video frames from a capture device.
6
7Description
8***********
9
10This sample application uses the :ref:`video_api` to capture frames from a video capture
11device then uses the :ref:`display_api` to display them onto an LCD screen (if any).
12
13Requirements
14************
15
16This sample needs a video capture device (e.g. a camera) but it is not mandatory.
17Supported boards and camera modules include:
18
19- `Camera iMXRT`_
20
21- :zephyr:board:`mimxrt1064_evk`
22 with a `MT9M114 camera module`_
23
24- :zephyr:board:`mimxrt1170_evk`
25 with an `OV5640 camera module`_
26
27- :zephyr:board:`frdm_mcxn947`
28 with any ``arducam,dvp-20pin-connector`` camera module such as :ref:`dvp_20pin_ov7670`.
29
30- :zephyr:board:`stm32h7b3i_dk`
31 with the :ref:`st_b_cams_omv_mb1683` shield and a compatible camera module.
32
33Also :zephyr:board:`arduino_nicla_vision` can be used in this sample as capture device, in that case
34The user can transfer the captured frames through on board USB.
35
36Wiring
37******
38
39On :zephyr:board:`mimxrt1064_evk`, the MT9M114 camera module should be plugged in the
40J35 camera connector. A USB cable should be connected from a host to the micro
41USB debug connector (J41) in order to get console output via the freelink interface.
42
43On :zephyr:board:`mimxrt1170_evk`, the OV5640 camera module should be plugged into the
44J2 camera connector. A USB cable should be connected from a host to the micro
45USB debug connector (J11) in order to get console output via the daplink interface.
46
47On :zephyr:board:`stm32h7b3i_dk`, connect the :ref:`st_b_cams_omv_mb1683` shield to the
48board on CN7 connector. A USB cable should be connected from a host to the micro USB
49connector in order to get console output.
50
51For :zephyr:board:`arduino_nicla_vision` there is no extra wiring required.
52
53Building and Running
54********************
55
56For :zephyr:board:`mimxrt1064_evk`, build this sample application with the following commands:
57
58.. zephyr-app-commands::
59 :zephyr-app: samples/drivers/video/capture
60 :board: mimxrt1064_evk
61 :shield: dvp_fpc24_mt9m114,rk043fn66hs_ctg
62 :goals: build
63 :compact:
64
65For :zephyr:board:`mimxrt1170_evk`, build this sample application with the following commands:
66
67.. zephyr-app-commands::
68 :zephyr-app: samples/drivers/video/capture
69 :board: mimxrt1170_evk/mimxrt1176/cm7
70 :shield: nxp_btb44_ov5640,rk055hdmipi4ma0
71 :goals: build
72 :compact:
73
74For :zephyr:board:`arduino_nicla_vision`, build this sample application with the following
75commands:
76
77.. zephyr-app-commands::
78 :zephyr-app: samples/drivers/video/capture
79 :board: arduino_nicla_vision/stm32h747xx/m7
80 :goals: build
81 :compact:
82
83For :zephyr:board:`frdm_mcxn947`, build this sample application with the following commands,
84using the :ref:`dvp_20pin_ov7670` and :ref:`lcd_par_s035` connected to the board:
85
86.. zephyr-app-commands::
87 :zephyr-app: samples/drivers/video/capture
88 :board: frdm_mcxn947/mcxn947/cpu0
89 :shield: dvp_20pin_ov7670,lcd_par_s035_8080
90 :goals: build
91 :compact:
92
93For :zephyr:board:`stm32h7b3i_dk`, build this sample application with the following commands,
94using the :ref:`st_b_cams_omv_mb1683` shield with a compatible camera module:
95
96.. zephyr-app-commands::
97 :zephyr-app: samples/drivers/video/capture
98 :board: stm32h7b3i_dk
99 :shield: st_b_cams_omv_mb1683
100 :goals: build
101 :compact:
102
103For testing purpose and without the need of any real video capture and/or display hardwares,
104a video software pattern generator is supported by the above build commands without
105specifying the shields, and using :ref:`snippet-video-sw-generator`:
106
107.. zephyr-app-commands::
108 :zephyr-app: samples/drivers/video/capture
109 :board: native_sim/native/64
110 :snippets: video-sw-generator
111 :goals: build
112 :compact:
113
114For controlling the camera device using shell commands instead of continuously capturing the data,
115append ``-DCONFIG_VIDEO_SHELL=y`` to the build command:
116
117.. zephyr-app-commands::
118 :zephyr-app: samples/drivers/video/capture
119 :board: mimxrt1064_evk
120 :shield: dvp_fpc24_mt9m114,rk043fn66hs_ctg
121 :gen-args: -DCONFIG_VIDEO_SHELL=y
122 :goals: build
123 :compact:
124
125For :zephyr:board:`stm32h7b3i_dk` with shell commands:
126
127.. zephyr-app-commands::
128 :zephyr-app: samples/drivers/video/capture
129 :board: stm32h7b3i_dk
130 :shield: st_b_cams_omv_mb1683
131 :gen-args: -DCONFIG_VIDEO_SHELL=y
132 :goals: build
133 :compact:
134
135Sample Output
136=============
137
138.. code-block:: console
139
140 Video device: csi@402bc000
141 - Capabilities:
142 RGBP width [480; 480; 0] height [272; 272; 0]
143 YUYV width [480; 480; 0] height [272; 272; 0]
144 RGBP width [640; 640; 0] height [480; 480; 0]
145 YUYV width [640; 640; 0] height [480; 480; 0]
146 RGBP width [1280; 1280; 0] height [720; 720; 0]
147 YUYV width [1280; 1280; 0] height [720; 720; 0]
148 - Default format: RGBP 480x272
149
150 Display device: display-controller@402b8000
151 - Capabilities:
152 x_resolution = 480, y_resolution = 272, supported_pixel_formats = 40
153 current_pixel_format = 32, current_orientation = 0
154
155 Capture started
156 Got frame 0! size: 261120; timestamp 249 ms
157 Got frame 1! size: 261120; timestamp 282 ms
158 Got frame 2! size: 261120; timestamp 316 ms
159 Got frame 3! size: 261120; timestamp 350 ms
160 Got frame 4! size: 261120; timestamp 384 ms
161 Got frame 5! size: 261120; timestamp 418 ms
162 Got frame 6! size: 261120; timestamp 451 ms
163
164 <repeats endlessly>
165
166If using the shell, the capture would not start, and instead it is possible to access the shell
167
168.. code-block:: console
169
170 uart:~$ video --help
171 video - Video driver commands
172 Subcommands:
173 start : Start a video device and its sources
174 Usage: start <device>
175 stop : Stop a video device and its sources
176 Usage: stop <device>
177 capture : Capture a given number of buffers from a device
178 Usage: capture <device> <num-buffers>
179 format : Query or set the video format of a device
180 Usage: format <device> <dir> [<fourcc> <width>x<height>]
181 frmival : Query or set the video frame rate/interval of a device
182 Usage: frmival <device> [<n>fps|<n>ms|<n>us]
183 ctrl : Query or set video controls of a device
184 Usage: ctrl <device> [<ctrl> <value>]
185 uart:~$
186
187
188References
189**********
190
191.. target-notes::
192
193.. _Camera iMXRT: https://community.nxp.com/t5/i-MX-RT-Knowledge-Base/Connecting-camera-and-LCD-to-i-MX-RT-EVKs/ta-p/1122183
194.. _MT9M114 camera module: https://www.onsemi.com/PowerSolutions/product.do?id=MT9M114
195.. _OV5640 camera module: https://cdn.sparkfun.com/datasheets/Sensors/LightImaging/OV5640_datasheet.pdf
196