V4L2 Userspace Library - Revision history

CityK: rename file names (prep'ing for image merge)

2009-06-07T17:37:09Z

rename file names (prep'ing for image merge)

← Older revision		Revision as of 17:37, 7 June 2009
Line 37:		Line 37:
	A general overview of the thing:		A general overview of the thing:

-	[[Image:~~General~~.png]]	+	[[Image:V4L2_Userspace_Library_General.png]]

	As a consequence:		As a consequence:
Line 59:		Line 59:
	General use case view of the extended v4l driver:		General use case view of the extended v4l driver:

-	[[Image:~~Diagramme1~~.png]]	+	[[Image:V4L2_Userspace_Library_Diagramme1.png]]

	=== Open/close access the hardware ===		=== Open/close access the hardware ===
Line 78:		Line 78:
	# The extended device driver asks the base driver to unlock the device.		# The extended device driver asks the base driver to unlock the device.

-	[[Image:~~OpenCloseAccessTheHardware~~.png]]	+	[[Image:V4L2_Userspace_Library_OpenCloseAccessTheHardware.png]]

	'''Exceptions'''		'''Exceptions'''
Line 113:		Line 113:
	# If the v4l helper is here, see the extension point named "Provide compatible video formats". Otherwise, see alternative A1		# If the v4l helper is here, see the extension point named "Provide compatible video formats". Otherwise, see alternative A1

-	[[Image:~~ControlTheHardware~~.png]]	+	[[Image:V4L2_Userspace_Library_ControlTheHardware.png]]

	'''Alternatives'''		'''Alternatives'''
Line 147:		Line 147:
	# The application stops the acquisition with VIDIOC_STREAMOFF.		# The application stops the acquisition with VIDIOC_STREAMOFF.

-	[[Image:~~GetVideoFrames~~.png]]	+	[[Image:V4L2_Userspace_Library_GetVideoFrames.png]]

	'''Exceptions'''		'''Exceptions'''
Line 184:		Line 184:
	In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.		In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.

-	[[Image:~~UncompressFrames~~.png]]	+	[[Image:V4L2_Userspace_Library_UncompressFrames.png]]

	'''Pre-conditions'''		'''Pre-conditions'''
Line 197:		Line 197:
	# The extended v4l driver asks the v4l helper daemon to uncompress the newly available frame		# The extended v4l driver asks the v4l helper daemon to uncompress the newly available frame
	# The helper daemon answers as soon as the processing is done		# The helper daemon answers as soon as the processing is done
		+
	[[Category:Development]]		[[Category:Development]]

Jimbley: Added category: Development

2009-05-02T23:44:43Z

Added category: Development

← Older revision		Revision as of 23:44, 2 May 2009
Line 197:		Line 197:
	# The extended v4l driver asks the v4l helper daemon to uncompress the newly available frame		# The extended v4l driver asks the v4l helper daemon to uncompress the newly available frame
	# The helper daemon answers as soon as the processing is done		# The helper daemon answers as soon as the processing is done
		+	[[Category:Development]]

CityK: 6 revision(s)

2008-11-20T02:25:52Z

6 revision(s)

← Older revision

Revision as of 02:25, 20 November 2008

CityK: V4L2UserspaceLibrary moved to V4L2 Userspace Library: name

2008-11-20T02:18:30Z

V4L2UserspaceLibrary moved to V4L2 Userspace Library: name

← Older revision

Revision as of 02:18, 20 November 2008

Tmerle at 20:00, 20 August 2007

2007-08-20T20:00:48Z

← Older revision		Revision as of 20:00, 20 August 2007
Line 18:		Line 18:
	* Elegant, flexible.This is why I specify before coding :)		* Elegant, flexible.This is why I specify before coding :)

		+	== The ideal solution ==
		+	The ideal solution would be to create a v4l2 library provided to applications.
		+	They would access the v4l2 drivers via this library that would manage decompression if needed.
		+	If this library existed from the v4l project start, applications would have linked to it forcibly.
		+	The v4l project has a long history and it is more and more difficult to break compatibility now and force applications to link to a new library.

	== A solution ==		== A solution ==

Tmerle at 20:33, 28 June 2007

2007-06-28T20:33:26Z

← Older revision		Revision as of 20:33, 28 June 2007
Line 52:		Line 52:


-	~~[[Image:Exemple.jpg]]~~	+	General use case view of the extended v4l driver:
-	General use case view of the extended v4l driver	+

		+	[[Image:Diagramme1.png]]

	=== Open/close access the hardware ===		=== Open/close access the hardware ===
Line 72:		Line 72:
	# The extended device driver wipes out all internal allocations and asks the v4l helper to do so...		# The extended device driver wipes out all internal allocations and asks the v4l helper to do so...
	# The extended device driver asks the base driver to unlock the device.		# The extended device driver asks the base driver to unlock the device.
		+
		+	[[Image:OpenCloseAccessTheHardware.png]]

	'''Exceptions'''		'''Exceptions'''
Line 105:		Line 107:
	# The application negociates with the extended v4l driver the video data format, with VIDIOC_ENUM_FMT, VIDIOC_S_FMT.		# The application negociates with the extended v4l driver the video data format, with VIDIOC_ENUM_FMT, VIDIOC_S_FMT.
	# If the v4l helper is here, see the extension point named "Provide compatible video formats". Otherwise, see alternative A1		# If the v4l helper is here, see the extension point named "Provide compatible video formats". Otherwise, see alternative A1
		+
		+	[[Image:ControlTheHardware.png]]

	'''Alternatives'''		'''Alternatives'''
Line 137:		Line 141:
	# The VIDIOC_DQBUF/QBUF mechanism is repeated to get each frame sequentially.		# The VIDIOC_DQBUF/QBUF mechanism is repeated to get each frame sequentially.
	# The application stops the acquisition with VIDIOC_STREAMOFF.		# The application stops the acquisition with VIDIOC_STREAMOFF.
		+
		+	[[Image:GetVideoFrames.png]]

	'''Exceptions'''		'''Exceptions'''
Line 172:		Line 178:

	In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.		In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.
		+
		+	[[Image:UncompressFrames.png]]

	'''Pre-conditions'''		'''Pre-conditions'''

Tmerle at 20:17, 28 June 2007

2007-06-28T20:17:28Z

← Older revision		Revision as of 20:17, 28 June 2007
Line 29:		Line 29:
	For v4l drivers for hardware that can fullfill all application requested formats, this changes... nothing too.		For v4l drivers for hardware that can fullfill all application requested formats, this changes... nothing too.
	This architecture is interesting for v4l drivers that interface hardware that provide proprietary compressed video pictures or that are not able to fullfill the application needs by hardware.		This architecture is interesting for v4l drivers that interface hardware that provide proprietary compressed video pictures or that are not able to fullfill the application needs by hardware.
		+
	A general overview of the thing:		A general overview of the thing:
		+
		+	[[Image:General.png]]

	As a consequence:		As a consequence:
		+
	The only change for the application is only a new device to access (/dev/video1) so there is no extension of the v4l API/ABI.		The only change for the application is only a new device to access (/dev/video1) so there is no extension of the v4l API/ABI.
	The only change for the user is to launch another daemon in the starting scripts (/etc/rc.d/...). The v4l driver does not change, only extension points will be implemented if necessary.		The only change for the user is to launch another daemon in the starting scripts (/etc/rc.d/...). The v4l driver does not change, only extension points will be implemented if necessary.
		+
	The extended v4l driver is an instance of a v4l driver with some entry points for the helper daemon		The extended v4l driver is an instance of a v4l driver with some entry points for the helper daemon
-	Specification	+
		+
		+	== Specification ==
		+
	First, some obvious assumptions:		First, some obvious assumptions:
		+
	* The API available to the applications will be named v4l public API (headers in path /usr/include/linux/)		* The API available to the applications will be named v4l public API (headers in path /usr/include/linux/)
	* All extensions will be made by internal APIs. I will name it v4l extension private API (headers in path /usr/src/linux/drivers/media/video/...)		* All extensions will be made by internal APIs. I will name it v4l extension private API (headers in path /usr/src/linux/drivers/media/video/...)
Line 43:		Line 52:


		+	[[Image:Exemple.jpg]]
		+	General use case view of the extended v4l driver

-	~~== General use case view of the extended v4l driver ==~~

-		+	=== Open/close access the hardware ===
-	== Open/close access the hardware ==	+

	'''Summary'''		'''Summary'''
		+
	This step corresponds to the first and the last access to hardware via the extended driver.		This step corresponds to the first and the last access to hardware via the extended driver.
	Context		Context
Line 68:		Line 78:


-	== Control the hardware ==	+	=== Control the hardware ===

	'''Summary'''		'''Summary'''
		+
	This step corresponds mainly to the ioctl() entry points' behaviour.		This step corresponds mainly to the ioctl() entry points' behaviour.
	The extended v4l driver will have to interact with the v4l base driver and with the v4l helper daemon with specific ioctls.		The extended v4l driver will have to interact with the v4l base driver and with the v4l helper daemon with specific ioctls.
	The scenario will show the typical controls the application will address to the extended v4l driver.		The scenario will show the typical controls the application will address to the extended v4l driver.
	In most cases, the extended v4l driver will ask the base v4l driver to do the job. The v4l helper daemon will be used on specific control steps.		In most cases, the extended v4l driver will ask the base v4l driver to do the job. The v4l helper daemon will be used on specific control steps.
		+
	'''Context'''		'''Context'''
		+
	The application needs to get frames from the v4l peripheral, this task is fulfilled by the extended v4l driver with the help of the base v4l driver.		The application needs to get frames from the v4l peripheral, this task is fulfilled by the extended v4l driver with the help of the base v4l driver.
	'''Pre-conditions'''		'''Pre-conditions'''
		+
	The extended driver is opened properly on /dev/video1.		The extended driver is opened properly on /dev/video1.
		+
	'''Description'''		'''Description'''
		+
	# The application gets the device hardware capabilities with VIDIOC_QUERYCAP.		# The application gets the device hardware capabilities with VIDIOC_QUERYCAP.
	# The application enumerates inputs with VIDIOC_ENUMINPUT.		# The application enumerates inputs with VIDIOC_ENUMINPUT.
Line 91:		Line 107:

	'''Alternatives'''		'''Alternatives'''
		+
	A1 - the v4l helper daemon is not running (the /dev/v4lctl driver is not opened by the helper daemon)		A1 - the v4l helper daemon is not running (the /dev/v4lctl driver is not opened by the helper daemon)
	the extended v4l driver transmits the request to the base driver without interaction...		the extended v4l driver transmits the request to the base driver without interaction...


-	== Get video frames ==	+	=== Get video frames ===

	'''Summary'''		'''Summary'''
		+
	As the driver is correctly configured, the hardware captures and streams data from the device.		As the driver is correctly configured, the hardware captures and streams data from the device.
	This step will show how to managed the incoming data and provide them to the Application.		This step will show how to managed the incoming data and provide them to the Application.
Line 103:		Line 121:

	'''Pre-conditions'''		'''Pre-conditions'''
		+
	The extended v4l driver is properly configured on /dev/video1.		The extended v4l driver is properly configured on /dev/video1.
		+
	'''Description'''		'''Description'''

Line 119:		Line 139:

	'''Exceptions'''		'''Exceptions'''
		+
	EX1 The v4l helper daemon is not active or does not respond correctly to the extended v4l device driver request		EX1 The v4l helper daemon is not active or does not respond correctly to the extended v4l device driver request
	The extended v4l device driver returns -EFAULT (internal error) to the application, this ends the process.		The extended v4l device driver returns -EFAULT (internal error) to the application, this ends the process.

-	== Provide compatible video data formats ==	+	=== Provide compatible video data formats ===

	'''Summary'''		'''Summary'''
		+
	This step specifies how the extended v4l driver is able to provide to the application the hardware and software compatible video data formats		This step specifies how the extended v4l driver is able to provide to the application the hardware and software compatible video data formats
-	~~Context~~	+
	'''Pre-conditions'''		'''Pre-conditions'''
		+
	The v4l helper daemon has been registered		The v4l helper daemon has been registered
		+
	'''Description'''		'''Description'''

Line 139:		Line 163:


-	== Uncompress frames ==	+	=== Uncompress frames ===

	'''Summary'''		'''Summary'''
		+
	This part concerns the interactions with the userland v4l helper in order to uncompress frames.		This part concerns the interactions with the userland v4l helper in order to uncompress frames.
		+
	'''Context'''		'''Context'''
		+
	In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.		In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.
		+
	'''Pre-conditions'''		'''Pre-conditions'''
		+
	The extended v4l driver is properly configured on /dev/video1.		The extended v4l driver is properly configured on /dev/video1.
	The v4l helper is present and available (has answered correctly to all steps before)		The v4l helper is present and available (has answered correctly to all steps before)
	The extended v4l driver received a new compressed frame from the base v4l driver		The extended v4l driver received a new compressed frame from the base v4l driver
		+
	'''Description'''		'''Description'''

Tmerle at 20:09, 28 June 2007

2007-06-28T20:09:38Z

Show changes

Tmerle: New page: The need What do we need in userspace, only for v4l (dvb is not concerned): * particular video decompression algorithm like stk11xx or usbvision (the decompression algorithm is in ke...

2007-06-28T19:50:33Z

New page: The need What do we need in userspace, only for v4l (dvb is not concerned): * particular video decompression algorithm like stk11xx or usbvision (the decompression algorithm is in ke...

New page

The need

What do we need in userspace, only for v4l (dvb is not concerned):

* particular video decompression algorithm like stk11xx or usbvision (the decompression algorithm is in kernelspace since it is of linear complexity but shall be moved to userspace)
* colorspace translations
* filters that be done in hardware if the selected hardware can, otherwise software plugin

The general constraints

* Application shall still have access to the basic kernel driver. Thus, applications will have access to the 'raw' input data without software post-processing. Application developers love to develop optimized codecs :)
* The original v4l driver infrastructure should not change. We do not want to break any drivers. However, the extended v4l driver will use common functionalities like video_ioctl2 if this contributes to simplify the architecture.
* As a consequence, the v4l API does not change for applications.
* v4l driver and extended v4l driver shall mutually exclusive since they share the same hardware
* Limit as much as possible kernel memory and CPU consumption.
* Elegant, flexible.This is why I specify before coding :)

A solution

I thought it would be funny to create userspace drivers like FUSE based projects do.
Applications would see nothing but a new device driver to access, all decompression algorithm would be in userspace.
The path from the application to the v4l2 kernel driver:
[Application]<->[/dev/video1 (created by FUSE-like driver)]<->[/dev/video0 (created by kernel v4l2 driver)]
For applications, this changes nothing...
For v4l drivers for hardware that can fullfill all application requested formats, this changes... nothing too.
This architecture is interesting for v4l drivers that interface hardware that provide proprietary compressed video pictures or that are not able to fullfill the application needs by hardware.
A general overview of the thing:

As a consequence:
The only change for the application is only a new device to access (/dev/video1) so there is no extension of the v4l API/ABI.
The only change for the user is to launch another daemon in the starting scripts (/etc/rc.d/...). The v4l driver does not change, only extension points will be implemented if necessary.
The extended v4l driver is an instance of a v4l driver with some entry points for the helper daemon
Specification
First, some obvious assumptions:

* The API available to the applications will be named v4l public API (headers in path /usr/include/linux/)
* All extensions will be made by internal APIs. I will name it v4l extension private API (headers in path /usr/src/linux/drivers/media/video/...)
* I will name uncompressed frames formats the ones defined in the v4l public API.
* I will name compressed frames formats the ones not defined in the v4l public API.

General use case view of the extended v4l driver

Open/close access the hardware
Summary
This step corresponds to the first and the last access to hardware via the extended driver.
Context
The application opens the extended v4l driver. The scenario will assume that /dev/video0 is the base device entry point and /dev/video1 is the extended entry point.
Pre-conditions
Description

1. The application opens /dev/video1.
2. The extended device driver asks the base driver to lock the device.
3. The application controls/gets incoming frames from the hardware (see further).
4. The application closes /dev/video1.
5. The extended device driver wipes out all internal allocations and asks the v4l helper to do so...
6. The extended device driver asks the base driver to unlock the device.

Post-conditions
Exceptions

* At open, the base v4l driver is already locked. The extended v4l driver returns -EBUSY.

Control the hardware
Summary
This step corresponds mainly to the ioctl() entry points' behaviour.
The extended v4l driver will have to interact with the v4l base driver and with the v4l helper daemon with specific ioctls.
The scenario will show the typical controls the application will address to the extended v4l driver.
In most cases, the extended v4l driver will ask the base v4l driver to do the job. The v4l helper daemon will be used on specific control steps.
Context
The application needs to get frames from the v4l peripheral, this task is fulfilled by the extended v4l driver with the help of the base v4l driver.
Pre-conditions
The extended driver is opened properly on /dev/video1.
Description

1. The application gets the device hardware capabilities with VIDIOC_QUERYCAP.
2. The application enumerates inputs with VIDIOC_ENUMINPUT.
3. The application sets desired input with VIDIOC_S_INPUT.
4. The application enumerates supported video standards with VIDIOC_ENUMSTD.

For all these previous steps, the extended video driver transmits the request to the base driver without interaction.

5. The application negociates with the extended v4l driver the video data format, with VIDIOC_ENUM_FMT, VIDIOC_S_FMT.
If the v4l helper is here, see the extension point named "Provide compatible video formats".
Otherwise, see alternative A1

Post-conditions
Alternatives
A1 - the v4l helper daemon is not running (the /dev/v4lctl driver is not opened by the helper daemon)
the extended v4l driver transmits the request to the base driver without interaction...
Exceptions
Get video frames
Summary
As the driver is correctly configured, the hardware captures and streams data from the device.
This step will show how to managed the incoming data and provide them to the Application.
This is the most important part to understand from this specification.
Context
Pre-conditions
The extended v4l driver is properly configured on /dev/video1.
Description

1. The application requests the driver to allocate frames with VIDIOC_REQBUFS.
2. The extended v4l driver requests the v4l helper daemon to allocate video buffers
3. The application requests each allocated buffer characteristics with VIDIOC_QUERYBUF.
4. The application requests the driver to enqueue each frame with VIDIOC_QBUF.
5. The application starts the acquisition chain with VIDIOC_STREAMON.
6. The application waits and dequeues a frame with VIDIOC_DQBUF.
7. The extended v4l driver waits and dequeues a frame from the base v4l driver. If the frame needs to be uncompressed, see the extension point named "Uncompress frame".
Otherwise the extended v4l driver transmits the request to the base driver without interaction.
8. As soon as the frame is processed, the application calls VIDIOC_QBUF to re-enqueue this buffer.
9. The VIDIOC_DQBUF/QBUF mechanism is repeated to get each frame sequentially.
10. The application stops the acquisition with VIDIOC_STREAMOFF.

Post-conditions
Exceptions
EX1 The v4l helper daemon is not active or does not respond correctly to the extended v4l device driver request
The extended v4l device driver returns -EFAULT (internal error) to the application, this ends the process.
Provide compatible video data formats
Summary
This step specifies how the extended v4l driver is able to provide to the application the hardware and software compatible video data formats
Context
Pre-conditions
The v4l helper daemon has been registered
Description

1. Either the application has requested to enumerate all supported video formats,
1. The extended v4l driver ask the v4l helper daemon all possible formats for this hardware.
2. The v4l helper daemon provides all additional software video format.
2. Or the applications has requested to set a specific video format,
1. The extended v4l driver asks the v4l helper daemon if the requested video format is possible for this hardware
2. If the helper daemon is able to process by software the video format, it acknowledges with the negociated video format. Otherwise it returns a NACK

Post-conditions
Exceptions
Uncompress frames
Summary
This part concerns the interactions with the userland v4l helper in order to uncompress frames.
Context
In order to precise the context of a frame decompression, here is a collaboration overview of all components involved.
Pre-conditions
The extended v4l driver is properly configured on /dev/video1.
The v4l helper is present and available (has answered correctly to all steps before)
The extended v4l driver received a new compressed frame from the base v4l driver
Description

1. The extended v4l driver prepares data for the v4l helper daemon. This implies some generic preprocessing such as eventual memcopies.
2. The extended v4l driver asks the v4l helper daemon to uncompress the newly available frame
3. The helper daemon answers as soon as the processing is done

Post-conditions
Exceptions