Embedded Camera API Standards Industry Exploratory Group Overview

Exploratory Group for Embedded Camera and Sensors

Press Release Presentation FAQ FAQ PDF Get Involved

Khronos and the EMVA are hosting an Embedded Camera API Exploratory Group to explore industry interest in the creation of open royalty-free API standards for controlling embedded cameras and sensors.

Open to all companies, universities, consortiums, and open-source participants at zero cost, participants will be enabled to discuss use cases and requirements for new interoperability standards to accelerate market growth and reduce development costs in embedded markets using vision and sensor processing and associated acceleration. If the Exploratory Group reaches significant consensus then Khronos and EMVA will work to develop the proposed standardization initiatives at the appropriate organization.

The Need for Embedded Camera API Standards

A pressing industry question arises whether innovation and efficiency in the embedded vision market is becoming constrained by the lack of open cross-vendor camera control API standards to reduce development and integration costs of multiple advanced sensors and cameras. Such API standards could potentially enable control of a wide range of camera sensors, depth sensors, camera arrays and ISP hardware to generate sophisticated image streams to increase the effectiveness of downstream local processing by diverse accelerators, as well as increasing application portability.

Increasingly, the emerging field of embedded vision is resulting in camera sensors being tightly integrated with image, vision, and inferencing accelerators in self-contained systems. Embedded vision applications on these integrated systems often lack a pervasively available API to portably generate sensor streams for local accelerated processing.

The Embedded Camera API Exploratory Group has been created in response to industry requests to address this and other interoperability issues slowing industry adoption of advanced embedded cameras and sensors.

“Judging by the significant industry interest, the time seems right to organize an effort around identifying and aligning on the need for interoperability APIs for embedded cameras and sensors. This is a topic that is very relevant to Khronos as our acceleration APIs, such as OpenCL™, SYCL™, and OpenVX™ are often used to accelerate sophisticated sensor stream processing. Our work is also very complementary to EMVA, and we are delighted that the two organizations are working together to bring a meaningful quorum from diverse parts of the industry into this cooperative exploratory process.”

Neil Trevett

Khronos Group President

“We are delighted to work with Khronos on this initiative to commonly understand the industry needs for the future of embedded vision. Both the EMVA and the Khronos group have a well-established history of standardization developments which enable industry to develop new products more simply, whilst ensuring friction is reduced in the market. This Exploratory Group is an excellent approach to understanding broader industry needs and will bring together many companies and views in an open forum. We look forward to working closely with the Khronos Group and welcoming all new and existing participants to this important initiative for the vision community.”

Dr. Chris Yates

EMVA President

“Embedded vision is a natural progression from full-sized PC-based vision systems to systems on a chip and is critically important to the future of the vision industry. The industry has seen great benefits from digital interface/interoperability standards such as GigE Vison and USB3 Vision in expanding markets, reducing costs, and simplifying technology application. It makes great sense to continue these standardization concepts at the embedded level.”

Jeff Burnstein

President of the Association for Advancing Automation (A3), parent association to AIA - Advancing Vision+Imaging

“Lack of API standards for advanced use of embedded cameras and sensors is an impediment to industry growth, collaboration and innovation. Enterprise AR customers and systems integrators/value added providers will benefit from greater clarity, open interfaces between modular systems and innovation in the component provider ecosystem. Standards for camera and sensor control will increase opportunities for powerful new combinations of sensor and AR compute resources, integration with existing IT, and lower cost and complexity of future solutions.”

Christine Perey

Interoperability and standards program leader for the Augmented Reality for Enterprise Alliance (AREA)

“The establishment of this Exploratory Group provides a great opportunity to connect with the Khronos Group, EMVA and industry partners to ensure that together we can create the best experience for embedded cameras on all Linux platforms. The Linux camera community has seen a need for standardisation and interoperability in the embedded camera space for more than a decade. We launched the libcamera project two years ago to address that need, initiating an ambitious effort to reach out to the industry and improve Linux camera support for mobile, embedded and desktop systems. We are eagerly looking forward to actively participating in the Exploratory Group and deepening our collaboration with all the involved parties.”

Laurent Pinchart

Lead architect of libcamera

Exploratory Group Process

This Exploratory Group will use Khronos’ proven New Initiative process, with the support and participation of the EMVA, to ensure that industry requirements are fully understood before starting detailed standardization work. The Exploratory Group has no predetermined outcome as it is too early to say what new camera and sensor APIs will be needed until industry requirements and use cases have been gathered and discussed.

All Exploratory Group participants will have an equal voice in exploring industry consensus to create a Statement of Work (SOW) document describing the objectives and high-level direction of standardization initiatives that would be of value to the industry. The Exploratory Group is expected to meet online over a period of several months. The Group is open to all proposed directions but will not discuss detailed technical design contributions to protect participants intellectual property (IP). All Exploratory Group discussion will be covered by a simple project NDA, to encourage open discussions.

If consensus on a Scope of Work can be achieved, the Exploratory Group will release the Scope of Work document from the project NDA and Khronos and EMVA will work to initiate the standardization work at the most suitable host organizations or open source projects, using those organizations’ normal collaborative agreements and IP frameworks. The contents of the Scope of Work document will determine whether EMVA or Khronos will be the appropriate organizations to undertake work items, or they would be better executed elsewhere.

Get Involved to Shape the Future of Embedded Camera Standards

Any company is welcome to join the new Embedded Camera API Standards Industry Exploratory Group with no cost or IP Licensing obligations. The Exploratory Group will enable industry dialog to seek consensus on whether the industry should cooperate to define open, royalty-free camera/sensor/ISP interoperability API standards. If a need is agreed, the Group will discuss what API(s) are needed, and how and where should the industry create them.

Exploratory Group meetings are expected to start in March 2021. To join please download and execute this NDA and email it to .(JavaScript must be enabled to view this email address).

Once your executed NDA is received, any employee of your company will be able to register and obtain a login to the Exploratory Group Portal. The Portal contains a shared document repository, a discussion mailing list and a Zoom meeting schedule. Anyone from your company is welcome to participate in Exploratory Group discussions.

Background to Khronos and EMVA

The EMVA (European Machine Vision Association) manages the GenICam standard for machine vision which is a widely used generic programming interface for industrial cameras that has become increasingly sophisticated as digital cameras integrate local processing capabilities. EMVA is also developing the emVision standard for embedded vision.

The Khronos Group is an open standards consortium that manages multiple API standards for vision, compute and inferencing acceleration such as OpenVX , OpenCL, SYCL, and Vulkan. Khronos has previously worked on the OpenKCAM API standard for camera control that was never publicly released. Elements of the OpenKCAM design may be re-used if they help meet the requirements in the agreed SOW.

Embedded Camera API Exploratory Group FAQ

Why should my organization care about this Exploratory Group?

Embedded cameras and sensors are becoming increasingly prevalent throughout industries for uses such as automotive, robotics, retail, medical, and manufacturing. The Exploratory Group will be investigating whether interoperability API standards can help remove industry friction that slows adoption and deployment, expanding opportunities for all involved in making and using advanced embedded cameras and sensors.

Who should join the Exploratory Group?

Any camera, sensor and silicon vendor, together with system integrators, and software developers with an interest in sensor processing applications.

Why an Exploratory Group – why not start designing an API right away to save time?

We want genuine consensus on what we SHOULD do before we actually DO it. Also, producing a Scope of Work document does not need detailed design contributions, protecting participant’s intellectual property, which in turn encourages a meaningful industry consensus through wider participation.

Who can join the Exploratory Group – and do they have to also join EMVA or Khronos?

Any entity willing to sign the Exploratory Group NDA is welcome. There is no cost and no obligation to join either EMVA or Khronos.

Why do you ask Exploratory Group participants to sign an NDA?

The Exploratory Group will be discussing materials that are publicly available, but ensuring discussions are kept within the Exploratory Group encourages a more open dialog. Any final Scope of Work documents produced by the Exploratory Group will be made publicly available.

Why are EMVA and Khronos doing this?

An increasing number of companies are asking for a standards-based solution to streamline use of cameras and sensors. Consequently, EMVA and Khronos have created a safe space for the industry to study this possibility. Both organizations see a growing willingness to consider cross-vendor standards - so we believe this may be the right time.

Are discussions limited to ‘Embedded Camera APIs’? What about other sensors and markets?

Embedded Camera APIs are the anticipated primary Exploratory Group focus, but requirements for other markets, applications and devices such as Lidar and other depth sensors, camera arrays and ISP hardware are relevant if there is participant interest.

Will any resulting API specifications be created at EMVA or Khronos?

We don’t know yet. If an agreed Scope of Work document is produced, whatever organization or open-source project that is best suited will be asked to host that work under their standard membership and IP frameworks. It may be EMVA, or Khronos or somewhere else entirely.

How is Exploratory Group participation organized logistically?

Once an organization signs the NDA, any employee with that email domain is able to register for an account to access the Exploratory Group portal. The portal facilitates communication by providing a repository to share and archive documents and an email reflector. All meetings will be virtual - no travel is required.

How often and long will the Exploratory Group meet?

We will host weekly Zoom calls beginning March 25^th, 2021. The number of calls will depend on how discussions progress. It is anticipated the Exploratory Group will meet for approximately 3 to 6 months. After the Exploratory Group reaches a recommendation, next steps will be determined.

With the diversity of participants, how will discussions be organized to achieve consensus?

Discussions will be in three broad phases:

Level Setting: All participants are invited to present on the current industry situation, pain points and key requirements;
Brainstorming: The Exploratory Group brainstorms on potential solutions to fill any identified standardization gaps;
Convergence: The Exploratory Group develops consensus and a Scope of Work document that would garner industry participation.

If the industry determines there is sufficient interest, how long would it take to get to an API specification?

It depends upon the Scope of Work, but historically standards industry associations typically need between 12 to 24 months to generate a V1.0 Specification.

What information should I share with the Exploratory Group?

Participants are welcome to share any background information that is relevant to the goals of the Exploratory Group - but it must be publicly available elsewhere to prevent any intellectual property issues.

May I invite colleagues from my company to the Exploratory Group?

Yes, any employee with an email address domain of an organization who has signed the NDA may join.

Last updated 16th March 2021

Related Discussions

Related Press

More Press Releases