OpenVINO

Toolkit for deploying inference neural network model on Intel hardware

OpenVINO logo
Developer(s)Intel Corporation
Initial releaseMay 16, 2018; 6 years ago (2018-05-16)
Stable release
2024.1 / April 2024.[1]
Repositorygithub.com/openvinotoolkit/openvino
Written inC++
Operating systemCross-platform
LicenseApache License 2.0
Websitedocs.openvino.ai
As ofMay 2024

OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. It enables programmers to develop scalable and efficient AI solutions with relatively few lines of code. It supports several popular model formats[2] and categories, such as large language models, computer vision, and generative AI.

Actively developed by Intel, it prioritizes high-performance inference on Intel hardware but also supports ARM/ARM64 processors[2] and encourages contributors to add new devices to the portfolio.

Based in C++, it offers the following APIs: C/C++, Python, and Node.js (an early preview).

OpenVINO is cross-platform and free for use under Apache License 2.0.[3]

Workflow

The simplest OpenVINO usage involves obtaining a model and running it as is. Yet for the best results, a more complete workflow is suggested:[4]

  • obtain a model in one of supported frameworks,
  • convert the model to OpenVINO IR using the OpenVINO Converter tool,
  • optimize the model, using training-time or post-training options provided by OpenVINO's NNCF.
  • execute inference, using OpenVINO Runtime by specifying one of several inference modes.

OpenVINO model format

OpenVINO IR[5] is the default format used to run inference. It is saved as a set of two files, *.bin and *.xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application's API or a dedicated converter.

Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference.

The supported model formats are:[6]

  • PyTorch
  • TensorFlow
  • TensorFlow Lite
  • ONNX (including formats that may be serialized to ONNX)
  • PaddlePaddle

OS support

OpenVINO runs on the following desktop operation systems: Windows, Linux and MacOS.[7]

See also

References

  1. ^ "Release Notes for Intel Distribution of OpenVINO toolkit 2024.1". March 2024.
  2. ^ a b "OpenVINO Compatibility and Support". OpenVINO Documentation. 24 January 2024.
  3. ^ "License". OpenVINO repository. 16 October 2018.
  4. ^ "OpenVINO Workflow". OpenVINO Documentation. 25 April 2024.
  5. ^ "OpenVINO IR". www.docs.openvino.ai. 2 February 2024.
  6. ^ "OpenVINO Model Preparation". OpenVINO Documentation. 24 January 2024.
  7. ^ "System Requirements". OpenVINO Documentation. February 2024.
  • Agrawal, Vasu (2019). Ground Up Design of a Multi-modal Object Detection System (PDF) (MSc). Carnegie Mellon University Pittsburgh, PA. Archived (PDF) from the original on 26 January 2020.
  • Driaba, Alexander; Gordeev, Aleksei; Klyachin, Vladimir (2019). "Recognition of Various Objects from a Certain Categorical Set in Real Time Using Deep Convolutional Neural Networks" (PDF). Institute of Mathematics and Informational Technologies Volgograd State University. Archived (PDF) from the original on 26 January 2020. Retrieved 26 January 2020. {{cite journal}}: Cite journal requires |journal= (help)
  • Nanjappa, Ashwin (31 May 2019). Caffe2 Quick Start Guide: Modular and scalable deep learning made easy. Packt. pp. 91–98. ISBN 978-1789137750.
  • v
  • t
  • e
Intel software
Items in italics are no longer maintained or have planned end-of-life dates.
DevelopmentComponentsOpen sourceSoftware programs
  • Telekinesys Research1
    • Havok1
    • Vision1
Organizations