LF AI & Data Foundation is proud to support the release of NNStreamer 2.0, one of our incubation-stage projects. NNStreamer is a set of Gstreamer plugins that support ease and efficiency for Gstreamer developers adopting neural network models and neural network developers managing neural network pipelines and their filters. Features of the new release include Edge-AI capabilities and new stream types. With this new release, NNStreamer 2.0 achieved the CII Best Practices passing badge last month. This recognizes that the community maintains high-quality code, documentation, testing, and of course, a high level of security as one of the best practices in Open Source. With the new release, NNStreamer now clears all of the 19 detected security vulnerabilities from LGTM analysis since this month.
Key Features of 2.0.y LTS
With Edge-AI capabilities, users may connect independent and remote pipelines with NNStreamer-provided protocols designed for AI data streams. In other words, a lightweight IoT device may offload its AI workloads to its neighbor devices with GStreamer/NNStreamer pipeline descriptions, and an AI service may publish its output data streams for other AI pipelines easily. Several features and optimizations are scheduled to follow with subsequent version releases.
As new stream types are introduced with this release, the “single tensor” stream type is becoming obsolete in NNStreamer 2.0, however compatibility will remain with possible warning messages in subsequent releases. The single tensor stream type with “other/tensor” MIME will be obsolete with the NNStreamer 2.0 release. Users are recommended to use “other/tensors” instead. The standard tensor data streams may have different data types: static (default), dynamic, and sparse. The conventional and default tensor streams are static. A dynamic tensor stream may have different dimensions with each data frame. A sparse tensor stream assumes that most elements are zeros. As new stream types are introduced with this release, the “single tensor” stream type is becoming obsolete in NNStreamer 2.0, however compatibility will remain with possible warning messages in subsequent releases. The single tensor stream type with “other/tensor” MIME will be obsolete with the NNStreamer 2.0 release. Users are recommended to use “other/tensors” instead. The standard tensor data streams may have different data types: static (default), dynamic, and sparse. The conventional and default tensor streams are static. A dynamic tensor stream may have different dimensions with each data frame. A sparse tensor stream assumes that most elements are zeros.
Other major features include:
- New hardware accelerators and AI frameworks supported: TVM, TensorRT, NNTrainer, Tensorflow-lite delegation (GPU/NNAPI/XNNPACK),
- Tensor-converter/decoder support subplugins and custom functions.
- New elements: stream branch (Tensor-if), stream join (Join), crop (Tensor-crop), rate and QoS control (Tensor-rate)
NNStreamer Key Links
LF AI & Data Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI & Data’s GitHub or Wiki
