By Aaron Behman, Director, Video & Vision, Corporate Strategy & Marketing Xilinx, Inc. And Dan Isaacs, Director, Industrial IoT, Corporate Strategy & Marketing Xilinx, Inc.
Broad IP, software, hardware and design services offerings enable smarter, connected, highly differentiated systems based on Xilinx All Programmable devices.
Systems with unprecedented levels of software-based intelligence, optimized hardware and any-to-any connectivity are shaping the future of embedded vision and the Industrial Internet of Things (IIoT). Xilinx® has strengthened and expanded the ecosystem that supports the development of IIoT and embedded vision systems based on Xilinx All Programmable devices, to make it easier for users to develop all manner of smarter, connected and highly differentiated systems.
The exciting applications emerging in industrial/ embedded vision and IIoT cut across the industrial, scientific, medical, pro A/V, consumer, aerospace and defense, and automotive market segments.
The key barrier to using the superior performance and performance/watt characteristics of Xilinx All Programmable devices has been the programming model.
With its ecosystem expansion, Xilinx is making its All Programmable devices just as easy to use as CPUs and GPUs, but with superior performance/watt.
C/C++ users are more accustomed to writing code for CPUs and, more recently, GPUs. With Xilinx’s Vivado® High-Level Synthesis (HLS) for software-defined hardware and SDx™ environment for software-defined systems development, many more system developers can make use of the software-defined All Programmable devices in the Xilinx Zynq-7000 SoC and Zynq UltraScale+ MPSoC families. With its ecosystem expansion, Xilinx is making its offerings just as easy to use as CPUs and GPUs, but with superior performance and performance/watt.
The pipelines for embedded vision and IIoT systems have much in common. Both start with sensing and data acquisition. For embedded vision systems, that data takes the form of a series of images or a video stream. Sensed data for IIoT systems includes video but also encompasses a long list of additional sensed parameters, including acceleration and vibration; acoustic/ultrasonic; chemical and gas; electric/magnetic; flow; force, load, torque and strain; humidity and moisture; leak and level; machine vision; optical; motion, velocity and displacement; position, presence and proximity; pressure; and temperature.
RISING NEED FOR SENSOR FUSION
Several embedded vision and IIoT systems require sensor fusion, or the processing and merging of data from multiple and different types of sensors into actionable intelligence. For embedded vision systems, multiple video streams may be combined to produce more usable or more useful video streams. For example, vehicle based vision systems often combine video streams from four, five, six or more video cameras to produce a single bird’s eye view that gives the driver 360° 2D planar or 3D spherical vision. Vision systems drive local displays but also send locally processed video to the cloud for further processing, for combination with other video streams and for storage.
IIoT systems may combine video with additional sensed data to define needed actions. For example, the new CPPS-Gate40 Smart Gateway from System-on-Chip engineering incorporates a variety of I/O ports commonly used in industrial control systems, combined with local, high-speed data processing, and places the resulting data on a dual-redundant optical Ethernet ring using High Availability Seamless Redundancy/ Parallel Redundancy Protocol (HSR/ PRP). A defining characteristic of IIoT systems is the ability to use sensed data for high-speed, real-time control not possible if relying on cloud-based processing and decision-making.
Of course, there are many alternative ways to design such systems using a CPU or GPU, but Xilinx Zynq-7000 SoCs and Zynq UltraScale+ MPSoCs offer several significant advantages and benefits when you’re designing differentiated systems:
Highest performance/watt. Xilinx All Programmable devices combine hardware, software and I/O programmability, letting you collapse a two-, three- or four-chip design into one chip to maximize system performance while lowering power consumption.
Sensor fusion. Xilinx All Programmable devices offer unique abilities to ingest and process multiple types of information, ranging from low-bit-rate data, such as temperature and pressure, to high-bit-rate data, including multiple, simultaneous high-definition or super-high-definition video streams.
Any-to-any connectivity. The programmable I/O capabilities of Xilinx Zynq-7000 SoCs and Zynq Ultra- Scale+ MPSoCs offer an unmatched ability to adapt to nearly any conceivable sensor I/O requirement, from multiple video interface standards (such as MIPI and HDMI) to intelligent sensor interfaces (such as I2C and SPI) and high-speed A/D converters (including JESD204B and LVDS).
Multilevel security and multilayer safety. The Zynq UltraScale+ MPSoC’s quad-core ARM® Cortex™-A53 application processor and dual-core ARM Cortex-R5 real-time processor with hardware security features offer unique abilities to implement security and functional-safety protocols.
“Chameleon” All Programmable platforms. The hardware and software processing and I/O flexibility of Zynq-7000 SoCs and Zynq UltraScale+ MPSoCs allow you to create reusable, software-defined hardware platforms with configurable and extensible range both up and down the end-product family cost curve, from low-cost to high-performance systems, and to extend a brand into new markets across multifunction product lines. This is not a hypothetical advantage; many Xilinx customers are already putting it into practice.
Here are four examples of Chameleon All Programmable platforms, all leveraging the Xilinx Zynq UltraScale+ MPSoC to target distinct markets.
EXAMPLE 1: ADVANCED DRIVER ASSISTANCE SYSTEM
An advanced driver assistance system (ADAS) combines video data from several video cameras and additional vehicle sensor data, including inertial navigation and even GPS map data, to make decisions about braking, steering and driver alerts. The block diagram in Figure 1 shows a typical ADAS design based on a Zynq UltraScale+ MPSoC.
As Figure 1 shows, this design leverages the heterogeneous processing capabilities of the quad-core ARM Cortex-A53 application processor and dual-core ARM Cortex-R5 real-time processor in the Xilinx Zynq UltraScale+ MPSoC. The five red boxes in the diagram depict MIPI video-interface IP available directly from Xilinx. The six blue boxes show high-speed IP processing blocks provided by other companies in the Xilinx ecosystem that implement high-level functions, including pedestrian detection, driver monitoring, lane departure monitoring, blind-spot detection and sensor fusion.
The depicted ADAS system takes full advantage of the Zynq UltraScale+ MP- SoC’s any-to-any connectivity to communicate with any sensor interface, including MIPI for the video cameras. Nonprogrammable devices from competing vendors cannot easily adapt to new sensor interfaces without adding I/O chips to handle the additional I/O interfaces and protocols. In addition, the Zynq UltraScale+ MPSoC’s superior hardware-based video-processing performance allows it to handle more video channels than competing standard devices. Unlike such devices, the Zynq UltraScale+ MPSoC handles a programmable number of video streams.
Because of the Zynq UltraScale+ MPSoC’s I/O flexibility and processing power, very little hardware is needed outside of the MPSoC itself, except for the sensors and external memory. The performance/watt metric for this system is approximately 3x better than for a comparable system using CPU-based silicon from a leading competitor.
EXAMPLE 2: 4K VIDEO SURVEILLANCE
A Zynq UltraScale+ MPSoC connects to multiple sensors, including different types of video cameras, in the 4K multichannel, multisensor video surveillance system shown in Figure 2. The red boxes in the block diagram again depict Xilinx interface IP for MIPI-interfaced video cameras and displays and for different I/O interfaces that connect other sensor types. The six all-blue boxes depict processing IP available from Xilinx ecosystem companies. The two red/blue boxes indicate IP blocks available both from Xilinx and from companies in its expanded ecosystem.
The performance/watt metric for this Chameleon All Programmable system is approximately 5x better than for a comparable system designed with CPU/DSP/GPU-based silicon from a leading competitor. The safety and security features of the Zynq UltraScale+ MPSoC, including ARM TrustZone® capabilities and the device’s hardware-based AES encryption, are especially useful in security applications like this one.
EXAMPLE 3: SMART-GRID SUBSTATION AUTOMATION
Our third example, a substation automation system targeting smart-grid design, is an IIoT application that deals with multiple Ethernet streams from various sensing components that monitor substation parametrics. A system block diagram for this Chameleon All Programmable system example appears in Figure 3.
A key feature of this example IIoT system is its ability to connect to a large number of interface units throughout the substation over standard industrial Ethernet systems using standardized IEEE-1588 Precision Timing Protocol (PTP) and IEC 62439 HSR/PRP. It does so through a compatible industrial Ethernet switch instantiated in the Zynq UltraScale+ MPSoC’s programmable logic using IP sourced from SoC-e, a company in the Xilinx ecosystem. This Ethernet switch appears as the large blue box in the diagram. Data from the various sensor sources can be processed in high-speed IP blocks (represented by the red/blue box in the diagram) from Xilinx and from companies in the Xilinx ecosystem, or the processing algorithms can run on one or more of the Zynq UltraScale+ MPSoC’s six ARM processor cores, depending on performance requirements.
The performance/watt metric for this system is approximately 1.2x better than for a comparable system based on CPU/DSP silicon from competitors, and there’s a 2:1 reduction in the number of chips needed in this design, thanks to the Zynq UltraScale+ MPSoC’s massive programmability, processing capacity and superior I/O flexibility. Clearly, a security application must protect the power grid from malicious attack, so the built-in functional-safety and security features of the Zynq UltraScale+ MPSoC are especially useful in this application.
With this expansion of its ecosystem, Xilinx has made it easier for product design teams to achieve aggressive project goals within equally aggressive project schedules.
EXAMPLE 4: INDUSTRIAL AUTOMATION
The final Chameleon All Programmable system example is for industrial control and might take the form of a motion controller, programmable logic controller (PLC) or human-machine interface (HMI) system. This IIoT example uses the Zynq UltraScale+ MPSoC to integrate an entire system that might otherwise require four chips (a CPU, a functional-safety processor, a shaft encoder and an FPGA for high-speed power modulation and motor control) into one device, resulting in a 30 percent improvement in performance/ watt and a substantial reduction in system pc board real estate. A system block diagram appears in Figure 4.
As in the three other examples, this industrial control system benefits from the Zynq UltraScale+ MPSoC’s any-to-any connectivity and from the functional-safety features embodied in the lockstep capabilities of the integrated dual-core ARM Cortex-R5 processor.
THE ECOSYSTEM LOWDOWN
All four of these examples make ample use of hardware and software IP from Xilinx and its ecosystem member companies. This IP is essential to easing your job of creating advanced, intelligent systems, especially Chameleon platforms that pick and choose which IP to use within each product built with one hardware platform.
Xilinx ecosystem members provide hardware and software IP in four major categories:
Domain-specific hardware and software IP for embedded vision and IIoT applications, plus a variety of real-time operating systems;
Design enablement, including several high-level design tools;
Modules, evaluation boards and production-ready systems-on-modules (SOMs) based on the Zynq-7000 SoC or the Zynq UltraScale+ MPSoC for rapid hardware development and proliferation; and Design services.
Every design team is pressed for time, even as project requirements entail ever-increasing performance and increasingly complex product features. No design team can do it all, quickly. With this newly announced expansion of its ecosystem, Xilinx has made it easier for product design teams to achieve aggressive project goals within equally aggressive project schedules.
To learn more about Xilinx’s expanded and strengthened ecosystem, please visit http://www.xilinx.com/ alliance/featured-solution-partners/ solutions-by-megatrend.html.