Loading...
Loading...
Loading...
AINovember 27, 20245 MIN READ

AI Inference: Benefits of Using a Hybrid Cloud Solution

A security camera on an airport runway collecting security data for AI inference.

AI inference is the process by which an artificial intelligence (AI) model or algorithm uses its AI training to come to conclusions based on new data. As many organizations look to use vision-based AI inference as part of their digital transformation strategy, they are presented with a choice: 

  • Send video to the cloud for processing

or

  • Handle inference at the edge on AI-optimized PCs 

The question is - does it have to be a binary choice? What about a hybrid cloud solution? 

Explore AI Solutions

While cloud inference deployments can provide easy scalability, the cost of data transmission for video streaming from edge to cloud can be expensive, often too great for a viable return on investment. Inferencing in the cloud also increases latency, meaning it is harder to act on the inference data in real time. However, deploying an exclusively edge-located inference solution means that only local operators can take advantage of the data being generated. Enter hybrid cloud inferencing.

Hybrid-cloud inference

Inferencing and encoding video streams locally then sending optimized video feeds to the cloud enables quicker reactivity to events, lower transmission costs, and a scalable method for edge video inference. 

AWS, Intel, and OnLogic have collaborated to develop a solution that combines edge inferencing with the power of the AWS cloud.

A diagram highlighting solution architecture for edge video streaming and inferencing

Source: Intel

An example of edge video streaming and inferencing

To illustrate the concept, let’s see how a hybrid cloud solution could be applied to solve two common airport challenges: 

  • Ensuring safety compliance for ground operations

    • Fuel trucks parked in designated area

    • Employees wearing high-visibility vests

  • Monitoring runway perimeters for intrusion

    • Watching fences for wildlife or human intrusion

Handling the video stream

In both airport scenarios, the video streams are connected from cameras deployed where they are needed. Incoming compressed video streams are sent to the system and decoded, i.e. uncompressed from the incoming codec format, to prepare for inferencing with a supported video codec.

It would be important to use the correct type of camera for the application. For example with perimeter detection, a night vision compatible camera would be required since the deployment location may have minimal light during the night. Whereas at the gate, there is plentiful artificial light.

Video processing

The video frames are run through the AI inference algorithm including classification, object detection, and optical character recognition. This also overlays information such as bounding boxes and inference metadata to the video frames. Additionally, an action can be taken locally based on the information gained from inferencing.

Hardware for AI inference

Depending on the complexity of inferencing operations, workloads can use a great deal of hardware resources. This has been accomplished in the past with discrete GPUs, which excel at the computation for AI applications. With OpenVINO, Intel’s AI acceleration toolkit, you can take advantage of optimization for Intel CPUs, Iris Xe Integrated Graphics, and discrete Intel Arc GPUs to perform inference depending on project requirements.

For safety compliance:

  • Use object detection & classification to identify employees in frame and ensure that high visibility vests present in frame are overlapping with the employee.

  • Use object detection to identify the fuel vehicle and ensure it is parked in the correct location. 

    • This could also include optical character recognition to identify the hazard placard ensuring the correct type of fuel is being loaded into the aircraft. Ensuring only jet fuel is fueled into large aircraft instead of aviation gasoline.

For perimeter security:

  • Use object detection & classification to track and classify animals and people to ensure they don't enter or cross the perimeter.  

In both cases an important part of the AI application is tying the inference results into the action needed to resolve the issue shown. Depending on the severity this may simply require logging the event or taking immediate action to rectify the situation.

  • High Visibility Vests: Log a video snippet/frame showing infraction and notify ground operations manager.

  • Fuel Vehicle: Log a record of the incorrect parking for later review. Use smart controllers on the fuel vehicles to stop the fuel flow if an incorrect type of fuel is about to be used.

  • Perimeter Security: Play loud noises or turn on a light if something is coming towards the fence to dissuade unintended intrusion. If intrusion does occur, notify security and air traffic control immediately and sound alarms.

As mentioned above, these different AI operations have different hardware requirements and deployment locations. The ground operations use-case could centralize systems on edge servers in a controlled environment. For perimeter detection, small edge devices could be used to sit directly next to cameras. In that case cellular connectivity might be needed as hardwired network connectivity may not be available.

Offline storage vs. cloud availability

An important decision to make as part of a large video streaming solution is what information needs to be sent to the cloud vs. stored offline. 

  • For streams that need to be continuously monitored, it is helpful to serve those to users in the cloud. 

  • For event logging use-cases, it can be enough to store video streams on-site for a few days before deletion and only send to the cloud key video segments when an event occurs.

AV1 encoding & KVS formatting

To create highly efficient streams, our example solution would use the AV1 video codec, a highly efficient format ideal for streaming to AWS Cloud. Intel processors and GPUs contain optimizations for the AV1 codec. 

Additionally, formatting to the AWS Kinesis Video Streams (KVS) format allows for easy ingestion directly to the AWS Cloud. KVS is an AWS service built for securely streaming video back from edge devices for usage with other AWS services. By creating these highly efficient streams, data transmission costs from edge to cloud, especially from 4G/5G connected devices, are significantly reduced.

In the AWS cloud

Once streams are sent to the cloud, the streams can easily be served to other applications or streamed to relevant users using KVS. For perimeter security, this can be streamed directly to the security team for constant monitoring, while the safety team can access logged video streams to see events and correct issues.

Hybrid cloud inference: a powerful solution for vision-based AI

In conclusion, the hybrid cloud approach to AI inference offers a powerful solution for organizations seeking to optimize their vision-based processes. By leveraging the strengths of both edge and cloud computing, this solution enables real-time responsiveness, cost-effective data transmission, and scalability. 

Getting started

Ready to get started? Through collaboration with industry leaders like AWS and Intel, OnLogic can help you unlock the full potential of your data. Explore OnLogic’s AI Solutions including consulting services by clicking the button below. 

Explore AI Solutions

Related OnLogic Content

What is OpenVINO and What are the Hardware Requirements?

3 Reasons you Need Wireless Communication for Edge Computers

What is a GPU?

AWS Compatible Gateways

OnLogic Edge Servers

Getting started with AI Inference

References

KVS Edge Deployment

AWS Marketplace

Intel

Cole Wangsness
Cole Wangsness
Sales Engineer, OnLogic
Cole Wangsness is a Solution Architect at OnLogic. Since joining the team in 2018, Cole has focused on empowering companies to solve complex technology challenges at the edge. Through his work with customers, partners, vendors and the team at OnLogic, Cole gets to feed his passion for making complex technologies easier to understand and deploy.

Stay in the know.

Get first access to the newest products and Industry 4.0 innovation, OnLogic Live! events and more.