AIMarch 19, 20254 MIN READ

Unleashing the Power of AI in Healthcare

A stethoscope rests on a circuit board, symbolizing the intersection of healthcare and technology.

Artificial intelligence (AI) in healthcare has the potential to fundamentally transform the practice of medicine and the delivery of medical services according to the National Institutes of Health (NIH)*. Benefits are wide ranging from enhanced patient care, to accelerated drug development, and even improved administrative efficiency.

However, secure access to sufficient computing power to generate decisions in real-time has slowed the adoption of AI in healthcare. That is changing with technological advances in edge computing.

What is the edge?

The edge is part of a distributed computing environment where processing power is placed close to the data source, allowing for real-time analysis and faster response times, as opposed to sending data to a remote cloud server.

The edge is no longer just a network boundary; it's a critical frontier for innovation, especially in sectors like healthcare. For doctors, hospital administrators, and technology officers, understanding the potential of AI at the edge is crucial for improving patient care and optimizing operations.

To tackle the challenge of bringing AI to resource-constrained and highly regulated edge environments, industry leaders OnLogic and Red Hat have partnered to develop a cutting-edge solution with real-world implications for the healthcare industry.

The AI challenge in healthcare

Traditional AI deployments often relied on powerful cloud infrastructure which presents significant hurdles for healthcare applications. Major roadblocks include latency, bandwidth limitations, connectivity issues, and most importantly, strict HIPAA regulations for data privacy and security.

The solution - AI at the edge

This collaborative solution from OnLogic and Red Hat confronts these challenges head-on, proving that training and deploying AI models at the edge within the secure hospital environment is not only feasible but also highly advantageous for healthcare providers. This joint solution offers the following benefits:

Reduces resource constraints: Hospitals often have limited compute power and storage at the edge. The solution leverages Single Node OpenShift and MicroShift, Red Hat's lightweight Kubernetes distributions, to optimize resource utilization and enable model training on OnLogic’s powerful industrial edge hardware.
Enhances connectivity: Real-time access to information is crucial for clinical decisions and the patient experience, even when network connectivity is disrupted. The solution demonstrates how to train, deploy, and manage AI models locally, shrinking reliance on cloud connectivity for critical operations.
Promotes data security and reduces latency: Bringing AI to the edge minimizes data transfer, enhancing security and fostering HIPAA compliance. It also reduces latency for real-time decision-making, which is paramount for healthcare professionals.

AI at the edge in action for healthcare

The OnLogic and Red Hat solution showcases a fine-tuned LLM, optimized for medical applications, running directly on edge hardware. We use Single Node OpenShift to enable model training (on a separate, more powerful machine within the hospital) and MicroShift for running containerized workloads and inference on small edge devices without cloud dependency.

The core functionality revolves around Retrieval Augmented Generation (RAG): in combination with RAG, the LLM analyzes doctor queries, retrieves relevant information from anonymized patient data and medical research, and generates personalized treatment suggestions.

Hardware's crucial role in healthcare AI

At the edge, hardware isn't just a component; it's the foundation for secure and reliable AI in healthcare. The Axial AX300 edge server from OnLogic, experts in industrial computing hardware, delivers the robust and reliable edge platform this solution requires.

OnLogic Axial AX300 edge server in a warehouse

The processing happens entirely on the AX300 edge server within the hospital network. While the solution focuses on a disconnected (from the outside world) environment for HIPAA compliance, the architecture can easily integrate with internal hospital systems for seamless access.

The AX300 edge server is a purpose-built machine designed to thrive in demanding edge environments. Its robust features, including support for multiple high-performance GPUs, ample storage, and powerful processing capabilities, are essential for handling complex AI workloads. The AX300's rugged design, advanced thermal management, and flexible deployment options ensure reliability and adaptability in a hospital environment.

Explore the AX300 Series

The solution setup

Tech stack: RHEL, Single Node OpenShift, MicroShift, Red Hat OpenShift AI with GPU, PyTorch, Jupyter Notebooks, potentially specialized medical NLP libraries, a fine-tuned LLM.
Model training: The LLM is trained on Red Hat OpenShift AI using Jupyter Notebooks and a curated dataset of medical information. The training may also occur on a more powerful machine within the hospital network.

The solution uses a comprehensive dataset of anonymized patient data and medical literature for training the LLM. Key metrics for LLM performance are tracked during training. The "wow" moment comes from witnessing doctors asking complex questions about patient cases and receiving detailed, evidence-based suggestions in seconds, all while ensuring data privacy.

Overcoming challenges and scaling up in healthcare

Deploying AI at the edge in healthcare presents challenges, primarily resource limitations, connectivity issues (internal network reliability), and stringent security and compliance requirements. This solution from OnLogic and Red Hat addresses these by utilizing lightweight Kubernetes distributions, optimizing hardware usage, and adhering to strict data governance policies.

Scalability is a key advantage. As the hospital system's data volume and usage grow, they can easily add more servers to the edge network, distributing the workload and maintaining performance and compliance.

Conclusion - leveraging AI in healthcare

This AI at the edge solution showcases a paradigm shift in how we approach AI deployments in healthcare. By bringing the entire AI implementation to the edge, within the secure hospital environment, we unlock new possibilities for real-time insights, enhanced patient care, and groundbreaking medical research.

RHEL Certified Hardware Solutions