GPU Memory Offload for Affordable On-Premises LLM Training and Inference with Phison

In this 31-minute conference talk, discover Phison's aiDAPTIV+ technology that makes on-premises AI processing more affordable for small to medium-sized businesses, governments, and universities. Learn how their innovative approach offloads memory demands of large language models from GPUs onto flash storage, addressing the challenge of limited GPU memory capacity without requiring additional GPU purchases. Explore how this solution enables loading LLMs onto high-capacity, cost-effective flash memory while allowing GPUs to access data in slices for processing, significantly reducing costs compared to traditional deployments. The presentation covers Phison's partnerships with OEMs to integrate this technology across various platforms including desktops, laptops, and IoT devices, with pre-tested solutions for seamless user experience. Understand how Phison addresses the knowledge gap in LLM training through educational programs and university partnerships, including their teaching PC offered with Newegg to democratize AI education. The talk highlights how organizations can fine-tune pre-existing foundational models with domain-specific data while maintaining data privacy. Presented by Brian Cox, Product Marketing Director at Phison, recorded in Santa Clara on April 24, 2025, as part of AI Infrastructure Field Day.