Onboarding Generative AI on the Edge

Fatih Porikli

Australian National University, Qualcomm, AUS & US

Abstract

Generative AI is emerging as a transformative force, capable of creating multimodal content such as text, speech, images, video, 3D, and more. It can handle complex dialogues and reasoning about problems, reshaping traditional approaches across various domains and redefining the user interface to computing devices. This disruptive technology promises substantial advancements in utility, productivity, and efficiency across industries. However, as the adoption of generative AI accelerates, its computational demands are surging, making on-device processing more crucial than ever to improve efficiency and responsiveness. In this lecture, we will explore the pivotal role of on-device AI deployment and full-stack AI development. We aim to provide insights into the efforts needed to bring AI computation closer to the user. These efforts include architecture optimizations, model distillation, quantization, and pipelining on SoC accelerators. Join us to learn about these cutting-edge techniques and understand how they contribute to the future of AI on the edge.