ECE PhD Prospectus Defense: Shiva Raja

Title: Hardware Accelerators for Long Sequence AI Models

Presenter: Shiva Raja

Advisors: Professor Ajay Joshi, Professor Milos Popovic

Chair: Professor Martin Herbordt

Committee: Professor Ajay Joshi, Professor Milos Popovic, Professor Martin Herbordt

Google Scholar Link: https://scholar.google.com/citations?user=0S2UdP8AAAAJ&hl=en

Abstract: Understanding long sequences is essential for AI systems that process temporal data, such as speech, sensor signals, and text. While models like RNNs, CNNs, and Transformers have made strides in learning long-range dependencies, they often fall short on very long sequences due to fixed context limitations. State Space Models (SSMs) and biologically inspired SITH models offer better scalability by using continuous memory representations, but they require solving differential equations, making them computationally expensive to train and infer on conventional hardware like CPUs and GPUs.

To address this challenge, we propose a specialized hardware accelerator, EpochCore, to enhance the efficiency of SSM-based models such as S4 and Liquid-S4. EpochCore uses a systolic array architecture with a custom processing element, which supports standard and specialized matrix operations. Alongside this, we designed a novel dataflow that enables high-throughput, energy-efficient execution. The resulting system achieves high throughput and lower energy costs over traditional hardware, with modest area overhead.

Building on this, our ongoing work focuses on a unified accelerator for newer models like Mamba, GSS, and H3. It includes support for gating mechanisms, layer normalization, and a custom ISA for host communication, targeting FPGA/ASIC deployments for efficient, scalable long-sequence AI.