Spatten Efficient Sparse Attention Architecture

June 14, 2026
Media Summary: Presentation at HPCA 2020 on paper "SpArch: Project & Seminar, ETH Zürich, Spring 2022 Hands-on Acceleration on Heterogeneous Computing Systems ... ISCA'23: The 50th International Symposium on Computer

Spatten Efficient Sparse Attention Architecture - Detailed Analysis & Overview

Presentation at HPCA 2020 on paper "SpArch: Project & Seminar, ETH Zürich, Spring 2022 Hands-on Acceleration on Heterogeneous Computing Systems ... ISCA'23: The 50th International Symposium on Computer Presented by Nitish Srivastava at HPCA2020, San Diego, CA, United States. Abstract: Tensor factorizations are powerful tools in ... This video was presented at PyData Sofia Online Meetup on 30.04.2020. Full Stream: ... Providing reliability is becoming a challenge for chip manufacturers, faced with simultaneously trying to improve miniaturization, ...

A video presentation of Kyuhan Lee, Hyeonsoo Jo, Jihoon Ko, Sungsu Lim, Kijung Shin, "SSumM:

Photo Gallery

Short Intro for HPCA'21 SpAtten: Efficient Sparse Attention Architecture by Hanrui Wang
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning, [HPCA 2021]
Chip Demo for "SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning"
Is Sparse Attention more Interpretable?
DeepSeek Sparse Attention Explained: 80% Cheaper Long-Context AI
Hanrui Wang's Talk at HPCA'20 on "SpArch: Efficient Architecture for Sparse Matrix Multiplication"
Arxiv 2021: Sparse attention Planning
HetSys Course: Lecture 10: Parallel Patterns: Sparse Matrices (Spring 2022)
ISCA'23 - Lightning Talks - Session1A - FACT: FFN-Attention Co-optimized Transformer Architecture wi
[HPCA'20] Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations
Sparse Transformers - Tsvetomila Mihaylova [PyData Sofia April 2020]
ParaDox: Eliminating Voltage Margins via Heterogeneous Fault Tolerance (Long Talk, HPCA 2021)

Related Security Resources

Short Intro for HPCA'21 SpAtten: Efficient Sparse Attention Architecture by Hanrui Wang SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning, [HPCA 2021] Chip Demo for "SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning" Is Sparse Attention more Interpretable? DeepSeek Sparse Attention Explained: 80% Cheaper Long-Context AI Hanrui Wang's Talk at HPCA'20 on "SpArch: Efficient Architecture for Sparse Matrix Multiplication" Arxiv 2021: Sparse attention Planning HetSys Course: Lecture 10: Parallel Patterns: Sparse Matrices (Spring 2022) Why You're Wasting Your Money On Topeka's Craigslist Fees The Quiet Life Of Bob Dylan: Uncovering The Mystery Of His Marriage How Much Has Sandra Lee's Net Worth Gained Since Sandy Toes? How To Buy Safe On Craigslist In Fort Myers Without Getting Burned The High Price Of Not Checking Credentials: How A Famed Expert Was Jailed Time Zone Takedown: How Your Watch Could Be Setting You Back 3 Hours Against The Clock Is Vanessa Valladares' Rapid Aging In The Spotlight A Red Flag Meet The Zillionaire Doe Who Defied Wall Street With An Unconventional Investment Strategy
View Detailed Profile
Short Intro for HPCA'21 SpAtten: Efficient Sparse Attention Architecture by Hanrui Wang

Short Intro for HPCA'21 SpAtten: Efficient Sparse Attention Architecture by Hanrui Wang

Short intro video for HPCA 2021 paper: "

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning, [HPCA 2021]

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning, [HPCA 2021]

Talk video for HPCA 2021 paper: "

Chip Demo for "SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning"

Chip Demo for "SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning"

We design and tape out the chip for

Is Sparse Attention more Interpretable?

Is Sparse Attention more Interpretable?

Video for ACL 2021 paper https://arxiv.org/abs/2106.01087.

DeepSeek Sparse Attention Explained: 80% Cheaper Long-Context AI

DeepSeek Sparse Attention Explained: 80% Cheaper Long-Context AI

00:00:00 Introduction to DeepSeek

Hanrui Wang's Talk at HPCA'20 on "SpArch: Efficient Architecture for Sparse Matrix Multiplication"

Hanrui Wang's Talk at HPCA'20 on "SpArch: Efficient Architecture for Sparse Matrix Multiplication"

Presentation at HPCA 2020 on paper "SpArch:

Arxiv 2021: Sparse attention Planning

Arxiv 2021: Sparse attention Planning

...

HetSys Course: Lecture 10: Parallel Patterns: Sparse Matrices (Spring 2022)

HetSys Course: Lecture 10: Parallel Patterns: Sparse Matrices (Spring 2022)

Project & Seminar, ETH Zürich, Spring 2022 Hands-on Acceleration on Heterogeneous Computing Systems ...

ISCA'23 - Lightning Talks - Session1A - FACT: FFN-Attention Co-optimized Transformer Architecture wi

ISCA'23 - Lightning Talks - Session1A - FACT: FFN-Attention Co-optimized Transformer Architecture wi

ISCA'23: The 50th International Symposium on Computer

[HPCA'20] Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations

[HPCA'20] Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations

Presented by Nitish Srivastava at HPCA2020, San Diego, CA, United States. Abstract: Tensor factorizations are powerful tools in ...

Sparse Transformers - Tsvetomila Mihaylova [PyData Sofia April 2020]

Sparse Transformers - Tsvetomila Mihaylova [PyData Sofia April 2020]

This video was presented at PyData Sofia Online Meetup on 30.04.2020. Full Stream: ...

ParaDox: Eliminating Voltage Margins via Heterogeneous Fault Tolerance (Long Talk, HPCA 2021)

ParaDox: Eliminating Voltage Margins via Heterogeneous Fault Tolerance (Long Talk, HPCA 2021)

Providing reliability is becoming a challenge for chip manufacturers, faced with simultaneously trying to improve miniaturization, ...

SSumM: Sparse Summarization of Massive Graphs (KDD 2020, Long)

SSumM: Sparse Summarization of Massive Graphs (KDD 2020, Long)

A video presentation of Kyuhan Lee, Hyeonsoo Jo, Jihoon Ko, Sungsu Lim, Kijung Shin, "SSumM: