Talk
Physics Colloquium - Dense Associative Memory: physical systems for novel AI architectures
When
· 4:00 PM MDT
Shown in the venue’s time zone (America/Denver), not yours.
Where
Boulder, CO
Tickets
No on-sale date recorded
This event came from a public calendar feed. Calendar feeds publish when a show happens, not when its tickets go on sale — so this is genuinely unknown rather than zero, and rather than “already on sale”.
What the source said
Presented by: Dmitry Krotov Abstract: Dense Associative Memories are recurrent neural networks with fixed-point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in the 1980s, Dense Associative Memories have a very large information storage capacity, making them appealing tools for many problems in AI. In this talk, I will provide an intuitive understanding and mathematical framework for this class of models and give examples of problems in AI that can be tackled using these new ideas. Specifically, I will explore the relationship between Dense Associative Memories and transformers. I will present a neural network called the Energy Transformer, which unifies energy-based modeling, associative memories, and transformers in a single architecture. I will demonstrate how Energy Transformers can be used for challenging tasks in image processing, solve partial differential equations, and serve as computational modules for energy-based language modeling. I will also discuss an exciting possibility of mapping these models onto analog hardware accelerators, which could enable much more energy-efficient inference compared with GPUs. Host: Andrew Lucas