Media Summary: Abstract: When trying to gain better visibility into a machine A light intro to LLMs, chatbots, pretraining, and transformers. Dig deeper here: ... The paper introduces a method called Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) to scale ...
Studying Large Language Model Generalization - Detailed Analysis & Overview
Abstract: When trying to gain better visibility into a machine A light intro to LLMs, chatbots, pretraining, and transformers. Dig deeper here: ... The paper introduces a method called Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) to scale ... This is a 1 hour general-audience introduction to It's an older paper, but it checks out. Rob Miles discusses the problem of 'Sleeper Agents' - where LLMs could have hidden traits ... Talk given by Eran Malach to the Formal Languages and Neural Networks discord on June 10, 2024. Thank you, Eran! Please ...