Hi! I’m Albert.

I’m a computational linguistics PhD and a philosophy MA at Brown University Prof. Ellie Pavlick. Quick update: I’m now a senior research scientist at Google DeepMind. I’m a core contributor to both Gemini pretraining and Bard RLHF. Additionally, I work on situational awareness and model organism. I apologize that I probably won’t be able to keep this website up to date!

I used to study representations in democracies (almost did a PhD in political science), and now I study representations of language and cognition. My current research focuses on two directions:

  1. Instruction-tuned models. I co/first-authored various large language models (T0, Flan-T5/PaLM, and BLOOM), with a focus on zero-shot generalization to unseen tasks that go beyond statistical pattern matching. At the moment, I’m working on LM agents with recurrent and adaptive computation with reinforcement learning.

  2. Finding where in pretraining and instruction-tuning corpora do models acquire zero-shot and few-shot abilities, understanding exactly how models generalize at test time, and explaining why models match human behaviors in profound ways in some settings (e.g., Dasgupta et al. 2022), while also exhibiting comically un-human-like behaviors in some other settings (e.g., Webson et al. 2023).

Anonymous Feedback

Did I do something inconsiderate, say something inappropriate, or made any mistakes big or small? Or did I do something good that I should amplify in the future? I will be very grateful if you can let me know through this anonymous form.

 

With my beloved PhD advisor, Ellie Pavlick, after my thesis defense.

With my beloved students, Qinan Yu and Alyssa Loo, after their first co-first-author paper. During their senior year at Brown, Qinan was accepted into Stanford PhD, and Alyssa was hired by Google DeepMind (Gemini pretraining team).

 

“Why should one section be taxed to construct a public improvement in another? ‘What interest has South Carolina in a canal in Ohio?’ And what if Ohio didn’t want it? Why should the national government decide such issues? The sovereignty of the individual states—their rights, their freedom—was being trampled.”

“‘What interest has South Carolina in a canal to the Ohio?’ The answer to that question expounds the whole diversity of sentiment between that gentleman and me… According to his doctrine, she has no interest in it. According to his doctrine, Ohio is one country, and South Carolina is another country… I, sir, take a different view of the whole matter. I look upon Ohio and South Carolina to be parts of one whole—parts of the same country—and that country is my country… I come here not to consider that I will do this for one distinct part of it, and that for another, but… to legislate for the whole.”

Robert A. Caro. Master of the Senate.