Hi! I’m Albert.

I’m a computational linguistics PhD and a philosophy MA at Brown University Prof. Ellie Pavlick. Quick update: I’m now a senior research scientist at Google DeepMind. I’m one of the main ICs in Gemini RL. I apologize that I probably won’t be able to keep this website up to date.

I used to study representations in democracies (almost did a PhD in political science), but then I did a PhD on representations of language and cognition, focusing on two directions:

  1. Instruction-tuned models. I co/first-authored various large language models (T0, Flan-T5/PaLM, and BLOOM), with a focus on zero-shot generalization to unseen tasks that go beyond statistical pattern matching.

  2. Finding where in pretraining and instruction-tuning corpora do models acquire zero-shot and few-shot abilities, understanding exactly how models generalize at test time, and explaining why models match human behaviors in profound ways in some settings (e.g., Dasgupta et al. 2022), while also exhibiting comically un-human-like behaviors in some other settings (e.g., Webson et al. 2023).

Anonymous Feedback

Did I do something inconsiderate, say something inappropriate, or made any mistakes big or small? Or did I do something good that I should amplify in the future? I will be very grateful if you can let me know through this anonymous form.

 

With my beloved PhD advisor, Ellie Pavlick, after my thesis defense.

With my beloved students, Qinan Yu and Alyssa Loo, after their first co-first-author paper. Qinan was accepted into Stanford PhD, and Alyssa was hired by Google DeepMind (Gemini pretraining team).

 

“Why should one section be taxed to construct a public improvement in another? ‘What interest has South Carolina in a canal in Ohio?’ And what if Ohio didn’t want it? Why should the national government decide such issues? The sovereignty of the individual states—their rights, their freedom—was being trampled.”

“‘What interest has South Carolina in a canal to the Ohio?’ The answer to that question expounds the whole diversity of sentiment between that gentleman and me… According to his doctrine, she has no interest in it. According to his doctrine, Ohio is one country, and South Carolina is another country… I, sir, take a different view of the whole matter. I look upon Ohio and South Carolina to be parts of one whole—parts of the same country—and that country is my country… I come here not to consider that I will do this for one distinct part of it, and that for another, but… to legislate for the whole.”

Robert A. Caro. Master of the Senate.