Lucas Dixon

Read smarter, not harder, with Lumi As research engineers, our reading lists are always exciting… and also way too long to finish. While we’d love to go through every research…

Helps you more quickly understand, ask questions, get to the parts of the original content that you are looking to understand. Would love to hear what you think!

Read more: medium.com/people-ai-re...

Github: github.com/PAIR-code/lumi

Lumi: A reading prototype by Google PAIR Explore research papers with AI features including annotations, granular summaries, and custom Q&A. Prototype by People & AI Research (PAIR) at Google

New open-source AI assited reading experience we built for arxiv papers: lumi.withgoogle.com

An image with the Vancouver skyline and the words "sign up to review". At the top are the logos of both the Actionable Interpretability workshop (a magnifying glass) and the ICML conference (a brain).

🚨 We're looking for more reviewers for the workshop!
📆 Review period: May 24-June 7

If you're passionate about making interpretability useful and want to help shape the conversation, we'd love your input.

💡🔍 Self-nominate here:
docs.google.com/forms/d/e/1F...

ARBORproject arborproject.github.io · Discussions Explore the GitHub Discussions forum for ARBORproject arborproject.github.io. Discuss code, ask questions & collaborate with the developer community.

Take a look at some initial research projects, and see if there's one you'd like to work on:
github.com/ARBORproject...
Or propose your own idea! There are many ways to contribute, and we welcome all of them.

Great thread describing the new ARBOR open interpretability project, which has some fascinating projects already. Take a look!

Looking for a small or medium sized VLM? PaliGemma 2 spans more than 150x of compute!

Not sure yet if you want to invest the time 🪄finetuning🪄 on your data? Give it a try with our ready-to-use "mix" checkpoints:

🤗 huggingface.co/blog/paligem...
🎤 developers.googleblog.com/en/introduci...

In December, I posted about our new paper on mastering board games using internal + external planning. 👇

Here's a talk now on Youtube about it given by my awesome colleague John Schultz!

www.youtube.com/watch?v=JyxE...

Yeah, I'm skeptical of how good LLMs alone can be, but when they get to use existing search based theorem provers, and lookup tools (SAT, induction provers, etc), then I would expect a good deal better w.r.t. gap sizes, and ability to find counter examples.

I have a hope that modern AI/LLMs might help here: by helping translate informal papers to formal mathematical statements, and informal proof to formal proof, and thereby help highlight gaps and help find counter examples. A few others are interested in this... @wattenberg.bsky.social maybe?

Love the idea! Is there any stats / evals for it! And how does one get to play with it? :)

Research scholar program Overview

Google research scholar programme applications open until 27th Jan. support early-career professors (received PhD within seven years of submission).
research.google/programs-and...

What's in an attention head? 🤯

We present an efficient framework – MAPS – for inferring the functionality of attention heads in LLMs ✨directly from their parameters✨

A new preprint with Amit Elhelo 🧵 (1/10)

We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus!
medium.com/people-ai-re...

I think this (LLMs makes making small scripts super easy) gets more profound agian when we start to make lots of tiny voice-powered apps/agents... would love to see more prototyping tools for this and play with them. Send me pointers!

Option 1.

I just wish search was semantic instead of substring! Still fun to see and explore! Btw - which embedding model did you use and what input text per paper?

That's neat! Did you also try a few different other styles/mood boards?

🚨 New Paper 🚨
Can LLMs perform latent multi-hop reasoning without exploiting shortcuts? We find the answer is yes – they can recall and compose facts not seen together in training or guessing the answer, but success greatly depends on the type of the bridge entity (80% for country, 6% for year)! 1/N

From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step When leveraging language models for reasoning tasks, generating explicit chain-of-thought (CoT) steps often proves essential for achieving high accuracy in final outputs. In this paper, we investigate...

arxiv.org/abs/2405.14838 by supervised learning curriculum of incrementally eliminating the start of a CoT they are able to train gpt2-small to do 9 digit multiplication without CoT; a fascinating and impressive result!

I'm learning a lot from this visually rich blog post. Also, I'm charmed by the rotating list of equal-contribution authors. Good knights-of-the-round-table energy!
dl.heeere.com/conditional-...

I want to describe my experience of coding with AI, because it seems to differ from other people's expectations. Earlier this morning, I saw a beautiful image here, based on roots of polynomials: bsky.app/profile/scon...
I wanted to try this idea myself, but with animation in a Javascript context!

It's so beautiful to see this kind of fluid interaction with huge data!

Many circles of different sizes, representing a visualization of inequality

The Gini coefficient is the standard way to measure inequality, but what does it mean, concretely? I made a little visualization to build intuition:
www.bewitched.com/demo/gini

Galilean Moons Timeline A meditative visualization of Jupiter's Galilean moons

A meditative toy, visualizing Jupiter's Galilean moons:
www.bewitched.com/demo/jupiter/

And here is the context page for the DeepMind student researcher programme: deepmind.google/about/studen...

People can now apply for Student researcher roles (basically a kind of internship) at Google/Deep Mind (until Dec 13)
www.google.com/about/career...

Neat! Are there any human-eval results for comparing the output w.r.t. Things like hallucinations and human enjoyment. I'd love to see eval here!