Bernardo Esteves

NeuralSolver: Learning Algorithms For Consistent and Efficient... We contribute NeuralSolver, a novel recurrent solver that can efficiently and consistently extrapolate, i.e., learn algorithms from smaller problems (in terms of observation size) and execute those...

Check out our paper: openreview.net/forum?id=IxR...
Project website: bernardoesteves.com/NeuralSolver

This method can also be applied to sequential decision problems, where small mistakes lead to catastrophic results. NeuralSolver can solve the doorkey environment of size 128x128 with a performance very close to the oracle, just by performing behavior cloning on smaller sizes.

With this approach, NeuralSolver achieves almost perfect algorithmic extrapolation by executing the same learned algorithm on much harder problems. While highly surpassing previous works.

We can notice this by looking at how the latent values converge to the final result and the number of recurrent iterations. NeuralSolver learns an algorithm that starts by solving the dead ends of the maze, until finding the correct path that goes from the player to the goal.

By keeping the recurrent module output size equal to the input size, the model learns small local algorithms that are executed in parallel and work with any input size.

We do this by leveraging a recurrent model with an optional pooling layer for different-size tasks.
The recurrent model keeps the input size constant at each iteration.
The pooling layer is then used to collapse the information from the latent state to the desired output size.

We developed NeuralSolver, a method capable of learning algorithms that solve much more complex tasks than the ones used for training. That works on tasks that have the same or different output sizes!

NeuralSolver: Learning Algorithms For Consistent and Efficient... We contribute NeuralSolver, a novel recurrent solver that can efficiently and consistently extrapolate, i.e., learn algorithms from smaller problems (in terms of observation size) and execute those...

Check out our paper: openreview.net/forum?id=IxR...
Project website: bernardoesteves.com/NeuralSolver

This method can also be applied to sequential decision problems, where a small mistake in the sequence can lead to catastrophic results. NeuralSolver can solve the doorkey environment of size 128x128 with a performance very close to the oracle, just by performing behavior cloning on smaller sizes.

With this approach, NeuralSolver achieves almost perfect algorithmic extrapolation by executing the same learned algorithm on much harder problems. While highly surpassing previous works.

We can notice this by looking at how the recurrent latent values converge to the final result, along a certain number of recurrent iterations. NeuralSolver learns an algorithm that starts by solving the dead ends of the maze, until finding the correct path that goes from the player to the goal.

By keeping the recurrent module output size equal to the input size, the model learns small local algorithms that are executed in parallel and work with any input size.

We are able to do this by leveraging a recurrent model with an optional pooling layer for different-size tasks.
The recurrent model keeps the input size constant at each iteration.
The pooling layer is then used to collapse the information from the latent state to the desired output size.

We explore problems that increase in complexity mainly through size. Our goal is to be able to train an agent on a small/easy set of tasks and then without any more training to be able to solve more complex tasks.

🚀 Excited to present NeuralSolver at NeurIPS 2024!

Humans can easily learn algorithms to solve much more complex tasks, from simple tasks. Machine learning methods usually fail to do this.
Can we build models that have similar algorithmic extrapolation abilities?

Thread 🧵: