Eric W. Tramel

What are the power dynamics involved with the amount of abuse I have to heap on small LMs to get them to stay on prompt?

Large models you can just ask nicely. Qwen and below you have to threaten with bodily harm.

The story so far:

- Sane ML in exile from X
- welcome to bsky we are happy to have you!
- let’s help build better bsky tools using our knowledge and skills!

HOW DARE YOU! WE WILL BURN YOU TO THE GROUND!

FL was a field that caught its own tail, in terms of hype cycle. Now it is just something entirely detached from reality.

Thankfully, while incredibly hyped, LLM/transformer field has actually delivered — or delivered enough to outpace itself.

But federated learning is a synchronous training owing to its need for data privacy — induces huge sync overhead for SMPC. Not sure FL is the right paradigm lens to look through.

Why not go back to EA-SGD, Hogwild, etc and build back up from there? Async breaks bottleneck and grads can be quant