Scaling Laws provide a valuable lens in guiding model design and computational budgets. Our recent work extends this lens to the realm of _fine-grained_ sparsity. Check out our #ICLR2025 paper, and the thread below from lead-author @tjin.bsky.social summarizing our findings.
22.04.2025 01:32 — 👍 2 🔁 1 💬 0 📌 0
YouTube video by Suvinay Subramanian
TPUs: Codesigning Computing Systems for Artificial Intelligence | MIT | 2023.11.07
And finally, for those interested in more technical details, and codesign across multiple layers of the stack from hardware, circuits to software and all the way up to the datacenter: youtu.be/RyV9xQDpO6U?...
13.04.2025 04:14 — 👍 0 🔁 0 💬 0 📌 0
A couple of fun videos that provide a sneak peek into TPUs and how they are plugged into our datacenters: [1] youtu.be/FsxthdQ_sL4?... [2] youtu.be/9i1ZM0dPyRo?...
13.04.2025 04:13 — 👍 0 🔁 0 💬 1 📌 0
YouTube video by SC Conference Series
SC24 IEEE-CS Seymour Cray Computer Engineering Award
For a historical account on the journey of developing TPUs, check out Norm Jouppi 's talk at SuperComputing'24: youtu.be/a-1xJmfYxyU?...
13.04.2025 04:11 — 👍 2 🔁 0 💬 1 📌 0
YouTube video by Google Cloud Events
Google Cloud TPUs and specialized AI hardware: Jeff Dean on what's next
This fireside chat with @jeffdean.bsky.social dives into the innovative features in the latest generation of TPUs, and what's in the pipeline: youtu.be/fNjH5izFeyw?...
13.04.2025 04:11 — 👍 1 🔁 0 💬 1 📌 0
At Google, we announced the latest generation of our AI supercomputers (TPUs) -- Ironwood -- this week. Check out the blogpost in quote for the highlights. blog.google/products/googl…
Pointers to deep-dives and more technical details in thread. [contd...👇]
13.04.2025 04:09 — 👍 2 🔁 0 💬 1 📌 0
Computer Architecture Podcast | comparchpodcast
A show that brings you closer to the cutting edge in computer architecture and the remarkable people behind it. Hosted by Dr. Suvinay Subramanian, who is a computer architect at Google in the Systems ...
Together with Lisa Hsu (Meta), we have been hosting the Computer Architecture Podcast -- we recently crossed 50K downloads. Check out our latest episode with Prof. Arka Basu: comparchpodcast.podbean.com -- we discuss GPUs, but a different vantage point than AI which is all the rage.
18.03.2025 19:04 — 👍 0 🔁 0 💬 0 📌 0
Starting with this exciting line of work from @tjin.bsky.social and colleagues at MIT. We tackle the question of: Can we train LLMs to parallelize autoregressive decoding automatically, backed by a performant runtime to exploit this parallelism for improved inference speedup?
18.03.2025 19:02 — 👍 0 🔁 0 💬 0 📌 0
Hello world! Dipping my toes into social media. My excellent intern(s) at Google with whom I have had the pleasure of working, were kind enough to nudge me to help signal-boost their work. Will also try to share updates on TPUs, AI chips & systems, and computer architecture.
18.03.2025 19:01 — 👍 1 🔁 0 💬 0 📌 0
Building technology for everyone's good | Thinker learning to be a doer
Writing The Pragmatic Engineer (@pragmaticengineer.com), the #1 technology newsletter on Substack. Author of The Software Engineer's Guidebook (engguidebook.com). Formerly at Uber, Skype, Skyscanner. More at pragmaticengineer.com
Professor, Programmer in NYC.
Cornell, Hugging Face 🤗
Research Scientist, Google DeepMind / Ex-academic / Deep learning to help people write code / ❤️s:🐱🐶☕️🍕
Professor at Wharton, studying AI and its implications for education, entrepreneurship, and work. Author of Co-Intelligence.
Book: https://a.co/d/bC2kSj1
Substack: https://www.oneusefulthing.org/
Web: https://mgmt.wharton.upenn.edu/profile/emollick
distributed (diloco) + modularity (dipaco) + llm @ deepmind | continual learning phd @ sorbonne
AI @ Google DeepMind - Gemini+Gemma. Ex NVIDIA (built AI for self-driving cars + GPU data science), Twitter (started AI team/Cortex), MadBits (founded+sold @ Twitter) 🇺🇸🇫🇷
Chief AI Scientist at Databricks. Founding team at MosaicML. MIT/Princeton alum. Lottery ticket enthusiast. Working on data intelligence.
Señor swesearcher @ Google DeepMind, adjunct prof at Université de Montréal and Mila. Musician. From 🇪🇨 living in 🇨🇦.
https://psc-g.github.io/
@PyTorch "My learning style is Horace twitter threads" -
@typedfemale
A Quite Interesting account from the team behind the BBC show QI.
AI x storytelling
AI Engineering: https://amazon.com/dp/1098166302
Designing ML Systems: http://amazon.com/dp/1098107969
@chipro
ML/AI researcher & former stats professor turned LLM research engineer. Author of "Build a Large Language Model From Scratch" (https://amzn.to/4fqvn0D) & reasoning (https://mng.bz/Nwr7).
Also blogging about AI research at magazine.sebastianraschka.com.
AI @ OpenAI, Tesla, Stanford
Cofounder & CTO @ Abridge, Raj Reddy Associate Prof of ML @ CMU, occasional writer, relapsing 🎷, creator of d2l.ai & approximatelycorrect.com
I lead Cohere For AI. Formerly Research
Google Brain. ML Efficiency, LLMs,
@trustworthy_ml.
Cofounded and lead PyTorch at Meta. Also dabble in robotics at NYU.
AI is delicious when it is accessible and open-source.
http://soumith.ch
✨ Keep it simple, make it scale. AI should be about empowering users and building understanding. 👩💻 AI Developer Experience @ Google DeepMind, ex-Github, ex-Google
