Maciej Strzelczyk

Optimizing vLLM inference on TPUs! Learn how to push your TPUs to their limits with proper vLLM inference configuration options.

Running vLLM on Google Cloud TPUs? My latest post details critical strategies for optimizing inference performance. Discover how to get the most out of this powerful hardware and software combination. Read more: medium.com/google-cloud...

#vLLM #GoogleCloud #TPU #MachineLearning #LLMops #AI

Serving Gemma 3 on TPU v5e and v6e using vLLM Learn how to serve Gemma 3 27B on v5e and v6e TPU VMs!

Do you want to try out Google Cloud TPUs, but don't know where to start? How about a simple Gemma 3 inference service on top of vLLM? It's easier than you probably think ;)

medium.com/google-cloud...