Vikram Sekar

A Primer on Transformer Architecture: Model Parameter Calculations, Estimating GPT-5 Architecture A detailed look at transformers, the role of attention, and decoder-only architecture; Learning to count model parameters based on architecture and making guesses on GPT-5 internal construction.

www.viksnewsletter.com/p/a-primer-o...

For paid subscribers:
→ Parameter counting using Google’s BERT-Base
→ Detailed look at GPT2/GPT3
→ An educated guess at GPT4
→ Speculating on GPT5
→ References

Downloadable Google Sheet you can use to make your own parameter estimates and speculate on GPT-5!

See comment 👇🏽

For free subscribers:
→ Birth of the Transformer
→ Tokens, Embeddings and Vocabulary
→ QKV, Context Window, and Batch Sizes
→ Attention Calculation Methods
→ Decoder Transformer Architecture

GPT-5 is here. 🥳 With sloppy graphs, hype and disappointment.

We know it has trillions of parameters, but where does it come from?

Today's deep dive post covers everything you need to know.

You will even make your own GPT-5 estimates at the end of the post.

7 Unwritten Rules for High-Impact Engineering Careers What they don’t teach you about thriving in technical organizations, but will define your success.

5. Your visibility is your own responsibility
6. Cross boundaries without stepping on toes
7. Spot CYA but don’t get sucked in

Read more here:
www.viksnewsletter.com/p/7-unwritte...

1. Your value is the specific problems you can solve
2. The best projects are rarely assigned — you have to find them
3. Know who can say yes (and when they will)
4. Survive high-stakes design reviews

7-rules for high impact engineering careers:

That no-one ever tells you...

No way to tell

Yes true- but over 90,000 people took the survey. Data set is large enough.

The Decline of MATLAB: What Engineers Should Learn Instead A frank look at why Python and Rust is the future of the hardware programming stack.

The 2024 Stack Overflow developer survey puts Matlab as the least admired programming language among 50 different ones.

Lower than Cobol, Visual Basic, Prolog and Fortran.

Read full post. 👇🏽
www.viksnewsletter.com/p/the-declin...

Passion versus Practicality in Engineering Careers A framework for choosing your specialization focus when industry trends and personal interests don’t align.

www.viksnewsletter.com/p/passion-ve...

Sometimes, being a little different early on can quietly compound over time.

More thoughts on following your passion vs being practical in your career choice... 👇🏽

We’re seeing the same thing today. Engineers who explored machine learning a decade ago—before it was mainstream—are now highly sought after.

There’s nothing wrong with choosing a stable path. But when too many take it, it gets harder to stand out.

The safest path in engineering can also be the most crowded.

Early in my career, I focused on RF while most went into digital design. It wasn’t the obvious choice—but it gave me a differentiator when wireless took off in the early 2010s.

Solid-State PAs vs. Traveling Wave Tube Amplifiers

Pros and Cons. 👇🏽

Tube amplifiers have been the mainstay of power generation in SATCOM.

The efficiency and output power of tubes was unmatched by solid state technology.

Until GaN arrived.

Read more here:
www.viksnewsletter.com/p/why-the-fu...

🍻 Semi Pub Quiz #1 Answers With short explanations and further reading materials.

Here are the answers (with questions) to yesterday's pub quiz on my newsletter, along with short explanations 🥂

www.viksnewsletter.com/p/semi-pub-q...

🍻 Semi Pub Quiz #1 It's hump day, and who says you can't have an online pub quiz.

This week's post is a semiconductor pub quiz! 🍻

How many can you answer?👇🏽

Answers will be published, along with a short explanation and references tomorrow.

So make sure you subscribe to the newsletter to get it in your email.

Good luck!

www.viksnewsletter.com/p/semi-pub-q...

SATCOM is leaving vacuum tubes behind.

For decades, traveling-wave tubes were the only way to generate >100W RF output.

Enter GaN.

GaN-based SSPAs are now rivaling TWTs — delivering high power, efficiency, and reliability.

Read it here:
www.viksnewsletter.com/p/why-the-fu...

A Comprehensive Primer on Advanced Semiconductor Packaging Almost everything you will ever need to know about semiconductor packaging.

If you want to read more about what packaging technology looks like today, check this out:

https://www.viksnewsletter.com/p/a-comprehensive-primer-on-advanced-packaging

Image

Turns out that you can now put TSVs in EMIBs.

Whoever created this image impresses me more than putting a TSV through a piece of silicon.

Check it more on IEEE Spectrum.
https://spectrum.ieee.org/intel-advanced-packaging-for-ai

I'm in SF attending the International Microwave Symposium.

If you'll be attending too, let me know. We'll chat at the conference.

If you're in the area and want to grab a beverage, hit me up.

I'm contemplating a deep-dive post on the role of GaN/SiC for AI data center power.

Along the lines of what Navitas is working on.

Good idea? Or is power not such a big deal at the moment?

The Gigabit Wall No, we don't need mobile data rates faster than 1 Gb/s and three upcoming wireless technologies that are actively focused on features other than speed.

Read about the gigabit wall:
https://www.viksnewsletter.com/p/the-gigabit-wall

I feel like RF folks tend to randomly throw out numbers like this to justify working on 6G without thinking of whether that kind of data rate is even feasible or practical.

I've explained in a earlier post that 1Gbps is what we will reasonably need in the near to mid- term.

I stand by that claim.

Image

I was reading a paper on use of glass substrates for 6G in the latest IEEE Microwave Magazine

The first chart is how we will need 10-100 Gbps (!) for immersive AR/VR experience!!

Using AI to Break the Black Magic of RF Design New research funding and emerging tools are setting a new standard for design.

Read it here:
https://www.viksnewsletter.com/p/using-ai-to-break-the-black-magic?r=222kot

Image

After I posted my article on the use of AI for RFIC design, a founder of a startup just out of stealth contacted me to say how their product does exactly that for MMIC design.

I'll be meeting them at IMS next week to see their demo. Exciting!

If you missed my post on AI for RFIC, link in comments.

Source: https://arxiv.org/pdf/2504.19878

Image

Turns out hooking up chiplets in a honeycomb-ish pattern in an SoC gives the best throughput somehow

Image

Time to upgrade to USB sticks.