Philipp Petersen

In spiking neural networks, neurons communicate - as in the brain - via short electrical pulses⚡(spikes). But how can we formally quantify the (dis)advantages of using spikes? 🤔

In our new preprint, @pc-pet.bsky.social and I introduce the concept of "Causal Pieces" to approach this question!

02.05.2025 08:06 — 👍 33    🔁 6    💬 1    📌 0

What happens if you lose 10$ per share in one week and gain 10$ per share the next alternating for 52 weeks ;)? Is the effect stronger if you replace 10$ by 20$?

12.04.2025 20:15 — 👍 0    🔁 0    💬 0    📌 0

The latest version includes:

✅ Significantly fewer typos
✅ More illustrations and figures
✅ Reorganized sections for better clarity
✅ Sharpened and improved arguments

08.04.2025 10:44 — 👍 2    🔁 0    💬 0    📌 0

table of contents

table of contents

table of contents

After receiving very helpful feedback from the community, Jakob Zech and I have revised our graduate textbook:

📘 𝘔𝘢𝘵𝘩𝘦𝘮𝘢𝘵𝘪𝘤𝘢𝘭 𝘛𝘩𝘦𝘰𝘳𝘺 𝘰𝘧 𝘋𝘦𝘦𝘱 𝘓𝘦𝘢𝘳𝘯𝘪𝘯𝘨

and uploaded the new version to arxiv:

🔗 arxiv.org/abs/2407.18384

If you have already read it—or plan to—we would really appreciate your feedback.

08.04.2025 10:44 — 👍 3    🔁 0    💬 1    📌 0

Great point in principle, but you seem to be having it at the ideal altitude and in the ideal season.

31.03.2025 08:42 — 👍 1    🔁 0    💬 0    📌 0

Other countries will only want to hire the top researchers, which only correspond to a small part of the budget.

31.03.2025 08:38 — 👍 0    🔁 0    💬 0    📌 0

🔍 Key insights:
* The singular values of the query-key matrix product are the most critical parameters for tracking stability.
* Self-attention and softmax operations are the worst offenders for error amplification.
* There are stable (and unstable) methods for normalization.

14.03.2025 07:05 — 👍 0    🔁 0    💬 0    📌 0

Behavior of relative error for increasing spectral norm of key and query matrices

Would you expect an LLM using over 100 billion floating-point operations in low precision to produce accurate outputs?
Not if you heard an introductory class to numerics. How bad can things get? To find out, we carried out a numerical stability analysis of the transformer arxiv.org/abs/2503.10251.

14.03.2025 07:05 — 👍 3    🔁 0    💬 1    📌 0