Our paper, "Parsing millions of URLs per second", written with @lemire.bsky.social became one of the most read articles in Journal of Software: Practice and Experience.
onlinelibrary.wiley.com/journal/1097...
Our paper, "Parsing millions of URLs per second", written with @lemire.bsky.social became one of the most read articles in Journal of Software: Practice and Experience.
onlinelibrary.wiley.com/journal/1097...
CSR
01.12.2024 20:33 — 👍 1 🔁 0 💬 0 📌 0
Andrej Karpathy @karpathy Follow People have too inflated sense of what it means to "ask an Al" about something. The Al are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an Al", think of it more as "asking the average data labeler" on the internet. Few caveats apply because e.g. in many domains (e.g. code, math, creative writing) the companies hire skilled data labelers (so think of it as asking them instead), and this is not 100% true when reinforcement learning is involved, though I have an earlier rant on how RLHF is just barely RL, and "actual RL" is still too early and/or constrained to domains that offer easy reward functions (math etc.).
But roughly speaking (and today), you're not asking some magical Al. You're asking a human data labeler. Whose average essence was lossily distilled into statistical token tumblers that are LLMs. This can still be super useful ofc ourse. Post triggered by someone suggesting we ask an Al how to run the government etc. TLDR you're not asking an Al, you're asking some mashup spirit of its average data labeler. 12:33 PM • 11/29/24 • 1.2M Views
Andrej Karpathy © @karpathy Follow Example when you ask eg "top 10 sights in Amsterdam" or something, some hired data labeler probably saw a similar question at some point, researched it for 20 minutes using Google and Trip Advisor or something, came up with some list of 10, which literally then becomes the correct answer, training the Al to give that answer for that question. If the exact place in question is not in the finetuning training set, the neural net imputes a list of statistically similar vibes based on its knowledge gained from the pretraining stage (language modeling of internet documents). 12:49 PM • 11/29/24 • 223K Views
Interesting 👀
After seemingly endless, frothy AI hype campaigns, it looks like there’s some expectation setting happening now.
(Karpathy co-founded OpenAI and is currently head of AI for Tesla.)
Excited for Advent of TypeScript, which starts tomorrow. It was a lot of fun last year, I made it to day 20. www.adventofts.com
30.11.2024 18:08 — 👍 161 🔁 16 💬 13 📌 1
As is always the case with Vite, it's a beautiful thing to see so many open source communities come together to collaborate on shared infrastructure. 💜💛
👀 I'm personally very excited to use the Environment API as the foundation for a new project! It's an elegant solution to a tricky problem.