X. It’s what’s happening From breaking news and entertainment to sports and politics, get the full story with all the live commentary.

The reason for my skepticism is that I'm not sure xAI would give away Grok 3 and push it on x.com so aggressively if it cost arm and leg to run, as gpt-4.5 pricing indicates

I know there are no official info, of course. I'm following these rumors pretty closely, too. The compute flops they have had could have been achieved on ~2T model, no? I think Elon said that they used a ton of synthetic generated data too, and many rollouts to find good solutions for RL, too

Source that Grok 3 is 10T? I'm very skeptical of that. Maybe they scaled training data substantially but parameters not *that* much.

Perhaps this laziness is an intentional nudge towards using reasoning models (which are not yet available, though - I mean reasoners based on 4.5)

We need an uncertainty knob similar to temperature

They promised to open source prev gen after releasing the next. So we will know

And then if anything goes wrong or unexpected during the "wet" phase a clueless wannabe would like to pull VLM with camera and ask the model for "debug instructions". You can't do it "on Google".

You cannot get precise and detailed instructions for making a bomb or poison or other very dangerous stuff from things you can buy legitimately in just "a few clicks on Google". As a very minimum it's days of research, including how to gaslight vendors, how to prepare things, etc.

Analyzing the ethics and risks of autonomous agents is crucial. Thank you for your insightful work @mmitchell.bsky.social @evijit.io @sashamtl.bsky.social @giadapistilli.com

Bad take. Censorship of "recipes for ruin" is good. A blanket deontological rule "censorship is bad" doesn't work.

I guess the equivalent in AI agent(cy) engineering, the equivalent transition will be towards the method design and decomposition: dialogue? multi-role debate? argument tree? the data model the model is operating on top? Reward design for RL post-training/fine-tuning?

(take not mine) the current AI/agent(cy) engineering is much like pre-DL computer vision, when people tried to massage the problem around a few fairly rigid algos like SIFT. There also was RAG: scikit-image.org/docs/stable/... and also didn't work very well

I guess the equivalent in AI agent(cy) engineering, the equivalent transition will be towards the method design and decomposition: dialogue? multi-role debate? argument tree? the data model the model is operating on top? Reward design for RL post-training/fine-tuning?

With DL and end-to-end training in CV, loss design became a more important skill than heuristic bricolage

(take not mine) the current AI/agent(cy) engineering is much like pre-DL computer vision, when people tried to massage the problem around a few fairly rigid algos like SIFT. There also was RAG: scikit-image.org/docs/stable/... and also didn't work very well

Wrong, this is still a liberal hysteria

I have yet to regret shooting a request to undermind. It always finds something interesting. My requests are always of the form "who have done research in roughly this shape" (where I'm sure that someone did, but hard to find via Scholar)

FWIW in my impression, none of the services in this category (Perplexity, You.com, etc.) live up to the "deep" label except undermind.ai so far. Didn't try PaperQA though.

Google's deep research is a total flop. I paid for a subscription to try it. Tried on maybe 10 requests, very broad range, from technical to culture to philosophical. It spit out bald, often outright wrong slop every. single. time. Idk why you keep praising it

Zero information. Consistently candid Sama will say whatever the specific audience likes. I'm sure in different rooms he says the opposite of this

Exa The Exa API retrieves the best, realtime data from the web to complement your AI

Claude is very hit-or-miss for perplexity-like questions. Same for everything else: ChatGPT, Gemini, exa.ai, You.com.

Meta search with all of them may be helpful, even if not fully automatable yet: if LLMs knew good answers to these searches they would not be so hit-or-miss to begin with.

Maybe no hard distinction. It's a continuum between Sonnet and "reasoning" models.

Good design. "Native" tool calling makes LLM APIs much more complex than they should be. Using just a single "tool" - code execution - to rule them all is better.

Does the book argue for expected utility/value based decision-making? "Radical Uncertainty" by @profjohnkay.bsky.social directly argues against that

I think o1/o3 should be better at this (I don't use them at the moment), but breaking the flow and waiting would be weird. o1-capable coder with access to the context that constantly does some analysis in background and makes insightful suggestions for me from time to time would be best

Similarly, when writing a somewhat long function that includes 2-3 copies of similar but not identical logic (e.g., loop bodies), LLMs are never capable of factoring those out to shorten the function overall.

I find it annoying that LLMs often tend to write their own functions for doing something instead of using the standard library or the "utils" that I've already created in my project.

Presumably, people on MechanicalTurk got 75%. However, I would argue that people on MechanicalTurk are self-selected for something like openness for new tasks and problems.

Ethan, do you cherry-pick the stuff you post on twitter/bsky? How many experiments do you do that never make it to your twitter in which none of the AIs do anything remarkable or badly misunderstand your intent?

Google just shadow banned my account, returning bullshit 403 to all requests. At least Anthropic and OpenAI don't have this BS