Calvin McCarter

reporting for duty here on bluesky to fight for anti-blueskyism. (i'm not fighting against blueskyism because i'm not a hater like that.)

i first read that as:

"power washing" but for the feet. idea to fill out later

Slightly off-topic, but one of my biggest pet peeves is when people say about baby sadness, "Don't worry: they won't remember this." In the long run, we are all memoryless!

The idiosyncrasies of tabular data The things that make tabular data different

What makes tabular data unique (and interesting!) is not merely that it's arranged into rows and columns. New blogpost: calvinmccarter.substack.com/p/the-idiosy...

yea

what are the reasons for this from your perspective?

maybe not more expensive to manufacture in an absolute sense. but if chinese internal demand rises, and it no longer needs us exports, then china may no longer see the value in exporting to the us. in which case, the cost of consumer goods *in the us* would rise.

also, immigrants will also do less of our low-wage / low-status work for us (declining LatAm TFR). meanwhile, AI will (initially) compete with high-wage / high-status workers. so we're still looking at a combustible political situation.

tbf, while AI is going to start doing more of our work for us, East Asia will start doing less of our work for us (their trade surpluses will decline due to their aging demographics and the end of the USD as reserve currency)

In principle I agree, but federal lands are relatively poorly managed wrt fire prevention and management. Obviously it would be preferable to just fix that, but I'm not sure whether that's realistic.

Related to your earlier assessment of the probability of the US striking Iran, how would you assess the probability of an Israeli strike? Is it harder to predict, because Israel doesn't need to bring assets into the region?

"autoregression for training, diffusion for inference"

alphaxiv does this for all arxiv papers -- just s/ar/alpha/ in the url -- and i've been told that it's coming soon to {bio,chem,med}rxiv as well.

I don't disagree with you exactly, but if an institution's natural defenders are AWOL (or even siding with its enemies) due to a litany of grievances, then it's probably already too late to save that institution.

(not that there's anything wrong with that)

I just delete my old repo, then give myself a new username...

Flipping the switch on far-UVC We’ve known about far-UVC’s promise for a decade. Why isn't it everywhere?

Has anyone tried far-UVC in their home? It's now dropped into the ~$300 price range where I'm interested in trying it for myself. substack.com/home/post/p-...

it's definitely a Michigan thing

I am slightly cynical about the Clean Label Project, given that it seems to be "pay to play". Also, afaict, it was started by the founder of ByHeart formula, and theirs was like the first thing to get certification. Not that this is necessarily bad -- ByHeart and Kendamil are the best formula IMO.

the website has a list: cleanlabelproject.org/product-cate... which is specific and helpful, though oddly it slightly differs from the brands in the report:

Here's a link to the report: cleanlabelproject.org/wp-content/u... (TLDR heuristics: whey is better than plant-based, non-organic is better than organic, unflavored is better than chocolate-flavored)

Ultra exciting! And it's gratifying to see that this method uses the kernel density integral preprocessing method that I published in @tmlr-pub.bsky.social (2023). (One takeaway: even if your ML research focus isn't deep learning, pursue directions that complement rather than compete with it.)

i cooked him

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it...

There are ways around this, eg arxiv.org/abs/1902.04094, but I think uniform rate MLMs are the cleaner approach. Plus, as shown in Eq 5 of the Factorization Curse paper, it's what you get after marginalizing over all XLNet-style autoregressive permutations.

This creates a train-inference gap if one is training at any fixed masking rate. Of course, with MLMs people don't even try that, and instead use pseudo-likelihood as an approximation for likelihood. Besides being approximate, the problem with this is that this takes L separate forward-passes.

When one evaluates log-likelihood of a sequence of length L via the chain rule of probability, the first term has missingness fraction of 1, the second has missingness of (L-1)/L, etc. So the inference-time masking rate is ~ Uniform[0, 1].

The other advantage of uniform rate is that your MLM would now be able to provide likelihoods not just pseudo-likelihoods (unlike standard MLMs) and would be able to do so for any generation order (unlike causal LMs).

The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More Today's best language models still struggle with hallucinations: factually incorrect generations, which impede their ability to reliably retrieve information seen during training. The reversal curse, ...

Besides trying a fixed 30% masking rate, did you also consider using random ~ Uniform[0, 1] rate? This is akin to XLNet and order-agnostic autoregression, and recent work from Meta also found that it improves retrieval performance: arxiv.org/abs/2406.05183

How do you go from a hit in your antibody screen to a suitable drug? Now introducing CloneBO: we optimize antibodies in the lab by teaching a generative model how we optimize them in our bodies!
w/ Nat Gruver, Yilun Kuang, Lily Li, @andrewgwils.bsky.social and the team at Big Hat! 1/7

this treaty already exists (Prometheus was punished by the alien-gods for uplifting hominids)