Gavia Gray's Avatar

Gavia Gray

@reflected.codes.bsky.social

Member of Technical Staff at Cerebras (views expressed my own) (also technically prof at Dalhousie University) https://github.com/gngdb https://reflected.codes/

423 Followers  |  277 Following  |  184 Posts  |  Joined: 09.08.2023  |  1.7661

Latest posts by reflected.codes on Bluesky

The Big Picture of Voice Feminization | A Comprehensive Overview | Lecture & Demonstration
YouTube video by TransVoiceLessons The Big Picture of Voice Feminization | A Comprehensive Overview | Lecture & Demonstration

The terms people usually use to describe the spectral differences are "vocal weight" and "resonance". Trans people and voice actors probably have the best resources on voice and how to talk about it, like @zheanna.bsky.social has some great videos on this www.youtube.com/watch?v=gZWQ...

11.02.2025 12:55 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

With Google removing their Responsible AI Principles, they no longer state that they will *not* engage in "Technologies whose purpose contravenes widely accepted principles of international law and human rights".
Concerns about surveillance and injury are also erased.
ai.google/responsibili...

04.02.2025 19:58 โ€” ๐Ÿ‘ 274    ๐Ÿ” 176    ๐Ÿ’ฌ 9    ๐Ÿ“Œ 21

Folks are mostly missing the forest for the trees re: the recent Meta announcement. Specifically, the focus on the move from fact checking to community notes. There two different changes that are a much bigger deal that have received less focus. ๐Ÿงต 1/11

09.01.2025 20:29 โ€” ๐Ÿ‘ 2712    ๐Ÿ” 974    ๐Ÿ’ฌ 90    ๐Ÿ“Œ 146
Preview
Generated condensed version of https://github.com/mattjj/autodidact Generated condensed version of https://github.com/mattjj/autodidact - autodidact.py

I got Claude to condense github.com/mattjj/autod... to a single file to make it more readable: gist.github.com/gngdb/a6bdf4...
It's the same design as the core Jax autograd engine as far as I know.

20.12.2024 16:49 โ€” ๐Ÿ‘ 5    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0
Preview
Empirical Upper Bounds for Unstructured Sparsity in... Sparsity in deep neural networks promises two improvements in computational efficiency, fewer FLOPs spent both to train the network and to perform inference. We find that both may be quantified...

Our poster is up in the compression workshop for Esha Singh's paper on bounds for scaling sparsity. We investigated better ways to compare sparsity inducing regularizers by grounding to compute efficient dense models. openreview.net/forum?id=qOn...

15.12.2024 17:36 โ€” ๐Ÿ‘ 2    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0
Post image

My poster is up!

12.12.2024 01:42 โ€” ๐Ÿ‘ 5    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0
Post image

How do a neural network's final parameters depend on its initial ones?


In this new paper, we answer this question by analyzing the training Jacobian, the matrix of derivatives of the final parameters with respect to the initial parameters.
https://arxiv.org/abs/2412.07003

11.12.2024 20:30 โ€” ๐Ÿ‘ 92    ๐Ÿ” 18    ๐Ÿ’ฌ 4    ๐Ÿ“Œ 0
NeurIPS Poster Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in TransformersNeurIPS 2024

code for this work is here: github.com/CerebrasRese...

11.12.2024 19:35 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0
NeurIPS Poster Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in TransformersNeurIPS 2024

I'll be presenting our poster on efficient estimation of critical batch size today at 4:30pm at NeurIPS 2024 neurips.cc/virtual/2024...

11.12.2024 18:04 โ€” ๐Ÿ‘ 5    ๐Ÿ” 0    ๐Ÿ’ฌ 1    ๐Ÿ“Œ 0

hi! thanks for adding me

11.12.2024 17:07 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

I would put this even more strongly: open source AI is probably our only realistic chance to avoid a terrifying increase in concentration of power. I do not want to live in a world where the people with all the money also have all the intellectual power.

29.11.2024 21:35 โ€” ๐Ÿ‘ 410    ๐Ÿ” 86    ๐Ÿ’ฌ 23    ๐Ÿ“Œ 12

I'm shocked you would fail to see how this is motivated. The sentiment from people cheering this on Twitter and elsewhere is obvious.

28.11.2024 19:49 โ€” ๐Ÿ‘ 8    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

A feature/bug of BlueSky is that you can back-date posts. People are using this to repost their tweets.
If you search for posts from the 1st century, there are jokes about the resurrection of Jesus. And, perhaps even funnier, some posts dated 24-11-20 instead of 2024-11-20.
bsky.app/search?q=unt...

20.11.2024 16:26 โ€” ๐Ÿ‘ 16    ๐Ÿ” 4    ๐Ÿ’ฌ 1    ๐Ÿ“Œ 2

when you try to convert your text into smaller pieces but all it gives you is Elvish, thatโ€™s a tolkienizer

20.11.2024 17:50 โ€” ๐Ÿ‘ 1021    ๐Ÿ” 105    ๐Ÿ’ฌ 36    ๐Ÿ“Œ 17
Preview
The Epidemic of Violence Against the Transgender & Gender-Expansiveโ€ฆ The 2024 Report

One of the hardest,and most meaningful, projects I do in my job is monitor the epidemic of fatal violence against the trans and gender-expansive community.

Today, on Transgender Day of Remembrance (#TDOR), we honor those weโ€™ve lost over the last year.

reports.hrc.org/an-epidemic-...

20.11.2024 14:03 โ€” ๐Ÿ‘ 209    ๐Ÿ” 107    ๐Ÿ’ฌ 4    ๐Ÿ“Œ 2
Preview
How a handful of Toronto businessmen got their way on bike lanes provincewide 'Non-partisan group' Balance on Bloor includes a former PC candidate, donors, and real estate industry players

Good reporting here: www.thetrillium.ca/news/municip...

14.11.2024 20:52 โ€” ๐Ÿ‘ 31    ๐Ÿ” 10    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 3
Preview
Open RL Benchmark: Comprehensive Tracked Experiments for... In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curves are rarely...

Neurips reviews are now publicly available.
Don't forget to check out Open RL benchmark, very useful when implementing algorithms or checking performance/impact of hyperparameters.

openreview.net/forum?id=ZDv...

14.11.2024 11:01 โ€” ๐Ÿ‘ 22    ๐Ÿ” 2    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 1

In summary, Ford's bike lane removals will, according to a new City Hall report:

- Cost $75 million in direct costs and sunk costs.
- Make traffic much, much worse for at least nine months during construction to remove lanes
- Have minimal long-term impact on car travel times.

13.11.2024 22:11 โ€” ๐Ÿ‘ 363    ๐Ÿ” 176    ๐Ÿ’ฌ 21    ๐Ÿ“Œ 58

We're cooked. The economy's cooked. The planet is really cooked.

But trans people are especially cooked, here in the U.S.

Because no one's coming to save us:

06.11.2024 15:42 โ€” ๐Ÿ‘ 40    ๐Ÿ” 3    ๐Ÿ’ฌ 1    ๐Ÿ“Œ 0
EDS, an HP Company 'Cat Herders'
EDS, an HP Company 'Cat Herders' EDS, an HP Company 'Cat Herders'

I have just learned that when my dad said "it's like herding cats" he was probably referring to a 1999 superbowl ad www.youtube.com/watch?v=m_Ma...

14.06.2024 15:45 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0
Preview
The legacy of Lynn Conway, chip design pioneer and transgender-rights advocate Conway, professor emerita of electrical engineering and computer science, has died.

RIP Lynn Conway, chip design pioneer and transgender-rights advocate

news.engin.umich.edu/2024/06/the-...

From wiki: IBM fired Conway in 1968 after she revealed her intention to transition. IBM apologized in 2020 (!).

en.m.wikipedia.org/wiki/Lynn_Co...

12.06.2024 00:23 โ€” ๐Ÿ‘ 735    ๐Ÿ” 332    ๐Ÿ’ฌ 7    ๐Ÿ“Œ 12

To do machine learning research it's important to have a degree in maths so you can read the math in most papers, a degree in computer science so you can write replicable code and a degree in physics so you can talk to all the physicists here. Luckily, getting them will leave no time for research.

31.05.2024 18:23 โ€” ๐Ÿ‘ 5    ๐Ÿ” 0    ๐Ÿ’ฌ 1    ๐Ÿ“Œ 0

Years of plotting research yet no use for anything except calling `fig.tight_layout()` and hope.

21.05.2024 19:20 โ€” ๐Ÿ‘ 2    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

When Karpathy said he can't watch any movie more than 20 years old he was actually using Cunningham's rule to get movie recommendations, obviously.

21.05.2024 12:55 โ€” ๐Ÿ‘ 2    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

The fact about polar bear fur being technically transparent is dumb if you think that means you shouldn't say the fur is white but much more interesting if you know that the same transparent structure makes the bears mostly invisible in IR, so night vision would be useless.

17.05.2024 13:37 โ€” ๐Ÿ‘ 3    ๐Ÿ” 1    ๐Ÿ’ฌ 1    ๐Ÿ“Œ 0

The geomagnetic storm seems to be winding down, so if you still need to communicate across time with your dead father via ham radio to solve your mother's murder, now's the time!

13.05.2024 00:17 โ€” ๐Ÿ‘ 1942    ๐Ÿ” 397    ๐Ÿ’ฌ 40    ๐Ÿ“Œ 18

The DVD rental store guy recommended this movie a few weeks ago and this post is within 1 hour of us pressing play on the movie and I didn't know this was what the plot was going to be. Unclear what powers over time the DVD rental store guy has.

13.05.2024 14:02 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

Is "back to the old drawing board" just from looney tunes or were drawing boards a universal experience in the 40s?

10.05.2024 13:00 โ€” ๐Ÿ‘ 0    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

It's wild how much better Fiesta Farms is

01.05.2024 20:57 โ€” ๐Ÿ‘ 0    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

Weird side effect of trying to use copilot: when I know what I want to type, I try to type fast enough that it doesn't have time to suggest anything.

01.05.2024 17:39 โ€” ๐Ÿ‘ 1    ๐Ÿ” 0    ๐Ÿ’ฌ 0    ๐Ÿ“Œ 0

@reflected.codes is following 20 prominent accounts