John Vinyard

You can play with this example yourself here: blog.cochlea.xyz/resonancemod...

Here's me scraping unskillfully on a "violin".

In this micro-experiment, I try to extract playable instrument models from a single audio recording. You can play with a handful of the models in your browser!

blog.cochlea.xyz/resonancemod...

You can play with it too: blog.cochlea.xyz/conv-instrum...

Text-to-music is amazing, but it sucks all the fun out for me, as a musician. I'm using machine learning to "extract" models that can be controlled in real-time. My goal is to give them all the subtlety and richness that you'd get from a real acoustic instrument.

A fun little WebAudio + MediaPipe toy for October 🛸 🧟

blog.cochlea.xyz/theremin.html

blog.cochlea.xyz/rnn-instrume...

Toward a Sparse and Interpretable Audio Codec Most widely-used modern audio codecs, such as Ogg Vorbis and MP3, as well as more recent "neural" codecs like Meta's Encodec or the Descript Audio Codec are based on block-coding; audio is divided int...

arxiv.org/abs/2505.05654

I have received an endorsement! Feel free to read the paper and provide feedback if you're so inclined!

Supplemental material with example audio reconstructions is here:

blog.cochlea.xyz/sparse-inter...

I wrote a paper summarizing the work I've been pursuing toward a sparse audio codec!

I'd like to post this to the CS -> Sound section of arxiv, if possible, so if anyone would be willing to "endorse" me, shoot me a message and I'll share the code!

blog.cochlea.xyz/toward-a-spa...

Here's the "dashboard" for a model that's training right now. Event times are represented bottom-left, and event vectors, which encode everything about the sound _except_ for time-of-occurrence, on the bottom-right.

To put it into context, an encoder model is trained to incrementally decompose a segment of audio into some number of event vectors and their times-of-occurrence. You can read more about the training process here:

blog.cochlea.xyz/sparse-inter...

This play-able scatterplot (clicking on points plays a sound) maps many points from an audio event decoder onto a 2D plane using t-SNE. I find it both fun to play with and pretty to look at!

blog.cochlea.xyz/scatter.html

I've added accelerometer support to the spring-mass synthesis toy. Try it out on your phone!

blog.cochlea.xyz/string.html

I believe that one key to sparse audio representations will be baking "resonance priors" into models; even many synth sounds are aping some real-world system. As such, I've gotten interested in physical modeling, and built this fun little Web Audio toy to learn more!

blog.cochlea.xyz/string.html