Viswanath (Vish) Sivakumar

Would recommend LLM-Training-Puzzles (which is about distributed training rather than about LLMs) as a companion for anyone reading the excellent Ultra-Scale Playbook by huggingface.

Wrapped up several of @srushnlp.bsky.social's puzzles over recent weekends. RASPy (and the original RASP language) was particularly elegant.

We're hiring!

Join Meta Reality Lab's EMG Wristband foundational research team! ⚡️

We're seeking research scientists with systems neuro/ML/AI/BCI/EE backgrounds, with skills on a spectrum from modeling to experimental expertise, as well as generalists who excel in both areas.

Human-Computer Input via an sEMG Wristband | Meta Quest Blog Today, we’re releasing a white paper that provides background on sEMG, describes its potential as a new input paradigm for human-computer interaction, and explains how sEMG sensing and its device inpu...

We published a brief white paper about surface EMG as a technology and its potential as a new and powerful human-computer input paradigm.

www.meta.com/blog/quest/s...

Screenshot of open roles at Fauna Robotics

Want a job in robotics in New York? faunarobotics.com

Live demos are hard, letting anyone try out the demo is harder. Hardware makes all of this doubly so.

Thank you to everyone who dropped by our posters, engaged in discussions, and tried out the surface EMG demo. We were humbled by the interest! #NeurIPS2024

Boichick on Fillmore is new and good!

Advancing Neuromotor Interfaces by Open Sourcing Surface Electromyography (sEMG) Datasets for Pose Estimation and Surface Typing We’re releasing emg2qwerty and emg2pose—two large datasets and benchmarks for sEMG-based typing and pose estimation, as part of the NeurIPS 2024 Datasets and Benchmarks track.

New 716 hour EMG dataset just dropped ✨ ai.meta.com/blog/open-so...

9/9

Gratitude to the entire CTRL-Labs team and partners at
Reality Labs, Meta.

Special shout out to @jseely.bsky.social, Michael Mandel, Sean Bittner, Alexandre Gramfort, Adam Berenzweig, Patrick Kaifosh, and TR Readon!

8/

And and that's not all!

Here's another large dataset we just released - emg2pose - focused on hand pose estimation using sEMG. emg2pose has 193 participants over 370 hours & >50 behavioral categories w/ hand motion capture ground truth.

arxiv.org/abs/2412.02725
github.com/facebookrese...

7/

Our baseline model built using standard techniques from the Speech Recognition literature shows that, with some personalization on top of a model pretrained with 100 subjects, we can quite accurately enable typing with sEMG, eventually without a physical keyboard.

6/

Towards that goal, we now release emg2qwerty - a wrist sEMG dataset collected while touch typing on a QWERTY keyboard. With 108 subjects, 1,135 sessions, 346 hours, and 5.2 million keystrokes, this is quite large by neuroscience standards.

arxiv.org/abs/2410.20081

github.com/facebookrese...

5/

But can we further push the boundaries of what is possible with sEMG?

We imagine many other applications for sEMG, including the ability to manipulate objects in AR or write full messages as quickly as—or faster than—typing on a keyboard, with very little effort.

Introducing Orion, Our First AR Glasses | Meta Quest Blog Today at Connect, Mark Zuckerberg unveiled Orion—our first pair of true AR glasses, previously codenamed Project Nazare.

4/

And at Connect 2024, we showed how one can use an EMG wristband with Orion—our AR glasses product prototype—for seamless and comfortable control over digital content to swipe, click, and scroll while keeping your arm resting comfortably by your side.

A generic noninvasive neuromotor interface for human-computer interaction Since the advent of computing, humans have sought computer input technologies that are expressive, intuitive, and universal. While diverse modalities have been developed, including keyboards, mice, an...

3/

Earlier this year, we unveiled a wristband device that can be seamlessly worn, non-invasively sense muscle activations in the wrist and hand via surface EMG, and achieve out-of-the-box generalization across individuals for multiple tasks including handwriting!

2/

The papers - emg2qwerty and emg2pose - are accepted at NeurIPS 2024 Datasets and Benchmarks Track.

If you’re in Vancouver next week, drop by our posters on Fri Dec 13 at 11am PST or at the Meta booth. We have some cool demos to show too!

neurips.cc/virtual/2024...

nips.cc/virtual/2024...

Advancing Neuromotor Interfaces by Open Sourcing Surface Electromyography (sEMG) Datasets for Pose Estimation and Surface Typing We’re releasing emg2qwerty and emg2pose—two large datasets and benchmarks for sEMG-based typing and pose estimation, as part of the NeurIPS 2024 Datasets and Benchmarks track.

1/

We just open-sourced two large wrist electromyography (EMG) datasets - one towards typing without a keyboard and the other for predicting hand poses - with baselines.

We believe these will help advance research into making high bandwidth non-invasive neuromotor interfaces a reality!

OK If we are moving to Bluesky I am rescuing my favourite ever twitter thread (Jan 2019).

The renamed:

Bluesky-sized history of neuroscience (biased by my interests)

Resonates a lot having lived in both places. I moved back to NYC as of last month, and personally, it’s been a steep QoL upgrade. But of course, SF will continue to be a fixture given I’m in tech.