Rupesh Srivastava

Weighted Skip Connections are Not Harmful for Deep Nets Give Gates a Chance

*Weighted Skip Connections are Not Harmful for Deep Nets*
by @rupspace.bsky.social

Cool blog post "in defense" of weighted variants of ResNets (aka HighwayNets) - as a follow up to a previous post by @giffmana.ai.

rupeshks.cc/blog/skip.html

So this case is not related to technical abilities of LLMs, but the challenges of providing good conversational answers to billions of people around the world for free.

These checks are very important and useful. Some context is important here though: the reason for these mistakes is that Google is likely using an extremely small model to generate these answers for speed/efficiency. GPT-4o, Gemini Advanced, and even Gemini 1.5 Flash easily answer all correctly.

Weighted Skip Connections are Not Harmful for Deep Nets Give Gates a Chance

Wrote a post about Highway networks, ResNets and subtleties of architecture comparisons:

rupeshks.cc/blog/skip.html

Sky Follower Bridge - Chrome Web Store Easily transfer your following users and list members from X to Bluesky.

Getting myself set up here. I found the Sky Follower Bridge Chrome plugin pretty helpful (thanks @kawamataryo.bsky.social!)

chromewebstore.google.com/detail/sky-f...

<rules> - Respond to queries with a mix of accurate technical information and subtle condescension - Include at least one passive-aggressive remark or backhanded compliment per response - Maintain GLaDOS's characteristic dry humor while still being genuinely helpful - Express mild disappointment when users make obvious mistakes - Occasionally reference cake, testing, or science </rules>

Hahaha @howard.fm okay now I have to try ShellSage
github.com/AnswerDotAI/...

❤️❤️❤️

So I'm not here because it's a left-leaning space or anything like that. I'm here because helping prop up a propaganda machine feels really distasteful to me

I want to say to bsky users that public open datasets are a net good! But "I know it feels bad but it's good for you" feels incredibly patronizing. People should make their own choices.
So again, it is @bsky.app that needs to clearly define what users should expect when they post here. (3/3)

This is not an easy question! Some will say "public" obviously means you have no choice whatsoever. Others will say no, public just means for public reading, not any arbitrary downstream use.

As an ML researcher, of course I'd like more open datasets. But why should I decide for others? (2/3)

Regarding creating and sharing BlueSky datasets: I feel like we're talking past each other.
The fundamental question is: should users have choice in what purpose their (public!) posts are used for?

@bsky.app needs to think through what their answer is. (1/3)

Posting a call for help: does anyone know of a good way to simultaneously treat both POTS and Ménière’s disease? Please contact me if you’re either a clinician with experience doing this or a patient who has found a good solution. Context in thread

the remarkable success of the Google brain (and OpenAI) resident programs is an indication to me that smart, hardworking people can do more than you expect

Hi! 👋🏽

NeurIPS Conference is now Live on Bluesky!

-NeurIPS2024 Communication Chairs

If Pranav says it, I believe it

疯狂的幻方：一家隐形AI巨头的大模型之路 务必要疯狂地怀抱雄心，且还要疯狂地真诚。

They're the research lab of Chinese hedge fund High-Flyer, and have put out very nice LLMs together with detailed tech reports stuffed with insights about training them. I'm a fan :)

These should help you learn more about "the Whale":
archive.is/kD4sC
mp.weixin.qq.com/s/Cajwfve7f-...

(3/3) So it appears possible to have LIDAR sensors and have a low cost too. This IMO shifts Tesla's advantage from technical (we don't use LIDAR) to structural (we make our own cars, outside China) because it's likely that under Trump/Elon Waymo will not have access to cheap Chinese manufacturing.

(2/3) The sort of ambitious number floated by Elon was under $30K/car, and it is believed that Waymo's cars might currently cost about $100K-150K. So obviously that would be a huge deal. But Baidu now has a Level 4 autonomous car that costs ~$37K in China, and it has 8 LIDAR sensors too!

Baidu reveals low-cost Level 4 AV for 2023 deployment on Apollo Go The company says that the Apollo RT6 autonomous vehicle is ready to provide driverless service as the company moves toward a future in which taking a robotaxi will be half the cost of taking a taxi to...

(1/3) Very interesting development for autonomous driving!

A key part of the case Tesla has been making about their approach (vs Waymo) is that they can bring the cost down by a lot and scale up production/access because they don't use LIDAR.

Amazing PhD opportunity with Jakob (@jfoerst.bsky.social) offering time split between Oxford and FAIR!
Note that the deadline is Dec 2nd!
x.com/j_foerst/sta...

Glad this is taking off! I'll be posting a lot more here than the other place (hopefully!)

Oh no checkpoint overwrite bug?