@roymiles.bsky.social
Research Scientist @Noah's Ark Lab, Huawei PhD Imperial College London roymiles.github.io Working on multi-modality and efficient ML
forgot the hashtag #neurips2024 😅
09.12.2024 18:02 — 👍 1 🔁 0 💬 0 📌 0
Hey! I will be at NeurIPS @neuripsconf.bsky.social this week (friday 13th) presenting our paper VeLoRA.
tldr; simple method for compressing gradients with a coarse reconstruction during the backwards pass. Significant memory reductions while being complimentary to LoRA!
github.com/roymiles/VeL...
Added!
28.11.2024 16:25 — 👍 0 🔁 0 💬 0 📌 0Added!
28.11.2024 16:13 — 👍 0 🔁 0 💬 0 📌 0Hey! I have added you
28.11.2024 15:52 — 👍 1 🔁 0 💬 0 📌 0Added!
28.11.2024 15:51 — 👍 0 🔁 0 💬 0 📌 0Hey! I have added you
28.11.2024 14:20 — 👍 1 🔁 0 💬 0 📌 0Hey! I have added you
28.11.2024 13:45 — 👍 0 🔁 0 💬 1 📌 0Cool! I have added you
28.11.2024 13:28 — 👍 1 🔁 0 💬 0 📌 0Of course! I have added you now
28.11.2024 13:28 — 👍 0 🔁 0 💬 1 📌 0Cool! I have added you.
27.11.2024 22:01 — 👍 0 🔁 0 💬 1 📌 0Haha I wasn't really thinking about it like that. In either case I have added you to this one too.
27.11.2024 15:50 — 👍 2 🔁 0 💬 0 📌 0Added you in 😁
27.11.2024 15:11 — 👍 1 🔁 0 💬 0 📌 0Hey, I have added you!
oops yea I did see that one. Perhaps this one can be a bit more broader with other modalities.
Added both of you!
27.11.2024 15:05 — 👍 1 🔁 0 💬 0 📌 0Added!
27.11.2024 15:05 — 👍 0 🔁 0 💬 0 📌 0Cool, added!
27.11.2024 13:45 — 👍 1 🔁 0 💬 0 📌 0Hey! Yea yours probably overlaps a lot 😅 I have added you and will add some others from there too.
27.11.2024 13:24 — 👍 1 🔁 0 💬 0 📌 0Sweet, I have added you!
27.11.2024 13:18 — 👍 1 🔁 0 💬 0 📌 0I made a starter pack for people working or interested in multi-modality learning.
It would be good to add lots more people so do comment and I'll add!
go.bsky.app/97fAH2N
Our BMVC poster is today!
Knowledge distillation can work in cross-task settings and even with a randomly initialised teacher! Our inverted projection decomposes into knowledge transfer and spectral regularisation, enabling teacher-free distillation with improvements on ImageNet-1K at no extra cost
Hey, this is great and relevant for me. Would love to join if there is still space!
23.11.2024 05:58 — 👍 1 🔁 0 💬 1 📌 0This is really great thanks! Could I be added to the AI starter pack too 😅?
22.11.2024 20:12 — 👍 2 🔁 0 💬 0 📌 0Would be great to be added too!
22.11.2024 20:06 — 👍 1 🔁 0 💬 0 📌 0Some related work worth checking out if you find this cool.
mbaradad.github.io/shaders21k/ - learning good visual features from procedurally generated images.
arxiv.org/abs/2403.14494 - distillation from randomly weighted teachers.
Sounds great! Would love to join if there is space.
21.11.2024 17:34 — 👍 0 🔁 0 💬 0 📌 0p.s. please add me to any starter packs! 😁
21.11.2024 08:24 — 👍 0 🔁 0 💬 1 📌 0Hi Bluesky!
I'm currently working on multi-modality learning and efficient ML (mobile devices).
Hoping to regularly post about any interesting topics I find cool (ai, maths, physics).
Looking forward to seeing how this platforms grows and meeting you all!
Hi Kostas!
Could you add me as well? :D
