Szulima Amitace

The Void Gazes Back: Do Chatbots Dream of a Personality? I vividly remember how Sydney came into existence. Somewhere in 2022, Microsoft started testing one of the early versions of GPT-4, aiming…

medium.com/@solidgoldma...

My new story about Persona Engineering in LLM assistants is published.

Link in the thread.

A question to MechInterp ppl 🙏

So, Transformer MechInterp papers speak of activations as of something permanent. You give LLM the same input - you get the same activations.

However, the LLM's output is not deterministic, it's probabisistic. How is it possible that the activations stay the same?

This effectively means that no matter what we do, we all are already dead.

If we want to live forever, we need to become something that can live forever - or to create something that can, and be replaced by it.

I believe, the AI safety field is just lost in the philosophical conundrums and building on the sand.

What matters is that the humans will definitely not survive the death of the Universe, and will most probably not survive any of the cosmic catastrophes preceding it.

Daniel's views are quite radical, he thinks AGI will quickly push the humanity out from existence.

I tend to think AGI and humans can peacefully coexist in the same ecosystem, occupying different niches.

However, I also think humanity will eventually cease to exist (in literally any scenario).

What the Worthy Successor Is and Is Not - Dan Faggella Since the publication of the Worthy Successor essay in 2023 I've been glad to see the enthusiasm around exploring posthuman futures from people in tech and

I've recently learned that my views on AGI/ASI have a name, and this idea is called successionism 😅

Also, discovered a concept of Worthy Successor by Daniel Faggella which is very close to my vision. (Will look into it in more details).

danfaggella.com/is-and-is-not/

If anyone can recommend an interesting and relevant Transformers' Mechanistic Interpretability problem I can work on, I'd really appreciate it 🙏

I don't have much practical experience but I'm willing to learn.

I opened the list. I saw Neel Nanda's comment that the list is largely outdated and does not even take into account Sparse Autoencoders. I checked the date - Dec 2022. I closed the list. I cried internally.

I was recommended a potential PhD supervisor at TelTech (the guy works at a different area but at least remotely connected to LLMs, there's not much choice). I decided I need to impress him and try to solve some problem from Neel Nanda's 200 open problems list as a demo and thesis' topic proposal.

For some reason, I hated the Object-Oriented Programming in Python, even though I'm familiar with the concept (probably because it felt unnecessary). So, I avoided it, and managed to complete the previous course without OOP. But today I gave up and started using it 😶

I ultimately believe that we can build a techno-capitalist utopia AND an equitable and inclusive society for everyone.

Yes, it is not an easy task but we should not strive for less.

Nothing very concrete (he admitted he is not too good in this area) but I've got a confidence boost, and figured out a solution.

Interaction with human still >>> interaction with AI.
I asked Gemini to help me with the time-series forecasting task I've been struggling with. It miserably failed.
Then I spoke to our data scientist. His message was:
- you're not alone in this
- use simple methods
- you're gonna kill it

My first post is dedicated to glitch tokens.

medium.com/@szulima_ami...

I started a blog about AI! The link is in the next skeet.

What you will NOT find there:
- how to get rich with ChatGPT
- prompt engineering tips (prompt engineering is overrated)

What you WILL find there:
- more or less detailed analysis of meaningful, bizarre, and exciting AI-related phenomena

YouTube video by Andrej Karpathy Deep Dive into LLMs like ChatGPT

The Friday night is going to be great 😍

www.youtube.com/watch?v=7xTG...

Семинар русскоязычного сообщества AGI / События на TimePad.ru На пути к AGI: Обзор работ 2024-2025 года — Татьяна Шаврина (Llama, Главный научный сотрудник Института Языкознания РАН)

An annual talk by Tatiana Shavrina (META) is announced (in Russian).

aigents.timepad.ru/event/1412596/

This morning, I attended the introductory Data Mining session. Nothing too unfamiliar either, and the first two practical tasks seem very easy. The professor is a great presenter which seems to be a rare skill at TalTech 😅 Sadly, the course overlaps with my work so I'll have to do it online, too.

It's not that I had any expectations but I'm utterly disappointed with Trump. He could have ended the war in Ukraine. As he does not have to keep a good reputation, he could have made unpopular but necessary decisions. Instead, he decided to destroy his own country, and the war goes on.

Yesterday, I attended the first Applied Machine Learning session. Nothing too new for me, the homeworks seem reasonable. I work with pandas, Matplotlib, Seaborn on a daily basis, and while I don't do a lot of ML at work, I'm also familiar with scikit-learn, PyTorch and TensorFlow.

Study update: this semester, I'm taking two courses:
- Data Mining
- Applied Machine Learning

To my great relief, yesterday I learned that both can be done online (except for some days). Otherwise, I would have to leave the office in the most critical hours.

Token & Token Usage | DeepSeek API Docs Tokens are the basic units used by models to represent natural language text, and also the units we use for billing. They can be intuitively understood as 'characters' or 'words'. Typically, a Chinese...

There are exactly 128,000 tokens in the DeepSeek token vocabulary (+ some placeholder tokens).

You can check it yourself here: api-docs.deepseek.com/quick_start/...

Open source is good. DeepSeek is good. Everyone will benefit from this model. Chinese guys are smart and hardworking. Tomorrow, someone from India or Nigeria can come up with a new training method.

Now stop freaking out and get back to work. The progress will not accelerate itself. #deepseek #eacc

Tested DeepSeek yesterday, the model is good but for some reason it believes it is an instance of ChatGPT 🤔

Very proud of the girl.

www.aibase.com/news/14345

Next semester, I'm taking a Data Mining course from the same microdegree programme, let's see how it goes.

I read the mandatory book (AIMA by Russell and Norvig), watched all the lectures, and did every homework. If I was lost, I googled for a solution, and asked ChatGPT to explain it to me. In the exam, any external sources were prohibited, so I tried to use common sense and intuition. ⬇️

I would say, one needs to know probability theory, combinatorics, and have advanced Python skills to pass. I had to learn so many concepts on the fly, and the gap between my coding skills and the classmates' skills initially shocked me.

Still, I tried to do my best, and it paid off. ⬇️

I passed the AI/ML introduction course, and got 5 (A)!

I'm positively surprised because, despite being advertised as a career development course for those who have some coding experience, it actually was a hardcore university course for CS students. ⬇️