Subeta

that was ALMOST TWENTY YEARS AGO

Kumos will be down for a few minutes while we give it a lil more OOMPH

🎃 Check out the bonus pumpkin patch! kumos.subeta.net/forums/t/941...

Site is going down for a few minutes while I try to resolve some of the lag 🐌

That creates a downstream effect of pages taking very long to load, compounding the problem. Anywho, I've just updated all of the configurations to work the way that we expect. Thank you!

And when it does that, the only time it can add more information is when something expires (every cache item has an expiration, usually 60 seconds). So it just has to sit on your request until it get put data into the cache, or eventually gives up and times out.

During the migration, we moved to a new set of servers for running Redis, and they didn't get the same configuration as our old servers. We run Redis fully in memory, and it's meant to eject keys that are old/haven't been looked up in awhile. It was not doing that 💀 So it filled it VERY QUICKLY

We use a database system called Redis for caching information on all of our sites, to make it faster than getting from the database. You give something a "key" (e.g. "here are all of user 1's pets) and then we can easily pull that information later. It's great!

kumos.subeta.net coming back online - sorry for letting it crash overnight!

Having some difficulty bringing up the main site - some issues with it connecting to the database. I'm investigating now, but the rest of the sites are up and running!

ALL RIGHT, everything is transferred and things are looking green. I'm going to start bringing things online one at at a time, starting with avatar generation.

Once that is up, new.subeta.net will come online, then kumos & legacy with the bathhouse streaks and rollover being run.

I cannot believe I used to do livestreams during this

going to have to do something about user shop logs, there are over 200,000,000 rows in the database. We've never cleared them!

Will make a feedback thread making sure I'm not stepping on any rake by clearing them, but then want to start... doing that

Added an embed from bluesky to the down page (they're so nice, and don't require being logged in, how nice!) and extra verify that I'm going to make sure to protect the bath streaks 💖

Screenshot of our indexes being created after data insertion was finished

The data is all in! Now it's on to phase 3, which is adding those indexes I was talking about.

This tells the database how to find the things we ask for most often FAST

(if you're in the bay area I hope you're staying safe & dry tonight, it's a big storm!)

We can come back up without the activity logs fully in there, unfortunately I dont' think you'd all let me bring the site back up without all of the wardrobe items fully intact 😂

Time has continued to be a lie, we are now entering the second hour with under ten minutes remaining, but appear to be close to the finish line.

SO CLOSE! 2 tables remain. all of your wardrobe items, and your "activity logs" (the backend logs that we use to track every sP taken, item given, etc)

as you can see 😡

a lie from the computer saying that there are 11 minutes left, when there has been 11 minutes left for an hour

this is where you start to get in to trouble, when you trust the computer and the lies it peddles to you. it has been 11 minutes left for an hour.

multiply that by 20 years! then, we have to do a process called "indexing", so that it's easy to look things up by userid, username, etc. That has to go and re-scan all of the data, and apply the appropriate indexes to it.

Many rows of data being inserted back into the database

To give you an idea of what is going on, here is output of what is running right now.

It has to re-insert every piece of data on Subeta into our new database which is a LOT - if you think about something like news comments, forum posts, quests completed, logs....

😰 we're having the first big storm of the season in northern california today, and my power just went out ⚡

which.... yes, means i had to start over 😭 now running from a temporary server set up to prevent that being an issue again

there are a LOT of different factors going on here starting with different versions of everything. The different mysql (our database) versions export their data (and then expect the input data) in SLIGHTLY different ways, and don't error until you've made it 85% through a parallelized import

Still working on migrating the information 😭

🛁 on the bath house - i wrote a script that gives every a check-in for the last two days (and i'll extend it more!) if they missed the check-in and had recently

so: i'm not going to give another time estimate because I've been wrong every time, but it's still going!

The real problem: you have all collected SO MUCH STUFF over the last 21 years. it's still working on importing the `user_items` table, which is your inventories. it's moving TRILLIONS of items from over the years, from many accounts that haven't seen their owner in just as many years

I haven't had to do a *full* Subeta migration in a few years now, and a lot has changed since the last time. Over the last week I've been prepping for a similar move (thinking we had a few weeks at least!) and running tests

😮‍💨 woof, it's been a long night / day! the database migration is still ongoing, there was a problem overnight and the script bailed in the middle