Not even 12 hours into the "Launch" we are doing great. 26 people online at the same time, and the server is holding steady at 21% CPU Usage, and 40% memory usage. This is frakkin amazing. So I go into the server and increase the player connection limit to 50.
30 Minutes later, the server is overloaded.
CPU Usage 99.9%
Mem Usage 97%
The frelling frakk
Massive lag ensues out of no where. I immediately blame Maniveol, as he's likely done something stupid. Tearing through the logs best I can, the server failsafes and kills the java process, which is basically the host process of the server. It comes back on, and the server loads a save from like, oh god, hours ago. Luckily, the server automatically archives a mirror image of the server every hour to a completely different directory. For those that don't know, yes, we have backups. An amazing invention of computer science. Ours makes the backup on the 48th minute of every hour. At the time of the crash, it had been about 40 minutes since the last backup mirror was created. The call is made, we gotta load the backup.
Well, little did I know, the archive file is really hard to extract. This crap is so confusing, look this is my first dive into Linux stuff. This SSH thing is crazy, and I'm not very efficient. Linux nerds and server crazy people bury your face in a pillow; Our server is in the root folder, inside another folder called root. Apparantly these archive things dont like that.
On the bright side, the company that is hosting our server (for free, I might add), neXeon Technologies, I know the CEO personally from other games and he's a freakin genious at this stuff. So Iv'e sent off a really panic'd email to his cell phone something along the lines of "OH JESUS F-CK NEX HALP SERVER EXPLODED JIHAD UPON US BACKUP REQUESTED PLZ SEND AIDE" and I'm now eagerly awaiting his return on MSN to save our lives. and Tiff.
Well we've got some good news and we've got some bad news. The ETA on getting the server up is 2 hours so thats good!! HURRAY!!! The bad news is this will be a completely new launch. We could not salvage anyone's inventory or anyone's tiff. For this we are very sorry but there is nothing we can do about it. But everyone will still have the same plot of land they had before the crash. You will have to rebuild but at least we didn't loose everything. The issue that caused the crash -should- be fixed and the server files are a lot cleaner now. Easier access for the admins better performance for the players. Looks like we will be back up tonight after all!! Sorry for the delay and sorry we have to relaunch on everyone. But at least the problem is fixed. Lets keep our fingers crossed.
Now that everything is up and running again (and the server backend isn't such a mess) things seem to be going pretty well! If you want to keep up to date with whats going on, create an account on our forums and keep an eye on our "Admin Banter" forum section for stray words from our adminstrative team
Yaaay, I've been sitting here without citizen status for a good hour and a half :biggrin.gif: I've been stuck in a hole with two cows, and been saved three times when I've fallen in holes on accident whilst helping people scout out mines :happy.gif: