FA+
Shop
Server Move Rundown
5 years ago
General
We completed our server migration, and it went off pretty smoothly (except for the occasional micro-outage, but we're working on 'em!).
We've been planning this upgrade for several months behind-the-scenes, and it's taken a lot of work to support a new infrastructure and required a heckin' amount of data to be transferred. Moving about 20TB isn't the world's fastest task. To help reduce a big chunk of the transfer time we bought a 14TB hard drive, filled it with as much content as we could, and shipped it cross country to our new data center on the west coast. You don't want to know how long it takes to copy that much data. Days. Days worth of copying. Imagine staring at the file transfer window and it just never leaves 99%. You keep thinking "Any minute now. It HAS to change!" But does it? It's digital molasses.
The rest of the data has been slowly syncing over for the past few weeks, quietly running away in the background while the site was running full time. Our final sync required a bit of down time to complete the move, which had the site in read-only for roughly 21 hours. I understand that was frustrating for a lot of people, but was necessary in order to finish the move to our new home.
Thankfully, we all had Pilaf (PRAISE BE THE QUEEN!) to keep us company during that time.
And with that, FA's infrastructure is entirely new. Brand new. And modern! Prior to that, FA's infrastructure was, and I'm going to be kind here... historic. A frankenmix of old and new. Our oldest server still in production was Novastorm, a Sun Fire x4150 that was the top of the line in 2008. In fact, that's the year we bought it. A 12 year lifespan for an active production server is legendary (Novastorm was our previous database server prior to acquiring Finn and Jake). While we've added multiple servers added to our arsenal since, Novastorm was the server that kept chugging all those years. The little server that could.
I'd like to give Yak a shout out and thank him. He owned this entire migration start to finish, and totally deserves a thousand otter pats on the back.
I'd also like to thank FA+ subscribers for helping supporting the site. I'll be real: asking $5 a month to support a website is a lot to ask, especially in a post-covid world. A lot of people are struggling, and we appreciate everyone who is able to offer their support. Servers cost money, and coding time costs even more money. Thanks to you, we were able to help fund this project. We've got a lot of fantastic new tech that'll fuel FA for quite some time. Every FA+ sub really does help the site, help us expand, and invest into our resources. We're grateful to all of you who've helped out.
Oh, and we have some neat things in store to use that new power, too. But more on that when the time is right. =3
But seriously, all this new hardware... MUAHAHAHAHAHA!
We've been planning this upgrade for several months behind-the-scenes, and it's taken a lot of work to support a new infrastructure and required a heckin' amount of data to be transferred. Moving about 20TB isn't the world's fastest task. To help reduce a big chunk of the transfer time we bought a 14TB hard drive, filled it with as much content as we could, and shipped it cross country to our new data center on the west coast. You don't want to know how long it takes to copy that much data. Days. Days worth of copying. Imagine staring at the file transfer window and it just never leaves 99%. You keep thinking "Any minute now. It HAS to change!" But does it? It's digital molasses.
The rest of the data has been slowly syncing over for the past few weeks, quietly running away in the background while the site was running full time. Our final sync required a bit of down time to complete the move, which had the site in read-only for roughly 21 hours. I understand that was frustrating for a lot of people, but was necessary in order to finish the move to our new home.
Thankfully, we all had Pilaf (PRAISE BE THE QUEEN!) to keep us company during that time.
And with that, FA's infrastructure is entirely new. Brand new. And modern! Prior to that, FA's infrastructure was, and I'm going to be kind here... historic. A frankenmix of old and new. Our oldest server still in production was Novastorm, a Sun Fire x4150 that was the top of the line in 2008. In fact, that's the year we bought it. A 12 year lifespan for an active production server is legendary (Novastorm was our previous database server prior to acquiring Finn and Jake). While we've added multiple servers added to our arsenal since, Novastorm was the server that kept chugging all those years. The little server that could.
I'd like to give Yak a shout out and thank him. He owned this entire migration start to finish, and totally deserves a thousand otter pats on the back.
I'd also like to thank FA+ subscribers for helping supporting the site. I'll be real: asking $5 a month to support a website is a lot to ask, especially in a post-covid world. A lot of people are struggling, and we appreciate everyone who is able to offer their support. Servers cost money, and coding time costs even more money. Thanks to you, we were able to help fund this project. We've got a lot of fantastic new tech that'll fuel FA for quite some time. Every FA+ sub really does help the site, help us expand, and invest into our resources. We're grateful to all of you who've helped out.
Oh, and we have some neat things in store to use that new power, too. But more on that when the time is right. =3
But seriously, all this new hardware... MUAHAHAHAHAHA!
There's so many peeps here who never put any tags in their submissions, and it makes it much harder to find content here. Which is why usually we have to resort to searching for it on e621.
Also we literally just retired one of our two NTP servers, a Dell PowerEdge 1850.... from I think maybe 2004... and we have other 1850/2850s, as well as 1950s, and other such stuff from eons ago!
Gotta love putting the new stuff in though!
Am very curious to know what FA's internet bandwidth is. :o At my job, we've got a 10gbps (PHY) fiber uplink to the network core that's supposedly rate-limited to 2gbps, but not actually limited in any real way, and we could theoretically burst up to the full 10gbps.
Congrats to you all on the migration working and being on new stuff! How long think will ride on the new hardware if had your way?
Redefining the Transport Layer - with bubble wrap and a FedEx van XD
(nice work BTW, that's one bitch of a migration to undertake, kudos!)
It's really awesome to see FA getting new hardware, can't wait to see what's in store for the future!
Yall gonna auction off the old servers?
I don't need to know. I already KNOW.
I'm always curious to know exactly what services are running that makes FA.. FA. Obviously you got your webserver, your database, your thumbnail server and storage. But there's more. I can feel it.
It's a bunch of these puppies chock full of RAM and SSD storage. There's no more one individual server powering anything. It's a completely virtualized platform with no one single primary server.
I'm not complaining though, good job on the server migration!
Is anthing planned to replace Flash if it dies out at the end of this year?
However, at a certain date/time we will cut off new Flash uploads and focus on providing video support. I don't have any ETA on it, but it's something we're looking into.
You can see a demo of it here. It works for maybe 50% of the Flash files out there. Anything using ActionScript 3 still leaves a bit to be desired.
Thank you for all the upgrades!
You never had the chicken pox?!
Damn, glad you're ok now.
Hope you got vaccinated for it?