FA+
Shop
Scumbag FA...
11 years ago
Day 1: Our site broke because we couldn't scale the notifications system to handle 10,000 plus users. Make sure to clean up your notifications people, even though we never mentioned (or probably never even realized) a performance bottleneck in this area before, it's all your fault!
Day 2: Mass cleanup of notifications disabled because it's terribly hard to actually offer the service we just got done admonishing you for not using.
In the year 2000, on a 300Mhz Pentium 3 I wrote a notifications system very similar to this one from scratch in PHP and MySQL for the largest portal/forum website covering local news for my town. When we hit performance issues as number of notifications scaled N^2 to number of posts (also at around 10,000 users and 5-10 million post items), do you know what I did?
I added a per-user field (scales linearly with number of users) with a date prior to which all posts can just assume to be read. Whenever you cleared stuff, that field was reset to the oldest thing not cleared, or to "now" if you cleared everything, be it via "nuke" or by hand.
To that end, "nuke" no longer accessed or altered the large notification table, it just updated the per-user date field and allowed late night cleanup processes to trawl through and delete notification entries older than that frontier.
End result? Both size of notifications table and number of read accesses into it's index dropped by 2 orders of magnitude, increasing site performance 50x the moment I pressed "commit".
I kept that website and all services online with four nines of uptime for four years. (after which time I left and eventually the building the underfunded servers were in burned down. THAT took site offline for a good while! :P) If the site was down for 45 consecutive minutes I got death threats from our users, none of whom paid us a dime for the service.
So .. nine days of downtime? Because of database inefficiency in the "unread posts" system? really?
Yeah, I'm done with this nonsense. I'll be here: https://happmacdonald.sofurry.com/ instead. :(
I'll check back on notes and such here off and on for a week to encourage communication and then I'm pushing the hide profile button.
Day 2: Mass cleanup of notifications disabled because it's terribly hard to actually offer the service we just got done admonishing you for not using.
In the year 2000, on a 300Mhz Pentium 3 I wrote a notifications system very similar to this one from scratch in PHP and MySQL for the largest portal/forum website covering local news for my town. When we hit performance issues as number of notifications scaled N^2 to number of posts (also at around 10,000 users and 5-10 million post items), do you know what I did?
I added a per-user field (scales linearly with number of users) with a date prior to which all posts can just assume to be read. Whenever you cleared stuff, that field was reset to the oldest thing not cleared, or to "now" if you cleared everything, be it via "nuke" or by hand.
To that end, "nuke" no longer accessed or altered the large notification table, it just updated the per-user date field and allowed late night cleanup processes to trawl through and delete notification entries older than that frontier.
End result? Both size of notifications table and number of read accesses into it's index dropped by 2 orders of magnitude, increasing site performance 50x the moment I pressed "commit".
I kept that website and all services online with four nines of uptime for four years. (after which time I left and eventually the building the underfunded servers were in burned down. THAT took site offline for a good while! :P) If the site was down for 45 consecutive minutes I got death threats from our users, none of whom paid us a dime for the service.
So .. nine days of downtime? Because of database inefficiency in the "unread posts" system? really?
Yeah, I'm done with this nonsense. I'll be here: https://happmacdonald.sofurry.com/ instead. :(
I'll check back on notes and such here off and on for a week to encourage communication and then I'm pushing the hide profile button.
