Hello Reefers,
I sincerely apologize for the recent outages and we are taking steps to prevent this in the future. I have spent the last 48 hrs buying new hardware and rebuilding this site. For those techs that want to know what happened, here it is:
Problem:
Server was running raid 5 with 3 solid state drives. Backups occur every night at 2am to a separate partition. I’m not exactly sure but the raid configuration broke and I was not able to get to the data or the partition where the backups are stored. Every month I would remove the backups and store them offsite. Unfortunately the last backup that was stored offsite was September 25th.
Fix / Solution:
New hardware was purchased and the restore of September 25 was restored. The old server has had its drives replaced and is now a repository for all the backups. Backups are stored both on SDR as well as FTP’ed to a separate repository server.
I’m still working on getting the last two weeks restored and will keep you posted.
Chad
Reply With Quote
Also, you could write a script to automatically upload backups to s3 which is pretty good for storage. Let me know if you need some help, I'm part of a software team that manages a distributed infrastructure on AWS.
Originally Posted by jessegarcia28
