Forum Downtime

[iandvl]

Folks - the service provider where the forum is hosted apparently experienced drive failure sometime early AM. Spike brought it to my attention bright and early today. In the absence of any communication from the service provider at that point in time, I presumed that the issue was with the physical virtual machine, and ultimately restored the VM from backup so that I could get it up and running asap. That worked, but since daily backups are scheduled for once a day at 13h00 (this is provided by the SP - I cannot configure this) it means that any posts from 13h00 yesterday have been lost. Sorry about that.

The joke is that just after restoring the latest backup, I received email from the service provider mentioning the fact that they had a critical hardware failure on the host. Which means that if I had perhaps not restored the backup, things may have been as they were. Maybe. Maybe not. No idea. If the drive failure resulted in my database crashing, I'm pretty sure there would have been other issues as well.

I guess I am going to have to not rely solely on the SP and make more regular backups on the host itself. I've got an idea to implement regarding this, but it is going to take a little scripting time which I don't have time for right now. So I'll look at getting this working over the weekend.

[iandvl]

Just closing the loop here. I have scripted a local backup thing which will sync forum content and dump the databases locally every hour. These local backups I keep for 24 hours.

This will allow me to rollback using the SP's backups, but also to roll forward hourly should the need arise. ie: Just so that I do not miss posts that may have taken place from the last SP backup. There is obviously still a slight window that posts could be lost. For example: should somebody make a post and something crashes before the next hourly backup, the post will be gone. But this provides a little more redundancy than previously.

[Peter Connan]

Thanks Ian
I really think that's not a problem.
Even geriatric me can still remember at least roughly what I said an hour ago if it's important.

[iandvl]

Thanks Ian
I really think that's not a problem.
Even geriatric me can still remember at least roughly what I said an hour ago if it's important.

Yes - I know. I've always relied on the 24 hour backup thing and it has worked well - here especially. Another forum hosted on the same server is a little busier than here and so I lost quite a few posts there with database crash. Anyways, the new thing works and makes everything a little more "robust".