Thursday, January 8, 2015
08:09: Our dalstorage4 storage server is being rebooted for a filesystem check. This announcement will be updated when complete.
08:55: This didn't go quite as planned. We are working the situation and will provide more information when we can.
09:17: All clients on this server have been emailled the following:
We are writing to you because you have active service on our "dalstorage4" server.
The reason for the message is to inform you of a disk controller or disk error with this server. This morning, the file system began throwing egregious errors. In an effort to salvage the situation, we rebooted the server to run fsck on the VM partition, only to discover that the disk array is no longer readable, and that fsck is throwing an error that every block is unreadable. Unfortunately, at this juncture it appears that your data is lost.
At this point, we've shipped a replacement server, which will be online tomorrow afternoon, relative to the United States.
We're offering two options:
1. We can recreate your VM tomorrow on the new server, and provide you with a full two months of credit for the issue.
2. We can provide a full refund back to your payment method.
Please note that we're unable to recover your data, and please do not open tickets regarding that. This has been a hard decision that we did not take lightly, and every effort has already been made to fully recover your data.
Please open a ticket with your choice, and if you have any other questions, please don't hesitate to task.
Thank you for your understanding,