Service Status

Resolved

ukpure3 and ukpure4 maintenance in progress now

2015-11-09: This action is complete and both servers are back online and booting VMs. 

Our maintenance on ukpure3 and ukpure4 is in progress. Here's the email sent to all affected users:

=====

Hello {$FIRSTNAME},

We are emailing you because you have a service with us on either ukpure3 or ukpure4 nodes in the UK.

Maintenance Window
Start: Tuesday, November 10, 2015, 2:00 AM GMT
End: Tuesday, November 10, 2015, 2:30 AM GMT

Services Affected
VPS's on ukpure3 and ukpure4 nodes

Data Center
United Kingdom

Expected Impact
Downtime around 15-20 minutes

Details
We are going to perform memory upgrade on these servers for better performance due to lower storage interaction.

The datacenter technicians are well prepared for this upgrade and we are planning to complete it in around 15-20 minutes. During the maintenance, both nodes will be inaccessible on your client area and therefore, you'd receive an error when you browse it.

We are going to gracefully shutdown your VPS. If you have any software running on your VPS that's not configured to start on boot, this would be a good time to configure it that way.

Should you have any questions regarding this event, please login to your client area and submit a ticket. Please do not reply this email address as this mailbox is not monitored.

Regards,
VPSDime Team

Resolved

Accidental downtime on yearly plans in Dallas

Unfortunately, we have made a mistake and auto migrated around 60 VPS to a different network and this caused the VPS's that are migrated not to work on the new server.

We are currently migrating these VPS's back to their old node and your VPS will be accessible again soon.

We are very sorry for the inconvenience.

Resolved

dalpure4 Outage

We are aware that dalpure4 node is in Dallas is down. We are currently working on resolving the issue.

Update 01:40: We had to hard reset the node and currently it's booting up the VPS's. Your should be online in the following 15-20 minutes. Please open a ticket otherwise.

Resolved

Outage on dalwest4

10/27/2015 13:59 - We are investigating an outage of our dalwest4 server. We will update this announcement as we have more information.

10/27/2015 14:08 - We have determined the server is not showing a network link. We're looking further at the intermediate network hardware.

10/27/2015 15:02 - 10gbit NIC and fiber have been replaced, and the server is now back online.

10/28/2015 01:57 - We are facing the same hardware issue with the server again. We are working on getting this fixed ASAP.

10/28/2015 02:03 - We have got the server back online by downgrading it to 1G connection for now. We are going to schedule a complete server chassis swap in the US morning time.

Resolved

lapure1 rebooted

Node lapure1 requires a reboot, vms will be back up shortly

Resolved

Network outage in our Los Angeles location

We are aware of an network outage in our Los Angeles, USA location. We are currently looking into it.

Resolved

Outage in Dallas

Our datacenter is reporting an issue with one of their switches: https://twitter.com/InceroStatus/status/648695085387747328?lang=en

This has killed connectivity to the nodeXssd and dalyearly servers, along with daldb. This announcement will be updated as we know more.

=====

 

This is now resolved, and all services should now be back online: https://twitter.com/InceroStatus/status/648706618335473664

Resolved

Downage in Dallas. We are working on the situation.

Several nodes in Dallas are currently down. Our technicians are working to bring them back as soon as possible.

Resolved

ukpure2 reboot

ukpure 2 requires an emergency reboot and will be going down.

Update: server rebooted and appears to be back online

Resolved

seapure1 is unresponsive

seapure1 is currently unresponsive and we are working on it.

Update: This is resolved. However, some of the VPS's got auto suspended due to extreme load on the server. These are now unsuspended.

Resolved

seapure1 is rebooted

Due to a kernel panic, we have rebooted seapure1 and your VPS should come back online in the following minutes.

Resolved

ukpure1 is rebooted

Due to a lock up, we have rebooted ukpure1 node. Your VPS should come back online in the following minutes.

Resolved

Network issues with LA servers

Our upstream in LA is currently encountering some network issues. We are looking into it.

Resolved

dalpure3 reboot

We are emergency rebooting dalpure3. It should come back in 10 minutes.

Update: All VPS's are back online.

Resolved

Major LA Packetloss

Investigating High Packetloss in our LA location. At this time it appears to be DC wide.

This appears to be resolved now

Resolved

node3 outage

We're currently investigating an outage of our node3 server in LA.

Update: This is resolved.

Resolved

dalwest2 high packetloss

We are investigating high packetloss on node dalwest2.

Packetloss should be resolved. Cause was incoming DDOS

Resolved

vlan outage in Dallas

2015-02-11 2039 CST: We are acknowleding a vlan outage in Dallas at the moment. We're engaging the datacenter to look into the issue. If your server is "down" and this announcement does not say [SOLVED], yes, your server is on this vlan.

2015-02-11 2055 CST: From the datacenter, this encompasses more than just us. Their twitter feed: https://twitter.com/InceroStatus

2015-02-11 2102 CST: Datacenter fixed issue, network back online. 

Resolved

rebooting dalpure7

We are rebooting dalpure7 at the moment. Your VPS should come back online in 10 minutes.

We are frequently having lock ups on this node and we are planning to migrate VPS's to a new node soon.

Update: All VPS's are back online.

Resolved

dalstorage3 kernel panic

We have faced yet another kernel panic on dalstorage3. The server is rebooting.

We are going to send a replacement for this server and migrate the data.

Update: The server is offline. We are working on a solution.

Update: The server was keep rebooting itself. We are thinking that this was due to half applied patches by Kernel care. The last kernel has been reinstalled to avoid this issue. We are hoping this would resolve the problems on this node.

Update: Unfortunately, this looks like an hardware issue. We are going to have to replace the mainboard and the memory of this server, since it can't stay online.

Update: We're still working through the situation. We're working to boot the system so that we can disable the VM filesystem, and then remove one of the hardware components that seems to be causing issues.

Update: We have replaced the motherboard, CPUs and the memory on the server. After doing fsck 3 times which took around 20 hours alone, the server has booted and is currently booting the VPS's.