Service Status

Resolved

Power feed failure in Dallas affecting some nodes

07/14/2024 13:46: The following servers are down at the moment due to a power feed failure in our Dallas DC. Our DC techs are working on getting them online ASAP.

dalpremium17
dalvz7highram15
dalvz7highram14
dalpremium9
dalpremium10
dalpremium12
dalvz7highram10
dalvz7highram24
dalvz7highram27
dalvz7highram28
dalvz7highram31
dalvz7highram32
dalvz7oldyearly1
dalvz7oldyearly2
dalvz7highram16
dalvz7highram74
dalvz7highram97

Update: 07/14/2024 16:01: All of the nodes are back online.

Resolved

UK Outage

Tuesday, 18 June 2024, 05:42 - Our a few servers in our UK location are currenty unreachable. We're working with the engineers on site to bring your services back online ASAP. We'll update this announcement as soon as we have any news.

Tuesday, 18 June 2024, 06:21 - Services are up and running already. RFO to be posted as soon as possible.

Resolved

dalpremium17 outage

Friday, 07 June 2024, 04:47 - We're investigating an issue with dalpremium17. We have no details at this point. We'll keep you updated.

Resolved

dalvz7highram15 and dalvz7highram16 outage

After the power failure which has affected all the servers in our Dallas location, we are not able to bring dalvz7highram15 and dalvz7highram16 nodes back online due to disk failures. We are still working on a resolution for these 2 nodes and we will update this announcement once we know more.

Update: Monday, 27 May 2024, 03:57: We did everything in our power to make the disks online however due to a firmware bug in them unfortunately, after hours of work, we are not able to read data from the disks of these 2 nodes.

We are re-creating the VPS's on these nodes in new nodes. If you have nightly backups addon purchased through us, we are going to restore your last backup to a new node.

We are going to send emails to the affected clients once this work is done.

We apologize for the inconvenience caused by this.

To remedy this situation, we are not going to use this particular SSD brand in the future.

Resolved

Dallas Outage

Sunday, 26 May 2024, 6:22 - Our Dallas location is currenty unreachable. We're working with the engineers on site to bring your services back online ASAP. 

Sunday, 26 May 2024, 07:34: Our datacenter has let us know that there is a massive power outage in Dallas, TX which is affecting a lot of datacenters and residents currently. We are waiting for the engineers to handle the problem.

Sunday, 26 May 2024, 08:24 - Please avoid ticketing in asking if there's any estimate, do you have any updates, etc. Support team doesn't have any new or different information that we wouldn't be putting here already. 

Sunday, 26 May 2024, 09:13 - this appears to be resolved and services back online. We're keeping an eye on things to make sure everything is active. 

Sunday, 26 May 2024, 09:18 - Some host nodes are not coming up. We're taking a look at these now. 

Sunday, 26 May 2024, 16:51 - All nodes except dalvz7highram15 and dalvz7highram16 are online. A separate announcement is created for those 2 nodes.

Resolved

ukvz7highram1 down

Friday, 26 April 2024, 12:54 - We're investigating an issue with node ukvz7highram1. We'll update this posting as the situation progresses. 

Friday, 26 April 2024, 15:42 - This continues to be escalated with the datacenter. We'll update this posting as the situation progresses. 

Friday, 26 April 2024, 16:23 - The datacenter is looking for a server to move the disks to.  We'll update this posting as the situation progresses. 

Saturday, 27 April 2024, 06:58 - The datacenter has offered a temporary server to move our disk array into, which we have accepted while waiting for a replacement server to arrive. We'll update this posting as the situation progresses.

Sunday, 28 April 2024, 09:11 - We continue to wait for the datacenter to complete this task. We'll update this posting as the situation progresses. 

Sunday, 28 April 2024, 10:36 - The temporary server is non-functional. We're asking them to put the disks into another server so we can migrate VPS to other host nodes in this location. We'll update this posting as the situation progresses. Sunday, 28 April 2024, 14:10 - Node is back online but with about 10% of the RAM it should have. We're working on migrating VPS from this host node to other host nodes in this location. 

Sunday, 28 April 2024, 19:48 - This is complete and everyone affected by this action is back online. 

Resolved

dalvz7highram41 power supply issue

09/29/2023 12:33 - we're noting an issue with node dalvz7highram41 and have pinpointed it to the power supply units. We are replacing these now and will update this posting when complete. 

09/29/2023 12:52 - this has been fixed and the server is back online

Resolved

Hardware problem causing downtime in dalwin22

A hardware addon card is causing downtime in dalwin22 at the moment.

We are working with the DC technicians to replace the card and bring the server back online as soon as possible.

Update 13:27: The bad card has been replaced and the server is back online.

Resolved

VPS migrations for some nodes is on-going

We are currently migrating the VPS's of the following servers to new servers:

dalvz7highram43
dalvz7highram45
dalvz7highram46
dalvz7highram48

If you have a VPS on these nodes, it will experience a few minutes of downtime as mentioned in the email that has been sent to you.

We will update this notice when everything is migrated.

Update 05:53 AM: All migrations have been completed.

Resolved

Emergency migration from dalvz7highram18

Due to a storage system backplane problem, we are migrating the VPS's off dalvz7highram18 as the problem is not resolvable while the physical server is online.

By performing this migration, we are ensuring a future unexpected downtime and/or data loss for your VPS.

You will experience a brief downtime while the migration is happening.

We apologize for the inconvenience.

Update: All the migrations have been completed.

Resolved

Moving off of uklegvz7highram4

Due to hardware problems occured in uklegvz7highram4 yesterday, we are migrating all the VPS's off of this node to a new node as a precaution.

You will notice a few minutes of downtime while your VPS turns off on the old node and powers on on the new one. There shouldn't be more than 15 minutes of downtime during this.

Let us know if you experience a longer downtime.

Resolved

Los Angeles Network problems

There are on-going network problems in Los Angeles. We are working on it.

Update 05:10: This has been resolved. The RFO from the datacenter: "There is a forwarding-issue with our Core. We have rebooted it to quickly fix this issue."

Resolved

UK Connectivity issues

Sunday, 02 April 2023, 11:59 - We're having connectivity issues in the UK. We're in touch with our DC & Upstream and are working to get this resolved. We'll keep you updated.

Sunday, 02 April 2023, 14:57 - The datacenter states that this issue has been resolved. Please open a new ticket if you're still noting a connectivity issue. Thank you. 

Sunday, 02 April 2023, 16:52 - we've noted this has occured again. We have notified the datacenter. 

Resolved

LA outage

Sunday, 26 March 2023, 12:12 - We're investigating a Los Angeles outage and will post more information here as we have it.
Sunday, 26 March 2023, 12:56 - Services are now operational. We'll post an RFO as soon as we hear from the DC.

Resolved

dalvz7higiram24 outage

Thursday, 16 March 2023, 09:10 - An emergency node reboot has been initiated on our end due to a kernel update. 
Thursday, 16 March 2023, 09:33 - The node has been rebooted. All VMs are up and running already.

Resolved

Power and network problems in London Location

We are aware of the power and network problems that have been going on in London location.

We are in touch with the DC to get these resolved ASAP.

Update: The problems have been resolved and we are awaiting RFO from the DC.

Resolved

dalvz7higiram23 outage

Wednesday, 08 February 2023, 08:59 - We're investigating an outage with dalvz7highram23. We'll post more information as soon as it is available.
Wednesday, 08 February 2023, 09:30 - Kernel panic has caused a node reboot. The server has been booted back and all VMs are up and running already

Resolved

Hardware problem with dalvz7highram50

We have detected a hardware problem on node dalvz7highram50 that is not being able to solved. Therefore, we are migrating the VPS's on it to another node.

You will experience a downtime around 10-15 minutes during the process.

We will update this announcement once the migration process completed.

Update 05:28 AM: All VPS's have been migrated.

Resolved

LA connectivity issues

Tuesday, 13 December 2022, 10:26: We're investigating Los Angeles connectivity issues now and will post more information here as we have it.

Resolved

ukvz7highram4 - Emergency preventative migration

Due to hardware problems occuring in our UK node, ukvz7highram4, we are migrating all the VPS's off of it.

You should experience a few minutes of downtime during the migration.

Your data and IP will be kept.

We will update this announcement once this is finished.

Update: All the VPS's have been migrated.