Service Status

Resolved

dalpure6 is down

We are aware that dalpure6 is down. We are working on it.

Update: It was a kernel panic. We have rebooted the server and your VPS should come back online soon.

Update: Now, the node is having a network issue. We are working on it.

Update: This issue is resolved. It was caused by multiple 10G DDoS attacks to multiple IPs that our filtering couldn't handle.

Resolved

node3 is being rebooted

node3 has again faced IO lock up issues and is being rebooted. Your VPS should come back online in the following 30 minutes.

Update: 09:27 AM: All VPS's are online.

Resolved

dalpure1 is under DDoS attack

dalpure1 is under DDoS attack. We are working on it.

Update: 07:34: This is mitigated.

Resolved

dalstorage3 downtime

dalstorage3 is offline and we are investigating.

Update 05:06: It was a kernel panic. We have power cycled the server and it's doing a FSCK now. VPS's should be back online in around two hours or so.

Update 07:40: Automated FSCK has failed and a manual one is forced. We are performing that now.

Update 10:22: FSCK is completed with success. VPS's are booting.

Update 11:59: All VPS's are online except 2 which are calculating quota.

Resolved

node3 is rebooted

node3 has been rebooted due to IO lock up. We are going to migrate this server to a pure SSD server soon to avoid these issues.

Resolved

dalstorage2 is rebooting

We are rebooting this server (dalstorage2) due to soft lock up. Please be patient until your VPS boots back.

Update 06:59 AM: All VPS's are booted. Please open a ticket if you are having troubles.

Resolved

lapure2 is having hardware issues

lapure2 is having network hardware issues. We are investigating.

Update 7:36 AM: No updates, yet.

Update 8:35 AM: Turns out the onboard network port was faulty. We are using the other port and network is now stable. The server is currently booting. Your VPS's should be online in the next 15 minutes.

Update 8:39 AM: We are still investigating the packet loss issue.

Update 9:17 AM: The issue is resolved.

Here is the RFO:

The issue started with a major packet loss on this node. We couldn't even get into the shell to check.

First, we have suspected a DDoS attack and alerted our datacenter. They told us that there was no attack.

Then we've gotten them to replace our cables as well as their cables to the top of the rack switch. This has not solved the problem.

Then we've gotten them to swap the network cable to the other ethernet port. Unfortunately, this has not solved the problem, either.

After eliminating the hardware issues, we have started to troubleshoot the issue (very slowly, due to the high load caused by the attack!) on the IPMI console further. And then we have realized that there was indeed an attack on one of the IPs which our DC failed to recognize.

We have nullrouted the target IP manually and the packet loss issue is gone.

Your VPS is currently up and running.

We are terribly sorry for the downtime this has caused you and we want to make it up to you. Please open a ticket to redeem $5 credit on your account for the trouble.

Resolved

node1ssd kernel panic

node1ssd had a kernel panic and we are rebooting it now. Your VPS should come back online in around 15 minutes.

Update 02:22 AM: All VPS's are back up.

Resolved

Rebooting nj1

We are rebooting this node due to a unstability issue.

Resolved

Rebooting Colocrossing Los Angeles Server

We are doing an emergency reboot of this server. Please be patient.

Update 06:34 AM: This has been completed. All VPS's are online.

Update 11/30/2014 8:08 PM: The issue has come back and we are booting it again unfortunately.

Update 11/30/2014 8:31 PM: The node is back online.

Resolved

dalstorage3 Problem

We are currently working on to get the server back online.

Update 11:32: This server has been brought back.

Resolved

Network problem in Los Angeles

There is an ongoing network issue on one of the nodes in LA. If you are on it, your VPS is down at the moment. We are working on it to resolve ASAP.

Update 18:21: This is resolved. Reason for outage will follow...

Resolved

Dallas issues

We are having an issue with one server. If you are on that one, your VPS is down at the moment. Please be patient while we are looking into the issue.

Update: 14:10: This is resolved. It was a DDoS attack.

Resolved

UK Maintenance

ukpure1 is under an emergency maintenance at the moment. ETA to come back online is 20 minutes. We are sorry for the trouble.

Update 17:53: Unfortunately, this is taking very long. We are still waiting an update from the datacenter technicians.

Update 17:57: The server is back up.

Resolved

New Jersey Network Issues

We are experiencing netwoırk problems in our New Jersey location.

Update 13:46: Issue has been resolved.

Resolved

node3 Los Angeles Network Issues

One of our Los Angeles nodes is with ColoCrossing and they are having network issues. They are currently working on it.

Update: 20:25: The network is back online.

The cause of the issue was: "The cause for the incident was due to a bug in the firmware our switches utilize. It had a bit of an impact on us because we just recently completed a significant network wide upgrade of all cabinet level switches. Subsequently all of our cabinets run the same type of switch now, so when an issue occurs it can happen on more than 1 cabinet at once."

Resolved

node3ssd ddos

A large DDoS attack halted the NIC on node3ssd before we had any chance to take action. This is resolved and VMs are in the process of booting up.

Resolved

DDoS attack on the Dallas storage VPS network

There is a huge DDoS attack happening on our Dallas Storage VPS network.

Resolved

node3ssd reboot

Node3ssd had to be rebooted for emergency unplanned maintanence. The server is back online and VMs should be coming up shortly.

Resolved

More lapure1 problems

lapure1 went offline again. We are looking into it.

Update: The node is back online. However, we are going to migrate off this server this week.

Please open a ticket if you are having troubles.