Overall Service Status
Up
All services are operating normally at this time.
All Regions
| Data Center | Current | Last Day | Last Month | Last Year |
|---|---|---|---|---|
|
Up | 100% | 100% | 100% |
|
|
Up | 100% | 100% | 100% |
|
|
Up | 100% | 100% | 100% |
|
|
Up | 100% | 100% | 100% |
|
Up | 100% | 100% | 100% |
|
|
Up | 100% | 100% | 100% |
|
Up | 100% | 100% | 100% |
Past Incidents
dalpremium8 Emergency Reboot
Due to an unstability issue with dalpremium8, we have to do an emergency reboot.
All VPS's should come back online in less than 5 minutes.
Update: 01/08/2019 02:38: All VPS's are back online.
Intermittent connectivity affecting some Dallas nodes
Friday, 21 December 2018, 02:07 - We're aware of the issue, and are attempting to follow up with our connectivity provider. Further updates will be posted to this issue as we receive them.
Friday, 21 December 2018, 07:12 - Connectivity should be restored now. Please let us know if you're still seeing accessibility issues and we will engage. Thank you.
Dallas Network Issue
Wednesday, 28 November 2018, 03:31 - We're aware of the issue in Dallas, and are attempting to follow up with our connectivity provider. Further updates will be posted to this issue as we receive them.
Wednesday, 28 November 2018, 04:21 - Everything should be operating normally already, please get in touch with us in case you're not able to access your service.
Network wide port blocks
Due to recent reflection exploits involving ports 111 (portmap/NFS), 161 (snmpd), and 389 (ldap), we've implemented a network-wide filter at our network edge to avoid your services running on these ports from being exploited and used to attack other computers on the internet.
We understand that this may be disruptive to your usage, and we can provide guidance on how to adjust these services to use different ports so that you may continue to enjoy uninterrupted connectivity.
These services are not installed on our VPS by default, so if you do not recognize any of the above services or ports, you're likely not affected by this action and this message is informational only.
New Jersey Maintenance
Thursday, 04 October 2018, 15:58 - We've just been briefed that our provider in New Jersey has scheduled a 30 minute maintenance window that may have service impacts for our clients on the nj1 node. See notice below:
Event Type: Network Upgrade
Start Time: 10/9/2018 12AM EST
End Time: 10/9/2018 12:30AM EST
Location(s): Piscataway, NJ
Device(s): c8-10-b4-1.pnj1
Client Affecting: Yes
Event Summary: Important maintenance will be performed on one or more devices that may affect network connectivity to your services.
Intermittent connectivity affecting all Dallas nodes
Thursday, 27 September 2018, 18:24 - We're aware of the issue, and are attempting to follow up with our connectivity provider. Further updates will be posted to this issue as we receive them.
Thursday, 27 September 2018, 18:33 - Our provider has confirmed that this was a misconfigured network-wide-filter applied on our upstream port(s). The rogue change has now been rolled back, services are back online.
We apologize for any inconvenience caused.
NJ Maintenance
Our New Jersey Datacenter is currently performing a scheduled maintenance
Event Type: Network Upgrade
Start Time: 2018-09-24 - 03:00:00 AM EST
End Time: 2018-09-24 - 05:00:00 AM EST
Location(s): NYC Metro Datacenter
Device(s): Core/Edge Network Devices
Client Affecting: Yes
Event Summary: Important maintenance will be performed on one or more devices that may cause intermittent network connectivity issues to your services. We do not anticipate extended outages for this network maintenance.
We will continue to post updates as we receive them.
ukhighram3 offline
Thursday, 23 August 2018, 06:21 - We have just been made aware of an event concerning this node. Further investigation is ongoing, more details will be posted to this issue as they become available.
Thursday, 23 August 2018, 06:38 - Our datacenter has confirmed that they're experiencing a network event that's affecting this server. An explicit ETA has not been communicated, but we remain hopeful about service restoration soon.
Thursday, 23 August 2018, 08:40 - This remains in progress. We will update this as we have more information.
Thursday, 23 August 2018, 10:01 - Datacenter has confirmed that this is still in progress.
Thursday, 23 August 2018, 11:22 - An update from the datacenter has specified that a PDU in the rack where this server is located is malfunctioning, and they are working on moving the server to a temporary rack to get it back online.
Thursday, 23 August 2018, 13:49 - We continue to wait for the server to be moved to a different rack to be put back online.
Thursday, 23 August 2018, 15:15 - We have noted that the server is back online, and we're waiting for confirmation of stability.
Thursday, 23 August 2018, 15:45 - Confirmed that server will remain online for the foreseeable future. We will be providing one month of credit for the downtime. All VPS are online and running, so if your server is still offline or inaccessible, please reach out to us and we'll investigate further.
dalstorage4 disk array rebuild
Tuesday, 07 August 2018, 00:08 - Earlier today, our dalstorage4 node suffered a disk failure and it is currently operating in a degraded state. There unfortunately isn't enough I/O to go around, so I/O bound programs are taking a very long time to respond (due to the I/O wait). We're investigating, further updates will be posted to this issue. Customers not on the dalstorage4 node are not affected by this issue.
Wednesday, 08 August 2018, 09:03 - This situation does remain in progress. Your VPS should be online and accessible though, since the I/O contention seems to come in short infrequent waves. If you're having an issue with access beyond being slower than usual, please inquire and we can investigate.
Thursday, 09 August 2018, 08:23 - This continues to be in progress. Reboots via the client area/VM control system have been disabled as rebooting won't magically fix this situation and will only contribute to more I/O contention. If you need your VPS rebooted and are truly unable to figure out how otherwise, please ticket in and we'll review your request.
Friday, 10 August 2018, 11:46 - We're going to shut down the server to replace the drives and let it rebuild without VM contention; this will take less time than if we left VMs running, resulting less net time back to normal. This shouldn't have much of an impact on actual usage given the speeds of the server as it stands. The current estimate is 24 hours for array rebuild.
Saturday, 11 August 2018, 06:25 - All the processes successed and we have booted up the VPS's. Everything should be healthy at this point.
Dallas connectivity
Tuesday, 24 July 2018, 10:19 - the Dallas connectivity issue is acknowledged and being investigated now.
nj1 downtime
11 July 2018, 10:45 - Planned network maintenance for this server has begun
11 July 2018, 11:25 - Planned network maintenance is complete and the server is back online
Connectivity Problem in Dallas
Wednesday, 30 May 2018, 12:21 - we are aware of the Dallas connectivity issue. Datacenter can be followed here: https://twitter.com/InceroStatus - we'll update this when we hear back.
Wednesday, 30 May 2018, 12:29 - connectivity restored
nj1 connectivity
Sunday, 22 May 2018, 18:00 - we're investigating an outage of our NJ1 server. We'll update this announcement as we have more information.
Sunday, 22 May 2018, 18:40 - server is back online without issue noted.
FW intermittent network issues
Monday, 21 May 2018, 07:49 - We're aware of the intermittent network issues plaguing our Dallas (DFW) POP. This seems to be due to heavy packet loss on our upstream's end. We're collaborating with our upstream to mitigate this, thank you for your patience.
Kernel panic on dalhighram10
We have experienced a kernel panic on dalhighram10 and we are currently looking into it.
Update: All the VPS's booted back up again.
Dallas network errors
Wednesday, 28 February 2018, 10:49 - We're investigating a Dallas connectivity issue. We'll update this announcement as we have more information. This is a network issue. We are experiencing high package loss. Our datacenter is working on the issue.
Wednesday, 28 February 2018, 11:41 - Root cause has been determined: outbound network attacks from insecure memcached installations. More info here: https://blog.cloudflare.com/memcrashed-major-amplification-attacks-from-port-11211/ We're looking at how to adjust them now.
Wednesday, 28 February 2018, 12:28 - This has been mitigated, root cause was a new memcached amplification attack. If you're still seeing connectivity issues, or truly need internet-accessible memcached access for some reason, please ticket in.
nj1 network connectivity
Sunday, 25 February 2018, 13:48 - we're investigating an outage of our NJ1 server. We'll update this announcement as we have more information.
Sunday, 25 February 2018, 14:52 - the server is back online, and we're awaiting confirmation that the network is stable
dalstorage1 unreachable
Tuesday, 20 February 2018, 02:55 - We've been made aware of an event on this node and are currently investigating.
Tuesday, 20 February 2018, 05:16 - Node is back up, this was due to a storage event. VMs are now booting back up. Please expect to be contacted individually about remedies.
dalhighram25 unreachable
The node is currently unreachable from the panel and the VMs are working extremely slowly. We're already looking into the issue but it has already taken more time than expected. Please excuse us for the inconvenience.
Update: The situation is under control now, please open a ticket if you're still having issues.
Dallas network problems
Tuesday, 06 February 2018 11:34 - We're receiving reports of an unexpected partial routing failure somewhere, or similar network issue. Please don't stop/start/reboot your VPS, it won't fix the issue. When opening a ticket to inform us that your VPS is offline, could you please send a TEXT traceroute between yourself and your service with us? This will assist us with resolving the issue.
Tuesday, 06 February 2018 11:59 - The issue should have been rectified now. We'll update the announcement with a RFO.
Friday, 02 March 2018, 10:46 - Formal RFO is available at the following link: https://d.pr/f/RN2AjO