Connection Issues Conclusions
We got a call from Travis Stoliker, Director of Marketing for Liquid Web Inc., Thursday morning as as we were promised. I’m just going to copy the bullet points for you that we were sent previous to the call.
- Communication procedures
- Staff notification system - Help clearly communicate issues
- PIMS alert updating immediately
- liquidweb.com/support notification updated immediately
- Development making tool to automate black holing of IP’s; This will remove the need for us to wait for ATT
- Adding significant connectivity to mitigate large DOS attack
- Added a wallboard for the networking team to monitor the network status in real time persistently
- The Sonar Monitoring team, which monitors our customers servers, now monitors our network too
- Improved supervisors ability to deal w/ defiant customers which are affecting our network
Our questions previous to this conversation were these: (a) Will there be additional connectivity added so that if one pipe goes down, the others can pick up the slack? (b) Will there be a method in place so that we can get immediate and specific information on the situation and not be told by someone picking up the phone that they don’t know? (c) AT&T. Just AT&T - fix them or drop them. (d) I can see not terminating someone the first time, but the second time, third time? Since this one site was attracting the attack, why was it allowed to remain after it was clear that the procedures in place were not dealing with the situation adequately?
While I would have liked to see these things addressed last month, LiquidWeb did not let the grass grow underneath their feet after it became fairly clear that this was a serious issue that could, to borrow a tech support phrase, be “duplicated” with the same “error”. Repeatedly.
Their response has gone above and beyond “adequate”. They have already addressed our ability to get information, their ability to throw people off who put the network at risk repeatedly by their deliberate irresponsible choices (and while I won’t go into it, there’s someone I’d like to slap around a bit, and they don’t work for LiquidWeb).
Probably the most important developments are that LW is working with AT&T so that they can nullroute themselves, and while that will take some time, the additional connectivity will be here this week to enable re-routing, which was not possible during these last issues.
In short, we’re impressed. Someone once told me that you really can’t judge a company by what they do when things are going well, you judge a company by how they handle things when things are going badly. Frankly, LW didn’t do the greatest job during the last few events, but they clearly have recognized, taken responsibility for, and apologized for what has happened and are sinking a lot of money and time and effort both in making sure it will never happen again, and explaining to people what they are doing to ensure that it will never happen again. They’re not trying to weasel out of anything. That’s important.
So, we’re not moving, everyone breathe, and at least this spurred us to diversify to prepare for things like this a little bit. (And we are, admittedly, getting a kick out of having Weedle all to ourselves). We’re currently eying the Harpertown, but we’re afraid if we get a new dual processor quad core, all the other dual processor dual cores will be jealous.
We’ll let you know.
Tags: connectivity, LiquidWeb
