ถ้อยคำแถลงคับ
Good Morning,
All sites should be working now. HOWEVER, if you changed any ips or had us change any ips on your sites they might not be working correctly and that is because cpanel sometimes does not change ips very well but that is the chance we took.
If you still have a downed domain I ask you to first change back your sites to the 38 networks and try them again in a few hours, Give them time to propagate. If it still doesn't work , please let us know.
Please put in servername (if known) and the domains that are broken, the ips you desire and we will do our best to accommodate.
Here is what happened...
After 48 hours of non stop work and configuring and reconfiguirng routers and switches to the Nth degree, we decided to try a new protocol as OSPF just wouldnt work (although my guys can configure this network in their sleep). It should have worked! But I gave the order today and onward we proceed with BGP. (which is better for us anyway)
So we got BGP up and running. I normally go to the DC and set up base camp there but my engineer asked me to stay back this time. I'm sure I was getting on his nerves by this time, after 2 days of being on his back to fix it. Smile Hes cool though. He understands I'm only trying to help. I am not a network engineer but dammit I can contribute most of the time.
So I respected Torrent's wishes and I stayed in the office (and its a good thing I did) to run tests while he was in the data center brining up BGP(a routing protocol used for failover, better for us again).
He finished and called me to ask how to wire up the server banks and other swithces. I told him how to do this on the phone, as he was plugging them in, I could tell the network was working and working good. I was really getting excited, then it crashed again. I said WAIT, what did you do? he said nothing just plugged these in. I said unplug it, it picked up again.
So there it was, a bad wire or bad server! Turned out to be a dedicated server. We had it solved!
Good thing I was in the office instead of hooking up those servers like I would have been doing while I was there. If I had been there I would have never seen the ping drop when it was plugged in. I was in the right place and the right time watching the right screeen. Good thing I listened to Torrent normally I want to be in the middle of it. I can't stand not being in the fire. If I had been there I would have missed the drop when he was plugging in servers and we would still be scratching our heads and I would probally be out of business by Monday.
It wasn't bad network equipment, it wasnt the cofigs, it wasnt the wires. It was some computer code that was infecting the network. I have no idea how this could happen. I have never seen a server crash an entire network before.
That server could have been under some kind of attack too.. Interesting thought, because it was on the 38 network so it never came up when we were running TW only, yet it was plugged in and booted up. Only when it could send and receive traffic would the network go down. I never changed its IP to a TW ip.
So believe it or not thats the story of the meltdown, SINCERE APOLOGIES TO EVERYONE. I know many of you lost money and so did I. I'm just happy its over.
Now there will be more downed domains due to yesterdays IP changes , probally..
If you changed ips today, go get the 38.'s back and utilize the 38 network again its a good network and need its traffic back.
I sure learned alot over the past few days thats for sure. I know this will only make me stronger.
Thanks to those that encouraged me every day to keep fighting. There were some very low points at times. I thank you, my wife thanks you and my 5 kids thank you.
Thanks for your support, that is if you still support me Smile
-Ts
PS.. If you are in other forums where I got bashed and feel it in your heart to take up for me and explain anything to anyone what really happened I would appreciate it.
เราเข้าใจคุณ
Host and Domain
