On Sunday the 14th we found a new virus around noon. Nothing special there, except that this one was using a brand new exploit against a brand new vulnerability: the MS05-039 PnP hole. I was the viruslab oncall manager for the week, so I called up other oncall people to work on the case. Jarkko analysed the virus from his home office and Jarno made his way to the office to test and publish a new update to detect this critter.
We added detection with the name "Zotob". I wrote a short blog entry on the incident and we moved on.
On Monday evening I got a call from Eki from our sales team. He's passing on a request from a customer who was having hard time fighting something in their network. Turns out the customer had done many things right: they had up-to-date antivirus in their network. They had installed latest Microsoft patches on their machines. But they hadn't rebooted the machines, and they were not running firewalls on the individual machines. And now they were hit with something that was causing hundreds of their Windows 2000 machines to reboot almost constantly.
I called up Alexey who was on call now. He had just left the office but he turned back to look at the case. It seemed to be a new Ircbot variant that had been modified to use the PnP exploit. Jusu was called in too to get out an update to everybody. Alexey stayed in for several hours to build a special tool to help the customer. Late in the evening I wrote a short blog note which contained a paragraph saying "Once again, patch now."
Otherwise things looked fairly calm, with very few reports of real-world problems from the field. But there was storm in the air. On Tuesday evening I'm going out to theatre with my wife and couple of friends. "Hairspray" is playing in Helsinki. I'm worried that if something big happens I might need to leave during the show. Katrin is nice enough to stand in for me for the evening although I'm on call.
The show is excellent and we have a great night. On my way home I send a text message with thanks to Katrin. Her response "no problem" arrives at 22:36 on Tuesday night.
I wake up right after 02:00: Rich from Microsoft is calling, asking if we're seeing increased PnP activity in the net. Literally while I'm speaking with him, my phone receives an automatic alert regarding network worm activity. Uh-oh. Better check out the situation.
I get to my computer to see that Ero in our US viruslab is already hard at work on the problem: there are at least two new worms spreading aggressively. Too bad, but I have to wake up Jusu again. He sounds wake enough and starts to make his way to the office.
We're getting reports of CNN having problems. I place two calls to CNN techies to get some kind of a handle of the situation, and it doesn't look good. Big companies have lots of Windows 2000 machines in their networks. Many of them haven't simply had enough time to test and deploy the patch everywhere.
Jusu gets the update out in high-speed mode and we issue a Radar alert at 03:36.
I'm chatting on the Messenger with Simon from Microsoft's Security Response Center and he seems to think the whole case is mostly media hype. But I'm not so sure. There are now several reports of infections at places like the Financial Times, New York Times and ABC.
Bob Sullivan from MSNBC calls me for a comment and we discuss at length on how in most places the infection must have entered via infected laptops. I'm walking around my work room while talking, trying not to wake the rest of the house. Bob ends up writing a pretty sensible piece on the situation.
In the wee hours I send Ero home with thanks and start to type out a blog entry titled "The global PnP problems".
My wife wakes up around 5 in the morning. It has been really cold during the night and I'm wearing one of her pink pullovers, which she finds highly entertaining. Oh well.
I receive couple of calls from CNN Center and they are asking if I could do a live phone interview. We do this at 05:15. It ends up being broadcasted in Asia and USA but not in Europe so I didn't see it. However, old friend Nick Fitzgerald from New Zealand sends an email and says he saw it over there. I'm mailing back to confirm what he actually saw, because the last time I did a phone interview with CNN they - get this - ended up showing archived footage of Symantec's headquarters with my voice-over! This time they had actually managed to find a JPEG of me from the web.
My wife takes the 6 o'clock bus to work and I doze off on the couch, only to be waken up 15 minutes later for another automated alert text message which didn't really tell me anything new and really was the last thing I needed right now.
I wake up again at 07:30 to check out the morning news. The CNN weathergirl makes a comment about how her forecast isn't very detailed today because only one of her computers is working...
I get to the office before 9 and we try to make some sense of the mess of all these different bot variants with Alexey, Jarkko and Katrin. Katrin posts the now-legendary high-tech illustration on the topic to the web.
In the afternoon I'm working with Mark and Jaana on a press release on the whole saga. It ends up getting reprinted in over 500 different journals during the following days.
During Wednesday, Thursday and Friday we find dozens of new worm and bot variants, all recycling the same 'Houseofdabus' exploit code. New infections are reported from several large companies and I spend almost on hour on phone on Thursday with one Swiss company trying to fight it.
But overall, the situation starts calming down. Many companies were not affected in any way during the whole outbreak. Most others started getting their patches out by the end of the week.
From our point of view, this PnP saga is now a case closed. I suppose we're now waiting for the next big thing.
These outbreak weeks are getting harder to recover from every year. And we aren't getting any younger, are we?
