Title: The Cuckoo's Egg by Cliff Stoll follows the author's efforts to catch a hacker who invaded his computers at the Lawrence Berkeley Laboratories in 1986
Author: danny

Danny Warner
Lessons From The Cuckoo's Egg by Cliff Stoll

The Cuckoo's Egg by Cliff Stoll follows the author's efforts to catch a hacker who invaded his computers at the Lawrence Berkeley Laboratories in 1986. It is therefore one of the first detailed accounts of a systematic intrusion detection program.

The book begins with the author starting a new job as a system administrator for the Lawrence Berkeley Laboratory. Within a few days of arriving, he is given the task of finding the source of an accounting error. The system that he runs is billed at $300 per hour of computing time, and while the accounting error is only $0.75, they are concerned that it could be from some sort of bug, which may have more serious consequences. When Stoll investigated the error, he found that a user with no billing information had used a few seconds of computer time without being charged, and since there was no one to bill, there was no way for the system to balance its books. The simple solution was to delete the invalid account.

The next issue to appear was when the LBL received a message from a computer in Maryland detailing a break in attempt from a computer at the laboratory. It was easy to find the source of the attack because only one user was logged on at the time, an account named Sventek. Sventek was a well-respected computer researcher who was unlikely to have attempted to break into a computer without authorization. He was not even in the country when his account was logged into the computer at Berkeley This led Stoll to believe that someone, probably a student, had figured out his login information and was using his account.

Cliff Stoll decided that he wanted to track the hacker back to his source, but this was in 1986 and there were no established protocols for intrusion detection, so in many cases they had to invent new methods as they went. The first step was to set an alert whenever anyone logged into the lab. This quickly became overwhelming because the Lawrence Berkeley Laboratory supported hundreds of users at the time, so they quickly narrowed the alerts to only sound when the user Sventek logged on. Once they were able to see Sventek log on, they determined that he was coming in from outside the laboratory on a modem, but they had 50 telephone lines and did not know which one he was using. The solution for this was to attach a line printer to every modem to output everything that passed over that modem. So they borrowed as many printers from offices as they could and hooked them up to let them run overnight.

They were able to determine which line the hacker was using, so they returned the printers but kept one logging all traffic from the hacker. It showed a disturbing development; the hacker was able to get system administrator privileges on their machine through a bug in the emacs email functionality. This meant that he was able to read all other users files and erase or rewrite system files if he chose to. He was also able to modify the accounting system to cover his tracks and erase the evidence of his presence.

This is where the story becomes interesting. The conventional wisdom at the time was to close the security hole, verify all user accounts, and usually make the users enter new passwords. However, that would tell the hacker that he had been made, and Stoll was interested in tracking him down 
because he had attempted to break into military computers, and he often searched for items pertaining to nuclear secrets and SDI technology. This led them to believe that it might be some sort of spy attempting to gain intelligence on US military activities. They also believed that he was operating through computers besides the LBL, so cutting him off at the LBL would not stop him, but it would render Stoll unable to view his activity. 

Stoll then created a system that would page him whenever the hacker signed in. From his time in Astronomy, he had learned that if something was not written down, then it didn't happen, so he applied this rule to intrusion detection and began taking a log book and saving printouts to keep track of the hacker's actions. He was interested in tracking the hacker and so began to contact law enforcement. Because they still believed that the hacker was in Berkeley, perhaps a student at the university, they began with local authorities.

Soon after this they saw the hacker log into milnet computers from their system. The milnet a research network for declassified military information. The hacker was searching for keywords like nuclear and SDI. This was concerning because there was always the possibility that there may be some classified information on a milnet computer even though there should not be. But even without gaining access to classified information it is possible for someone to determine military actions by aggregating unclassified information. 

As they followed the hacker, they began to see the methods that he used. It became obvious that he was not some sort of genius who rewrote system code or exploited unknown vulnerabilities in the operating systems. Instead he was a methodical hacker who broke into systems by guessing common default logins and passwords, and once in gained administrator privileges with well-known program bugs.

The next step was to trace the hacker back to his source. The Lawrence Berkeley Laboratory obtained a search warrant from the Oakland district attorney to trace the phone line the hacker was using from the laboratory to Tymnet, their modem provider, and into the public telephone network. They saw that the calls were coming from Virginia, however they did not have a search warrant for Virginia, so the phone company would not release the results of the trace.

One of the common themes of the book is the difficulty that Cliff Stoll encountered when dealing with government agencies and bureaucracies . In many cases he found individuals who were interested in his case, but were not officially able to help him. He found that unless he could assign a monetary loss as a result of the hacker's activities, the FBI did not have the resources to track the hacker. He was also told to contact the CIA and NSA, who would consult with him, but could not offer any resources either.

Stoll decided to treat the situation like a scientific research project, to apply the scientific method to determine the hacker's location. He looked at his notes from when the phone technicians had completed the trace. He had written down the numbers they had said while tracing the call, and while they didn't mean much to him at the time, he realized that they might be a phone number. He eliminated the numbers that were not in service and called the one that remained and got a response from a  modem. While this was not definite proof that the hacker was calling from that number, it was very good evidence.

The number belonged to a defense contractor known as Mitre. Stoll probed there computers and determined that due to a security hole in their system, hackers could call into Mitre and then dial out at will effectively using Mitre as a proxy. This had the effect of hiding the real location of the hacker and because the calls were originating from Mitre, they were charged for all phone services.

Throughout the process they tried to remain undetectable to the hacker. They did not install any software to monitor his activities because he might see that process running on the computer and realize that he was being observed. They monitored him only passively by printing out his activities directly from the line he was calling through. They also decided to communicate only through direct meetings or telephone, because the hacker could scan their email to see if they were talking about him. On several occasions however, Stoll felt the need to intervene to stop the hacker from downloading documents from military computers. He accomplished this by dangling his keys over the hacker's telephone line to introduce noise. This had the effect of blocking the hacker without raising too much suspicion.  

They also tried to measure the delay in the hacker's signal to estimate approximately how far away he might be. They determined that the signal took 3 seconds to travel from the hacker to them and then back. They knew that as the signal was passed from one node to another, each node would add some delay, but they did not know how much, so they sent signals farther and farther to find out round trip times to known locations. They connected from Berkeley to Los Angeles and found the delay was a quarter second. They stopped at Boston, Massachusetts where the delay was around one and a half seconds, still far short of the three second delay that they experienced when connected to the hacker, so they knew that he must be further away than Boston. Although this did not help to pinpoint him, it gave them more evidence that the hacker was not in Virginia.

One practice the hacker had was to download the password files of systems that he had gained access to. On UNIX systems of the time the password files were publicly readable, but because they were encrypted, Stoll did not see what use they could be to the hacker. However, Stoll later saw the hacker log in through accounts that he could only have compromised by decrypting passwords from this file. He realized that the hacker was using a dictionary attack, running the password file through a collection of words that were likely to appear in the English language. By examining the time between when the hacker had downloaded the password file and when he successfully logged on from the accounts in that file, Stoll determined that he must be using a personal computer. Curious as to how his own system might fare against such an attack, Stoll audited his password file and found that as many as 20% of the passwords contained there would likely be found in a hacker's dictionary.

The revelation that the hacker had infiltrated Mitre had woken up some of the authorities. Mitre locked down their phone system, but the hacker soon found other routes to the LBL. Stoll was able to get the proper warrants for a trace. They found out that the hacker was coming from West Germany, which made sense given the delay they had measured. It also explained some of the login names that he had created when accessing computers - Jaeger is the German word for hunter. They now had to get the German authorities permission to complete the trace. This took a few weeks but again they were able to obtain the proper warrants, but they faced another technical challenge.

The telephone exchange in Hanover, Germany that the hacker was calling through was used antiquated mechanical rotary switches to exchange calls. This meant that a technician had to physically test wires in order to complete a trace. The process could take up to an hour, but the hacker never remained on the LBL computers for more than a few minutes, because he knew that they did not contain anything that he was interested in. Stoll came up with an innovative solution for this. He set up what we today would call a honey pot. He created documents that appeared sensitive by taking Department of Energy memos that he received and modifying them appear to contain information about SDI technology. The hacker quickly found these documents and spent hours downloading and reading them, which gave the technicians in Germany enough time to complete the trace.

It would be a few months before the German authorities released the results of the trace, they still had to obtain the proper warrants and build their case against the hacker. They requested Stoll's notes and logs as evidence, and in the meantime the hacker continued to periodically connect to the LBL, giving them several opportunities to confirm the trace. When they finally made an arrest, they would not share the name of the hacker with Stoll, but the details leaked out quickly. The hacker's name was Markus Hess, a 28-year-old UNIX programmer who was passing the data he collected to the KGB. Although he did not find any classified information, Hess had revealed the surprising lack of security on military computers.

Over the course of ten months, Stoll watched Hess attempt to break into 450 computers and succeed in getting into about 30. Hess showed that he was experienced in several operating systems - VMS and UNIX, though Stoll could often tell by his command style that he was not familiar with the Berkeley version of UNIX, which helped them to realize earlier that the hacker was not a student at Berkeley. Markus was very cautious;  when he first logged onto a system, he would almost always see which other users were on and if any administrators were also logged in, he would immediately sign off. He tried not to damage any systems that he accessed, though he would sometimes kill accounting processes and other processes that might tip people off to his presence. In spite of his cautions, he did sometimes interfere with ongoing experiments, in one case a medical scan in progress.

The question of whether or not to remain open to the hacker was a common theme in the book. Of course leaving their computer un-patched was  a security threat since it was a well known bug in emacs that allowed Hess to gain root access to the LBL computer. As long as that hole was open, it could easily be used by other hackers. But it was also a problem of resources. Officially, Stoll's job was to maintain the computer lab, not to chase a hacker, which took time away from his real duties. The LBL had no budget for security, and no while many of the government agencies were interested in the results of the investigation, none of them were able to provide any funding. Stoll was often on the edge of calling the project off, but often he would find a clue that would keep him going and keep the authorities interested for a few more weeks. 

Another question that often arose was what to do about remote systems that the hacker accessed through the LBL. Stoll could see which systems he gained access to and what vulnerabilities he leveraged to gain administrative privileges. When he discovered the methods the hacker used, he would contact the system administrators at the remote location and inform them what he had learned. They most commonly shut the hacker down immediately by closing the holes he had entered through, though some chose to remain open to assist Stoll with the trace. Stoll noted that to the hacker it must appear as though the majority of the remote systems had noticed his presence except for Berkeley while the opposite was true - most of the systems would have had no idea he was there if the LBL had not detected his presence.

Though the events of the Cuckoo's Egg took place in 1986 and 1987, and the computers of the era were much more primitive than today, it is interesting to note that the security flaws that allowed Markus Hess to enter into so many networks were not system specific; they were mostly human errors that are still common today. The hacker was able to enter into systems using default logins and passwords that were never changed by system administrators. Once he was in, he was able to read the password file and perform a dictionary attack to quickly find encrypted passwords, so that he could log in later as different users. Because of this he established guidelines similar to what are taught today. System passwords should be sequences that do not appear in a dictionary, and should be updated periodically without letting users reuse previous passwords.  

Though many of the problems Stoll wrote about in 1986 still exist because of human nature, the attitudes of system administrators then and now are worlds apart. In the book Stoll often wrote about a sense of community and communication over computer networks in a way that seems naive today. In 1986 computers were still mostly the realm of university researchers, government agencies, and big businesses; they were big machines in special rooms that were operated by men in lab coats, and the concept of a microcomputer or personal computer was still brand new. Because there were only a relatively few universities and government computers that made up the precursor to the public internet, trust was assumed and taken for granted, so it was easy for a hacker to break in. Also, there were no laws that specifically dealt with computer crime in the US or Germany, although prosecutors could often find ways to apply existing laws. More importantly, law enforcement at the time was not experienced with tracing and prosecuting computer crime. Since then we have inevitably shifted into a less trusting environment where we assume that security should be turned on by default. We have advanced greatly in computer security in the last 25 years, often learning lessons the hard way, but then as now, the human element remains present in our networks and will always cause some vulnerability.