Complexity is the Enemy of Secure, Stable Systems

The New York Times article, Who Needs Hackers? shows just how many ways our systems can be compromised and most of them have nothing to do with hackers. The big problem, the article argues, is the complexity of systems and there are plenty of examples to support the claims.

International travelers flying into Los Angeles International Airport — more than 17,000 of them — were stuck on planes for hours one day in mid-August after computers for the United States Customs and Border Protection agency went down and stayed down for nine hours.
Hackers? Nope. Though it was the kind of chaos that malevolent computer intruders always seem to be creating in the movies, the problem was traced to a malfunctioning network card on a desktop computer. The flawed card slowed the network and set off a domino effect as failures rippled through the customs network at the airport, officials said.

Other examples include the Skype shutdown, the electric grid failure in the summer of 2003, the first launch of the space shuttle delayed due to a software bug, a four failure in the Arapent (the predecessor of the Internet) in 1980 because of an unforeseen interaction. These and other incidents have led to speculation about the most serious threats to stable systems operations.

Aviel D. Rubin, a professor of computer science at Johns Hopkins University, said that glitches could be an enormous problem in high-tech voting machines. “Maybe we have focused too much on hackers and not on the possibility of something going wrong,” he said. “Sometimes the worst problems happen by accident.
Dr. Rubin, who is director of the Center for Correct, Usable, Reliable, Auditable and Transparent Elections, a group financed by the National Science Foundation to study voting issues, noted that glitches had already shown up in many elections using the new generation of voting machines sold to states in the wake of the Florida election crisis in 2000, when the fate of the national election came down to issues like hanging chads on punch-card ballots.

Dr. Bellovin at Columbia said he also worried about what might happen with the massively complex antimissile systems that the government is developing. “It’s a system you can’t really test until the real thing happens,” he said.

There are also suggestions about how to address the problem:

The best answer, Dr. Neumann says, is to build computers that are secure and stable from the start. A system with fewer flaws also deters hackers, he said. “If you design the thing right in the first place, you can make it reliable, secure, fault tolerant and human safe,” he said. “The technology is there to do this right if anybody wanted to take the effort.”

Sounds good, but will it happen?

Dr. Neumann, who has been preaching network stability since the 1960s, said, “The message never got through.” Pressures to ship software and hardware quickly and to keep costs at a minimum, he said, have worked against more secure and robust systems.

“We throw this together, shrink wrap it and throw it out there,” he said. “There’s no incentive to do it right, and that’s pitiful.”

What do you think? Will the hardware and software industries make changes toward more robust systems designs? Is vPro and Centrino Pro a step in the right direction? I argued the Skype outage was in part due to insufficient testing of dynamic, distributed systems. Will that change?