Chapter 2 – History of Computer Networks Up to TCP/IPv4
A long time ago (in a Galaxy not too far away), people started connecting computers together. A few brave souls tried to do this with dial-up 1200 baud modems over phone lines. Pioneers brought up Bulletin Board Systems (message boards that one person at a time could dial into and exchange short messages, and later small files, with each other). I brought up the 8th BBS in the world, in Atlanta, about 1977, using code from the original CBBS in Chicago (created by Ward Christensen and Randy Suess) and a modem donated by my friend Dennis Hayes (of Hayes Microcomputer Products). Later there were thousands of online Bulletin Board Systems, all over the world. Soon there followed commercial “information utilities” like CompuServe and The Source, which were like giant BBS’es with many more features. Tens of thousands of users could connect to these simultaneously. It was like the first crude approximation to the Internet of today, based on circuit switched connections over telephone lines. Everything was text oriented (non-graphical) and very slow. 1200 bits/sec was typical at first, although later modems with speeds of 2400 bits/sec, 9600 bits/sec, 14.4KB/sec, 28.8KB/sec and finally 56KB/sec were developed and came into widespread use. Later these modems were primarily used to dial into an ISP to connect to the Internet, and some people are still using them this way.
2.1 – Real Computer Networking
While home computer users were playing around with modems and bulletin board systems, the big computer companies were working on ways to connect “real” computers at higher speeds, and with much more complex software.
2.1.1 – Ethernet and Token Ring
Much of this was based on Ethernet, which was created by a team at Xerox PARC led by Robert Metcalf between 1973 and 1975. The first specification (1976) ran at 3 Mbit/sec. Metcalf left PARC in 1979 to create 3com and create commercial products based on Ethernet. Working together with DEC, Intel and Xerox (hence the “DIX” standard), 3com released the first commercial products running at 10 MBit/sec. Ethernet was standardized in 1980 by the IEEE (Institute for Electrical and Electronic Engineers) as 802.3. Early versions ran on 10base2 (a small diameter coax cable) or 10base5 (a larger diameter coax cable). These used a “multidrop” architecture, which was subject to many reliability problems. With the introduction of the simpler to deploy and manage “unshielded twisted pair” (actually 4 pair, or 8 wire) cables (known as 10baseT, mid 1980s), and star architectures using “hubs” and later “switches”, local area networks really took off. Today, virtually all Ethernet networks use twisted pair copper wire (up to gigabit speed) or fiber optic cable (for higher speed and longer runs). I helped deploy a 10base2 coax Ethernet network in Hong Kong in 1993. Trust me, twisted pair cabling is a lot easier.
IBM for many years pushed a competing physical layer network standard called “Token Ring” (later standardized as IEEE 802.5). Token Ring was available in 4 Mbit/sec and 16 Mbit/sec versions. Later, a 100 Mbit/sec version was created, but by then Ethernet dominated the market and Token Ring quietly disappeared. FDDI (Fiber Distributed Data Interface) still in use today is based on the token ring concept.
2.1.2 – Network Software
Network software quickly evolved once Ethernet and Token Ring hardware became available. One of the main goals was to “hide” the differences between various hardware level technologies (Ethernet, Token Ring, Wireless, etc) from the higher level software. This led to the multiple layers of the network stack. The bottom layer is very hardware specific, and the upper layers introduce more and more hardware independence, so that applications can be written once, and run over any hardware transport.
Digital Equipment was one of the first to create networking software with DECNET (1975). IBM had System Network Architecture (SNA, 1974). Xerox created the PARC Universal Packet protocol (PUP, late 70’s) which eventually evolved into Xerox Network Services (XNS, early 80’s) at PARC (Palo Alto Research Center). XNS was the basis for the later Banyan Vines Network OS, based on “Vines IP” (similar to but incompatible with IPv4 from TCP/IP). Banyan Vines included the first network directory service, called “StreetTalk”. XNS also was the basis for Novell Netware (IPX/SPX, 1983), which eventually added its own Netware Directory Services (NDS, 1993). NetWare did not fully support TCP/IP until 1998, which allowed Microsoft (who supported TCP/IP first) to take over leadership in personal computing networks.
Microsoft worked with 3com to create their own network OS, called LAN Manager. It used SMB (Server Message Block) protocol on top of either NBF (NetBIOS Frames Protocol) or modified XNS. In 1990, Microsoft added support for TCP/IP as an alternate protocol (LAN Manager 2.0). With the release of Windows NT Advanced Server in 1993, Microsoft finally phased out LAN Manager. By Windows NT v3.51 (May 1995) Microsoft encouraged users to deploy only TCP/IP (four years ahead of Novell’s support for TCP/IP). This lead time allowed Microsoft to take over leadership in personal computer networks from Novell. Microsoft introduced their version of network directory services in Windows Server 2000, now known as Active Directory. The SMB protocol still survives as Microsoft’s “File and Printer Sharing” protocol (now layered on TCP/IP, instead of NetBIOS or XNS). An Open Source implementation of this is available as SAMBA.
2.2 – The Beginnings of the Internet (ARPANET)
While all this commercial activity was going on, the U.S. Military (at their Advanced Research Projects Agency, or ARPA), with the help of Bolt, Beranek and Newman (BBN) and Mitre, were designing a new, decentralized communication system based on packet switching. Existing communication systems (telephone, radio, etc) were centralized, and hence subject to being completely disabled due to the failure or loss of a few central nodes. Packet switched networks were highly decentralized, and had a fascinating new property, which is that you could lose large parts of a network, and the remaining parts would still work (assuming at least some links connected the working parts).
The first network protocol developed as part of ARPANET was called the 1822 protocol (named after BBN report 1822), and was implemented by a Network Control Program, so the protocol was often referred to as NCP. The first e-mail was sent over NCP in 1971, and the File Transfer Protocol followed in 1973. On Jan 1, 1983 (“flag day”), NCP was turned off officially, leaving only TCP/IPv4 on the Internet. You might think of the NCP era as phase 1 of the First Internet, with the IPv4 era being phase 2 of the First Internet. Otherwise the new Internet based on TCP/IPv6 will be THIRD Internet. Fortunately, there is no need for a flag day to go from TCP/IPv4 to TCP/IPv6, as they can co-exist (and probably will for perhaps 5 to 10 years).
In May 1974, Vint Cerf and Bob Kahn released the paper “A Protocol for Packet Network Interconnection”. This described a monolithic protocol called TCP that combined the features of both modern TCP and IPv4. Later John Postel was instrumental in splitting apart TCP and IP as we know them today. Vint Cerf is today considered the “father of TCP/IP”, and is now the IPv6 Evangelist at Google. He understands very well the problems with the current implementation of TCP/IPv4 (and why these things were done). He advocates for users to migrate to TCP/IPv6, which restores his original concept of a flat address space (no NAT), where any node can connect directly to any other node.
If you’d like to read about the creation of the First Internet, I recommend the book “Where Wizards Stay Up Late: The Origins of the Internet”, by Katie Hafner and Matthew Lyon. It is of considerable interest to those of us creating the Second Internet, as we facing some of the same problems they did. Only this time around, we’ve got over a billion legacy users (and staggering investments in hardware and software) to worry about. On the other hand, we’ve got three decades of operational experience with TCP/IPv4 to draw upon.
Higher level software protocols were built on top of the TCP and IP layers, called “application protocols”, such as SMTP (for e-mail), FTP (for file transfer) and Telnet (for terminal emulation) and more recently HTTP (used in the Web), and SIP & RTP (used in VoIP). The resulting suite of protocols became known for its two most important protocols, TCP and IP, or TCP/IP.
2.2.1 – UNIX
About this time (1973), Bell Labs (a research group within AT&T) created an interesting new operating system (called PWB-UNIX) and a new language (in which UNIX was written) called “C”. Because of a 1958 consent decree, AT&T as a regulated monopoly was not allowed to market or sell UNIX commercially. They licensed it (complete with source code) to a number of universities. One of these was the University of California at Berkeley (also famous for being the center of communist student activities at the time). The team at UCB extended UNIX in several very important ways such as adding Virtual Memory. They also integrated the new network protocol from ARPA as the first commercial implementation of TCP/IP. The “Berkeley System Distribution” of UNIX became a main branch. Over time, they rewrote most of it and wanted to release it for free. AT&T sued them in court, and it seems most of the examples of “stolen code” AT&T cited had actually been written at UCB. The judge ruled that if UCB rewrote the remaining 10% or so (so there was zero original AT&T code), they could release that. That rewrite became 386BSD, the starting point for FreeBSD (the first Open Source operating system). Interestingly enough, FreeBSD was chosen by Japan’s KAME project to deploy the first version (the “reference” implementation) of an IPv6 stack, in an eerie echo of BSD UNIX’s choice for the first commercial TCP/IPv4 implementation.
UNIX and TCP/IP became very popular on college campuses, and with high end workstation vendors, such as Sun, Silicon Graphics and Intergraph. Personal computers were not powerful enough to run UNIX until the Intel 386, at which point UCB ported the BSD version to the 386. However, as documented above, most personal computer networking was already moving to TCP/IPv4.
2.2.2 – Open System Interconnect (OSI)
While all of this was going on, ISO (the International Organization for Standardization) in Europe was creating a very thoroughly engineered suite of network protocols called Open System Interconnect (OSI), or more formally, X.200 (July 1994). This is where the famous “seven layer” network model comes from (TCP/IP is really a “four layer” model, which has caused no end of confusion among young network engineers). At one point the U.S. government decided to officially adopt OSI for its networking (this was called GOSIP, or Government Open Systems Interconnection Profile, defined in FIPS 146-1, 1990). Unfortunately, OSI was really more of an academic specification, not a real working network system, like TCP/IP was. After many years, GOSIP was finally abandoned and TCP/IPv4 was deployed, but GOSIP’s legacy has hindered the adoption of IPv6 in the U.S. (“here we go again – GOSIP Part 2!”). X.400 e-Mail and X.500 directory systems were built on top of OSI, and will not run on TCP/IP without substantial compatibility layers. One small part of X.500 (called X.509) was the source of digital certificates and Public Key Infrastructure, still used today. LDAP was an attempt to create an X.500-like directory system for TCP/IP based networks. That’s about all that is left of the mighty OSI effort today, outside of Computer Science textbooks and Cisco Press books.
2.2.3 – E-Mail Standardization
By this time, essentially all computer vendors had standardized on TCP/IP, but there were still a lot of competing standards for e-Mail, including Microsoft’s MS-Mail, Lotus’s cc:Mail, and MCI Mail. The Internet folks used a much simpler e-mail standard called SMTP (Simple Mail Transfer Protocol). It first became the connecting backbone between various e-Mail products (everyone had their e-Mail to SMTP gateways, so users could exchange messages across organizations). Soon, everyone started using SMTP (together with POP3 and later IMAP) all the way to the end-user. Today virtually all e-Mail worldwide is based on SMTP and TCP/IP.
2.2.4 – Evolution of the World Wide Web
Several other Internet applications evolved, including WAIS (Wide Area Information Server, for storing and retrieving documents) and Archie (the very first search engine). In turn, these efforts were merged with the idea of HyperText (documents with multi-level links) and evolved into HTML (HyperText Markup Language) and HTTP (HyperText Transfer Protocol). The World Wide Web was off and running. The first web browser and web server were created at the National Center for Supercomputing Applications (NCSA) at the University of Illinois, Urbana-Champaign campus. The people that created those software projects (primarily Marc Andreessen and Eric Bina) were soon hired by Jim Clark, one of the founders of Silicon Graphics, to start NetScape, one of the most successful companies in the First Internet. They created a new and more powerful web server (NetScape Web Server) and web browser (NetScape Navigator). Interestingly enough, the original browser created by Andreessen at NCSA later became the starting point for Microsoft’s Internet Explorer web browser.
2.3 – And That Brings Us Up to Today
That pretty much brings us up to the present day where the entire world has standardized on TCP/IPv4 protocols for both LANs (Local Area Networks) and WANs (Wide Area Networks). MPLS is not a competitor to TCP/IP, it is one more alternative at the Link Layer, peer to Ethernet and WiFi. More and more companies and organizations built TCP/IP networks and connected them together to create the Internet. Major telcos provided “backbone” WAN connections and dial-up access service (soon known as ISPs or Internet Service Providers). As the number of users (and the amount of traffic) on the Internet grew enormously, Internet Exchange Points were created around the world. These are places where ISPs connect to each other so that traffic from a user of any provider can reach users of any other provider, worldwide.