assumed to have a print line 72 characters wide, and a page of 66 lines. The new line convention will be carriage return (X′0D′) followed by line feed (X′0A′) ... The standard printer will accept form feed (X′0C′) as meaning move paper to the top of a new page. 6
• Move reflected Vittal's attention to user behavior. He noticed that one of the most common activities was to save a message in a file and then delete the message from the inbound mailbox. Vittal created the combined Save/Delete command, Move.
• Answer (now usually called "reply") is widely held to be Vittal's most insightful and important invention. Answer examined a received message to determine to whom a reply should be sent, then placed these addresses, along with a copy of the original Subject field, in a responding message. Among the challenges Vittal had to solve were the varying email-addressing standards and what options to give a user (reply to everyone? reply only to the sender of the note?). It took three implementations to get right. 36
• The wonder of Answer is that it suddenly made replying to email easy. Rather than manually copying the addresses, the user could just type Answer and Reply. Users at the time remember the creation of Answer as transforming—converting email from a system of receiving memos into a system for conversation. (There are anecdotal reports that email traffic grew sharply shortly after Answer appeared. 37)
• Forward provided the mechanism to send an email message to a person who was not already a recipient. How much of an innovation Forward was is unclear. Barry Wessler had to struggle with messages embedded in messages in NRD. But the formalization of the idea was new.
[A] field like "To: PDL, Cerf@ISIA" was ambiguous was "PDL" really "PDL@ISIA" (picking up the host from the end of the line)? Or was it "PDL@MIT-DMS" (picking up the host from the "From: JFH@MIT-DMS" elsewhere in the header)?Various mail programs adopted different such "abbreviations" which drove me crazy. ... To handle all of this protocol chaos, I wrote (and rewrote, and tweaked) a sizable (for a LISPish world) chunk of code to try to deduce the precise meaning of each message header contents and semantics based on where the message came from. Different mail programs had different ideas about the interpretation of fields in the headers.That code first tried to figure out where an incoming message had come from. This was not so obvious as it might seem because of redistribution and forwarding of messages, and differences in behavior of various versions of the other guy's software. So it wasn't enough to just look to see if you were talking to MIT-MULTICS. I remember having conditional clauses that in essence said "If I see a pattern like such-and-such in the headers, this is probably a message from version xx.yy of Ken Pogran's Multics mailer." With enough such tests, it formed an opinion about which mail daemon it was talking with, and which mail UI program had created a message.Having hopefully figured out the other guy's genealogy (and therefore protocol dialect), the code then acted based on a painfully collected set of observations about how that system behaved. 52
to take RFC680 plus what we felt were things which people were already doing that were useful to most, take out some things that weren't terribly useful and probably shouldn't have been in 680 in the first place, and come up with a new specification. There were several things that some systems were already doing: comments (e.g. the day of week in parentheses), association of people names with user names (like at places like Stanford, CMU and MIT, also using parenthesization), random date format preferences (Multics vs Tenex, etc.), and so on. Elements of 680 which were not perceived as necessary were mostly the military-like field names such as precedence, as well as syntactic inconsistencies (bugs), and syntactic limitations. These could all be accomplished by using the notion of user-defined fields. 57
UUCP network. The UUCP network (named for the Unix-to-Unix CoPy program over which it was built) began inside AT&T in 1978. 66 It used dial-up telephone links to exchange files and within a few months was moving email. AT&T soon distributed the software and the UUCP network, made up of cooperating sites, was off and running. Over the next decade it grew at a prodigious rate, such that by 1990, its population was estimated at a million users—comparable to the Internet's population. 67
The UUCP network was a multihop network. To reach machine V, an email from machine M might have to pass through intermediate systems Q and T. The motivation for this approach was to minimize phone bills. In the 1970s and early 1980s, long distance calls were expensive, and the rates differed by hour (with evening and night rates being sharply lower). Modems were slow (a couple hundred bytes per second was considered good) and files were (relatively speaking) large. So the typical operating mode at any UUCP site was to save up all email until 5 p.m., then call a nearby UUCP site to forward email along and receive inbound email. Indeed, over the course of the night, several phone calls would be made to push outbound mail and receive inbound mail. Depending on the calling schedules and the connectivity of the machines, email could travel a few or several hops before the nightly calling frenzy ended.
Initially, the person composing the email had to spell out the entire path a piece of email needed to take through the network. In the UUCP network, the hops were separated by exclamation points ("!," pronounced as "bang"). So, someone mailing the author via UUCP from UC Berkeley in the 1980s would send it to ucbvax!ihnp4!harvard!bbn!craig (in which each text string followed by a "!" is known as a hop; this example has four hops).
In 1982, Steve Bellovin wrote pathalias, a tool designed to compute paths from a network map. He refined it with Peter Honeyman. 68 Pathalias was distributed widely. Now, by keeping a map of regional connectivity, it became possible to email via landmark sites and have them fill in the missing hops. So, for instance, the author's address could be reduced to ihnp4!bbn!craig and the harvard hop would be dynamically inserted.
In 1984, Mark Horton began an effort to create a complete UUCP network map, which reached fruition about 1986. After that, UUCP users could simply type sitename!user, and pathalias would compute a path to sitename for them. An even fancier trick was to add a network domain to the sitename, such as bbn.arpa!craig, and pathalias would compute a path to an email gateway between the UUCP network and the Internet.
CSnet. By the late 1970s, the computer science research community realized that the Arpanet was changing how people did research. Researchers who had access to a network got information more quickly, and could collaborate and share work more easily. Thus was identified the first "digital divide"—between computer science departments that had access to Arpanet and those that did not. 69
The goal of the Computer Science Network (CSnet) was to bridge that gap. Created in 1981 by the National Science Foundation in cooperation with ARPA, CSnet linked computer science departments and industrial research laboratories to the Arpanet (and then the Internet). 70
CSnet was designed to become self-supporting. The ARPA and NSF funding was only to provide start-up capital and an initial operations budget. For the first two years, CSnet operations were distributed between the University of Wisconsin and the University of Delaware, with help from RAND (which ran a gateway on the West Coast). Beginning in 1983, the network was operated by BBN, where a team of roughly 10 people provided technical support (including writing or maintaining much of the email software used by CSnet members), user services, and did marketing and sales. By 1988, CSnet was self-supporting and had approximately 180 members, most of them computer science departments in North America.
Technologically, CSnet did everything possible to make its members feel part of the Internet community. Initially, connectivity was almost entirely email only, using dial-up phone service. Over time, direct access via IP was also supported over a variety of media, including IP over X.25 71 and the first dial-up IP network. 72
After 1983, email in CSnet all went through a single email gateway, Csnet-Relay, which sat on both CSnet and the Internet. Email was routed by addressing it to the relay, with the user address being the target address on the other network. The syntax used a percent sign (%) to divide the next hop user name from relay address. So, to get from the Internet to a CSnet host, one emailed to firstname.lastname@example.org. From CSnet, one emailed email@example.com. Email was formatted according to RFC-733 and 822 standards.
Bitnet. Bitnet was established in the same year as CSnet, but with a different driving force. Bitnet ("Because It's There" or, later, "Because It's Time") was created by university computer centers (now information technology offices) to interconnect their computing facilities with email and file transfer. Because the centers typically used IBM mainframes running the VM operating system, Bitnet was constructed from low-speed leased lines running IBM networking software, on which email was overlaid.
Like CSnet, Bitnet used Internet email standards (with the %-hack in the email address for gatewaying). Unlike CSnet, Bitnet did not have a central management or support center. Instead, most functions were volunteer activities, with coordination provided by Educom (Interuniversity Communications Council). In mid-1988, Bitnet had nearly 400 member sites.
The boards of Bitnet and CSnet overlapped and the two networks eventually merged, so one may wonder why they were distinct in the first place. The distinction lies in the relationship, often contentious, between computer science departments and computing centers in the 1970s and 1980s. Computer science departments typically maintained their own computing facilities, to enable research by computer science faculty. Computing centers were university-wide resources that sought to provide stable computing environments for researchers in other disciplines. The stereotype was that computer science departments ran cutting-edge operating systems on minicomputers and workstations while computing centers ran established commercial operating systems on mainframes. More important, from an institutional perspective, the computer science department typically provided a haven for those on campus who were (for whatever reason) disgruntled with the computing center. Neither party particularly wanted to rely on the other for network access, with the result that there were two networks: one for each community.
Email addressing across networks. The four networks (including the Internet) periodically viewed themselves as competitors. Yet the four networks were also committed to making email work among them. A number of sites brought up gateways between the networks. Even more sites made a point of residing on more than one network, to ensure ease of mailing for their users.
It is widely agreed that, by the early 1980s, email addresses were a disaster both for users trying to email across networks, and network administrators trying to keep the email flowing.
The disaster had two dimensions. First, one had to know which network a user was on. For instance, if someone told you he was bob@princeton, one had to immediately ask "which network" because princeton.bitnet and princeton.csnet were different machines and were not interconnected. If a user forgot, or her email software removed the network appellation (e.g., . csnet) the email would be delivered to the bob@princeton in whichever network the sender was in.
The second problem was that, even if one knew which network an email address was in, getting it there was not easy. To take a relatively common example, consider the following four addresses:
These represent the four likely addresses for reaching bob at Princeton's CSnet host, from the UUCP network, Bitnet, the Internet, and CSnet respectively. If the examples are not painful enough, consider the first address and how it would be handled in transit.
It starts in the UUCP network and is passed to ihnp4 (a key UUCP relay at Bell Labs in Naperville, Illinois). Ihnp4 must puzzle out firstname.lastname@example.org and decide if the email address is to the left of the @ sign (Internet style) or to the right of the bang (UUCP style). As ihnp4 is a UUCP-only system, it knows to use UUCP addressing and passes the message to ucbvax at the University of California at Berkeley. Ucbvax is a gateway on both the Internet and UUCP networks so it must puzzle out email@example.com. Thankfully, ucbvax was not on CSnet and clearly not the same system as csnet-relay.arpa, so bob%princeton.csnet is no good. Thus the message must be sent to the CSnet relay (and, because Arpanet did not strip mailing information, it remains firstname.lastname@example.org). CSnet's relay in turn extracts the address to the left of the @ sign, to get bob%princeton.csnet and delivers the email to Princeton.
Observe that there's ample chance for confusion. Another nasty problem was that each mailer had to make sure that the From address in the email was updated (and sometimes the To and Cc addresses as well) so that the recipient of the email could successfully reply to it. Yet another challenge was that, for a period, the United Kingdom decided to reverse the order of labels in a domain name (so Kirstein@uk.ac.ucl.cs) with the result that some mailers had to parse names backward and forward ("bothways" mode) to see if they made sense.
It is no surprise that the people who made major contributions to email MTAs at this time were people closely affiliated with email gateways.
delivermail. Allman's delivermail, the simplest of these MTAs, was written for Berkeley's BSD Unix operating system in 1979 and was a basic program 73 not greatly more complex in its workings than Bob Clements' 1973-vintage Sndmsg. When invoked by a user agent (or the inbound FTP server), delivermail expected to be given a message, which it would either deliver or return an error message. The big difference was that delivermail implemented a layer of indirection. Rather than delivering the message to a mailbox or a remote system, delivermail looked at the destination address and then picked a program to deliver the message to. So, for instance, to deliver Arpanet mail via FTP, delivermail called an auxiliary program called arpa and passed the mail to the arpa program and waited for a (real-time) response regarding delivery. If, by some mischance, the message had to be queued, arpa (not delivermail) would queue it.
To parse the address, delivermail used the simple expedient of assuming that an at-sign meant Arpanet mail, an exclamation point in the address meant UUCP, and a colon meant the local Berknet protocols. For each address type, delivermail could be configured either to call a program to deliver the mail, or call a program to relay the mail to the appropriate gateway (one email gateway per type).
The delivermail MTA had a powerful aliases features, in which a destination address could be expanded to a list of email addresses. It also had a first class logging system (a way to record what delivermail did) called syslog. Email systems were developing increasingly sophisticated logging mechanisms; syslog was so good that it eventually became a standard part of BSD Unix and is now used by a wide range of applications.
One surprising feature of delivermail was that part of its configuration was compiled into the program. That is, for each machine, one compiled a custom version of delivermail. So, for instance, if the machine was connected to Arpanet, one compiled delivermail with the –DHAS_ARPA flag to the C compiler.
mmdf. About the same time that Allman was creating delivermail, Dave Crocker was writing the first version of mmdf (the Multi-channel Memo Distribution Facility). 74 Rather than seek to process each message immediately, as delivermail did, Crocker sought to decompose the process into multiple stages.
When a message arrived (via the network or from a user agent), the message was given to a program called submit, which checked that the message format was correct (here the common use of 733 format was a big win) and then looked at the address to decide what network the message was to go out on. The message was assigned to a "channel." Each channel had its own queue: a directory where messages and their "envelopes" (control information) were stored. Simply, submit placed the message in the right queue.
Another program, called deliver, was regularly scanning the queues for messages. When a new message appeared, deliver called on a channel-specific program (e.g., mmdf's equivalent of delivermail's arpa program for Arpanet email) to deliver the message. If message delivery failed, submit was called to send the message back to its sender. If there was a transient error (e.g., the remote host was down), the message was left in the queue and deliver would try it again later.
The mmdf MTA also supported aliases and had a fine logging system.
An important contribution of mmdf was achieving an effective split of the message delivery process. Diagnosing email problems (whether configuration problems or problems with particular messages) was cleanly compartmentalized. Similarly, submit prevented junk from entering the system; deliver handled problems in delivery. An operator knew where the problem was by seeing which program was complaining in the logs.
Another contribution was restriction of privileges. One of the key problems in any mail system is that whatever program delivers mail to the user's mailbox needs special privileges. In mmdf, that was one small program, the local channel delivery process. All the other processes could be run as a regular user (usually called "mmdf").
The channel model also proved flexible. A message could go through multiple channels before leaving a system. Soon, mmdf developed a "list" channel to handle mailing lists. A message was placed in the list channel to have its destination address expanded. It exited the list channel by being placed in one or more channels to be delivered to members of the mailing list. Later, when MX resource records were introduced (see the "Email routing with domain names" section), they introduced a new error: a domain name that (because of DNS problems) could not currently be looked up. In mmdf this was trivially handled by creating a new channel, where submit placed messages whose addresses could not be resolved at the moment.
A downside of mmdf was that rather than one configuration file, there were several, scattered in different places. While each configuration file was simple (a list of attribute:value pairs), the sheer number of them could prove frustrating.
sendmail. Based on experience with delivermail, Eric Allman decided to write a new MTA for release with the 4.2 version of BSD Unix. The new MTA was called sendmail.
Culturally, sendmail was similar to delivermail. But from a practical perspective, it was quite different. Major differences included the following: 75
• Configuration was determined by a file, called sendmail.cf, rather than being compiled in.
• The address parsing rules and message delivery rules were defined by a grammar in the configuration file.
• sendmail now maintained its own message queue.
• Certain delivery programs (most notably email delivery via SMTP) were compiled into sendmail instead of client programs (e.g., arpa).
But this list understates the transformation from delivermail to sendmail: sendmail was almost an order of magnitude more complex (measured in lines of code) and tremendously more flexible.
The changes had an interesting mix of consequences. Probably the most important consequence was flexibility. Placing address parsing and configuration rules in a grammar made it possible to dynamically configure sendmail for arbitrarily complex email environments.
Another consequence was a reinforcement of delivermail's approach of putting all the email expertise into one program. SMTP was now embedded in sendmail. So too was queue management. It made sendmail a complex program and hard to change. Allman later noted that sendmail should have been better decomposed into constituent functions, even if only internally. 76
An unexpected consequence was that crafting and debugging sendmail's single configuration file ( sendmail.cf) became a central preoccupation (some would say headache) for system administrators over the next several years. A properly working email system required the configuration file be right. And sendmail's grammar (with a fondness for single-letter tokens, which made mnemonic naming impossible) gave administrators many opportunities to make a mistake.
This plan covers only the transition from the current text computer mail in the Arpanet environment to text computer mail in an Internet environment. This plan does not address a second transition from text only mail to multimedia mail. 108
DARPA is beginning a new phase of research into automatic electronic message handling systems. Ultimately it is intended that electronic messages incorporate multiple media such as text, facsimile, compressed digitized voice, graphics and so on. 79
• How to represent binary data efficiently on the network. At the time, network capacity was extremely expensive, so there was a motivation to save every possible byte (and bit). The X.400 team sought a compact data representation.
• How did applications embed data in messages? The issue here is that data formats on different computers may be incompatible. They were certainly incompatible in the early 1980s, when a byte could be 5, 6, 7, 8, 9, or 10 bits long depending on the system. Data to be sent over the network needs a standard, "external" format that is computer independent. In the early 1980s this concept was alien to most programmers. 114 The X.400 approach was to encourage programmers to define how to move their data into and out of a generic format called an external data format.
• Data and not garbage. X.400 envisioned a world in which user agents developed new features and new applications would embed data in a document and so the receiving user agent might receive an email message that contained header fields it did not understand and application data from an application it had never seen before. How to ensure the user agent didn't simply report it had received garbage? The solution that X.400 chose to this problem was to make the data self-describing.
References and notes
Craig Partridge is chief scientist for networking at BBN Technologies, where he has worked on data networking problems since 1983. He is best known for his work on email routing, TCP round-trip time estimation, and high-performance router design. He received an MSc and a PhD, both in computer science, from Harvard University. Craig is the former editor in chief of IEEE Network Magazine and ACM Computer Communication Review and is an IEEE Fellow.