Project Xanadu: Loss and Recovery

 The text below is from "Project Xanadu: Loss and Recovery," a side-bar I wrote for the AIMS (An Inter-Institutional Model for Stewardship) Project Final Report on the building of framework for archival processing of born-digital materials. The report was issued in Jan. 2012.  You can read about the project and get the final report here.
“What we're actually building at this point is only a part of Ted's original conception, though it's designed to be the first stepping stone to the whole thing.”
– Chris Hibbert, post to comp.multimedia newsgroup, 30 March 1992 (from file on XOR hard drive).
Ted Nelson’s Project Xanadu provided the original vision of hypertext as a system for document management, publication, linking and citation. Begun in 1960, the project to build Xanadu continued well into the 1990s. From 1989 to 1992, Autodesk funded Nelson’s Xanadu Operating Company (XOC) to complete software development. However, when a new group of programmers primarily from Xerox’s Palo Alto Research Center (PARC) joined the group in 1991, they abandoned the earlier version of Xanadu written largely by Roger Gregory and began a new version rewritten from scratch in PARC’s new programming language, Smalltalk. This forking of the project eventually led to the collapse of the Autodesk-funded effort. Keith Henson, an XOC investor, encouraged the Palo Alto-startup, Memex to pick up the project In 1994. Memex licensed Xanadu from XOC and brought the Xanada project to its office space on California Avenue. Before long, however, the arrangement collapsed. The team disintegrated, with Nelson and Gregory regaining control of Xanada, which would finally be released as the open-source Udanax system in 1999.
Shortly thereafter, in 2001, the Stanford Libraries acquired the papers of Keith Henson and his wife Arel Lucas. The papers provided some documents from the history of XOC and included six hard drives, identified only as being from XOC in the mid-1990s. These mysterious hard drives were included in the AIMS project, because their source suggested that they might be significant for documenting the history of Xanadu. Moreover, the task of recovering data could well provide an interesting challenge. Indeed, the Stanford team was able to successfully image only two of the drives. Mechanical or formatting issues with the other four drives prevented access to the files on them. In order to learn if it would be possible to recover data from these drives, one was sent to Recovery Services, Inc. (RSI), which has a proven reputation in the area of data recovery services. RSI determined that we were dealing with “severe physical failures, some of them associated with read and write head errors.” They concluded, however, that “it may still be possible” to recover at least some of the data. As stewards of the Henson Papers, we decided to cover the not insignificant cost (nearly $10,000) of the recovery option with RSI. We note that the expense of commercial data recovery may provide an obstacle for frequent use of this method.
The RSI effort was generally successful. It yielded three disk images, as well as capturing a significant number of files from the three drives from which RSI was unable to capture a complete disk image. Information gleaned from the recovered data reveals much about the provenance and significance of these hard drives. For example, several files document use of XOC’s backup system and from file creation dates we learned that nearly all of the files were created between 1989 and 1993. Some files include header tags such as “historical” or “xanadu archive,” so that we know that they were identified as being of historical interest. Specifically, many files contain source code and libraries in Smalltalk, with author names that correspond to the names of the XOC programming team during the Autodesk period. These files contain source code for the Xanadu version known today as Udanax Gold (formerly Xanadu 92.1), the version that was shut down when the Memex-based team disbanded. In addition to source code, the drives contain documentation about XOC, such as versions of a business plan that appears to have been written in 1982, text files from the 1983 “ninth printing” of Nelson’s self-published (and increasingly rare) Computer Lib, draft chapters of Computer Lib and Dream Machines from Sept. 1984, and later documents such as descriptions of Xanadu and the work at XOC during the early 1990s. These documents will fill gaps in the historical record of XOC and the development of the Xanadu system and thus contribute to history of hypertext and related technology such as the World Wide Web.