A 1970s disk drive that wouldn't seek: getting our Xerox Alto running again

Our vintage Xerox Alto has been running reliably for months, but a couple weeks ago the disk drive malfunctioned and the heads stopped moving. With a drive that wouldn't seek, our Alto wouldn't work.3 After extensive debugging and studying the drive's complex head movement control system, we discovered that the problem had a trivial fix. This blog post discusses our adventures debugging the Alto's Diablo hard drive and how we got it to work again.

The Alto was a revolutionary computer designed at Xerox PARC in 1973 to investigate personal computing. It introduced the GUI, high-resolution bitmapped displays, Ethernet, the optical mouse and laser printers to the world. The Alto I've been restoring came from YCombinator; the restoration team includes Marc Verdiell (curiousmarc on YouTube), Carl Claunch and Luca Severini.

The Xerox Alto's disk drive is the unit below the keyboard. The cabinet under the drive holds the computer itself.

For storage, the Alto used a removable 14" hard disk cartridge that held just 2.5 megabytes. (A user might have multiple cartridges for different purposes, similar to floppies a decade later.) This model 2315 cartridge was invented by IBM in 1964 and became an industry standard, used in minicomputers by HP, DEC, Wang and many other companies. The photo below shows how a disk cartridge is inserted into the Diablo drive. (The drive has been pulled out from the cabinet and its cover removed to show its internal mechanisms.)

A disk cartridge is inserted into the Alto's drive. The drive has been pulled out and the cover removed, revealing its internals.

Each disk cartridge contains a single platter. The drive has two heads, one for each side of the platter, and the heads seek (move back and forth) in unison. Each side of the disk contains 203 tracks at a density of 100 tracks per inch (.254mm spacing), so the heads need to be positioned with very high accuracy. The heads float 70 microinches (1.8 µm) above the disk surface on a cushion of air, so any contamination on the disk surface can cause a head crash, causing the head to contact the surface and scrape up the oxide layer.

Opening a disk cartridge reveals the single hard disk platter. The disk isn't scratched; it's just the lighting.

Our disastrous adventure started when we tried to help out another Alto owner whose disk drive suffered a head crash.1 (Because of the problems this drive has caused, it will be called the cursed drive, although diabolical fits too.) Replacing the heads in the cursed drive should have taken an hour or so, but became much more complex. I'll describe the full saga of the cursed drive in another post, but to make a long story short we installed new heads that immediately crashed so badly that the head arms were physically bent. After installing another set of heads and fixing various other issues the cursed drive finally seemed to work, so we connected it to our Alto.2 Boot almost worked, except any disk in the cursed drive got hopelessly corrupted. To make things worse, our previously-working drive started seeking erratically and then stopped seeking entirely. We suspected an electrical problem with the cursed drive had damaged the Alto's interface board or the good drive's circuitry. This was rather distressing since now we couldn't use our Alto.3

At this point, I should explain a bit about the Diablo drive and the complex mechanism it uses for seeking. The seek circuitry has two purposes. First, when the Alto wants to read from a particular track, the drive must seek, moving the heads to the desired track as fast as possible. Then, the heads must be held perfectly steady over the track. (Keep in mind that a track is only 0.007 inches (.18mm) wide.) Instead of a stepper motor, the drive moves the heads with a DC motor controlled as a servo. To make seeks faster, the motor runs at four different speeds, accelerating quickly and then slowing as it approaches the desired track. Once the head reaches the desired track, the servo mechanism constantly adjusts the head positioner motor to keep the head centered over the track.

The Diablo drive's circuitry pulls up for repair. The drive has three circuit boards on the left and three on the right.

The seek logic is implemented by the three circuit boards on the right.4 These boards mostly use simple DTL (Diode Transistor Logic) gates, integrated circuits from the 1960s that predated TTL. The innermost board receives the desired track number from the Alto. The next board computes the difference between the drive's current track and the desired track and determines how fast to move the head. Finally, the rightmost board is the analog board that drives the head positioner motor as well as processing head position signals from the transducer. 5 In a modern system, the seek logic could be compactly implemented with a microcontroller. But in the 1970s, controlling the heads took three boards full of integrated circuits.

The photo below shows the disk heads and the head position transducer, a key component of the seek circuitry.6 The heads are in the foreground, two barely-visible white ceramic circles on flat metal arms. The head positioner motor (hidden underneath) moves the heads in and out to the appropriate track. The head position transducer, the green disk in the photo below, provides electronic feedback on the head position. The yellow pointer and the scale on the transducer show the track number visually.

The green head positioner transducer provides feedback to the head servo mechanism. The pointer and dial indicate what track the heads are on.

The transducer generates two "quadrature" signals 90° apart, with one pulse per track.7 The disk drive counts these pulses to determine the current track number. By looking at the phase of the two signals, the drive can determine the direction of head movement.8 The video below shows the two transducer outputs displayed in X-Y mode on an oscilloscope. As the head is (manually) moved, the dot rotates 360° on the screen for each track. The direction of rotation indicates which way the head is moving. When the head is aligned over the track, the dot is at the top of the screen. Thus, the transducer outputs show the direction of head motion, the number of tracks moved, and alignment over the track.

Getting back to our disk drive that had problems seeking, we did some testing and determined that seeking had totally failed. The drive did not seek when requested by the Alto or Carl's FPGA-based disk controller. The drive didn't return the head to track 0 when the disk was unloaded. It didn't even hold the head in place over a track. This let us know that the problem was not with the signals from the Alto but something inside the drive.

We figured the complex seek control circuitry must have malfunctioned, so our strategy was to swap the three seek board with boards from a spare drive. Then we could replace boards individually until we found which board had the problem. Much to our surprise, the problem still remained even after we swapped the boards.

An oscilloscope trace shows signals in the malfunctioning disk drive. The motor control signal (yellow) causes the motor to be driven with +15V and -15V (pink), but nothing shows up in the current-sensing resistor (green). Xerox Alto oscilloscope-bad.jpg

Next, we checked out the drive's seek signals with an oscilloscope (above). We found that the seek circuitry was generating a motor control signal (yellow) and the motor driver board was sending +15V or -15V to the head positioner motor (pink). Although these signals weren't really what we expected to see, with full voltage to the motor, the heads should have been moving back and forth rapidly instead of remaining stationary. Also, nothing was showing up across the current-sensing resistor (green).9

The head-seek motor is driven through a large current-sensing resistor (left). (A disk cable or terminator is attached to the connector on the right.)

Although the seek circuitry was complex, the actual motor wiring was fairly simple. The motor received up to +/- 15V from a driver board, and was connected to ground through a large (10W) 0.2Ω current sensing resistor (above). A bypass capacitor across the motor (below) filtered out noise. We suspected a failure of the current-sensing resistor, the bypass capacitor, or the motor itself, so we tested these components. A multimeter verified the resistor hadn't burnt out. A LCR meter showed the capacitor had the right capacitance. We powered the motor directly from a power supply and the heads moved back and forth smoothly. This was a puzzle: all the components tested fine and we had measured voltage from the motor driver board, so why was nothing moving?

The head positioning motor moves the heads back and forth. Drive wires (yellow) are bolted to the motor. A bypass capacitor (black) is connected across the motor.

At this point, Carl noticed that one of the wires on the motor was loose. He tightened the nut and the seek problems were immediately solved. After all our investigation, the problem with our drive was simply a loose wire that prevented power from getting to the motor. Vibration must have slowly loosened the nut until the drive quit working. Apparently it was just coincidence that the problem happened when we had the cursed drive connected.

Conclusion

It was a bit anticlimactic to find a simple loose wire after all our investigation of the seek circuitry. But we were happy to have our drive back in operation, so we could use our Alto again. We still have to diagnose the problem with the cursed drive, but hopefully we're getting closer; I plan to write another blog post once we get that problem solved.

My full set of Alto posts is here. Follow me on Twitter or RSS to find out about my latest blog posts.

Notes and references

1. Since the head flies at high speed above the disk surface, any particles on the disk can cause the head to crash into the disk surface, scratching the disk and clogging up the head with oxide. Usually the heads can be removed and cleaned. After reinstalling the heads, they need to be realigned with a special alignment pack so they are properly positioned over the tracks. ↩

2. The disk drives have two connectors on the back, so multiple drives can be daisy-chained together. This lets you have a two-drive Alto configuration, for instance. A terminator is connected to the last disk in the chain. Thus, the Alto was connected to the working drive in the Alto cabinet, which was then connected to the cursed drive. ↩

3. Our Alto wasn't totally dead without a disk drive, since we could boot over Ethernet using my Ethernet gateway. However, without a working disk drive the Alto was very limited. ↩

4. The three boards on the left of the drive aren't relevant for this repair, but I'll describe them for completeness. The leftmost board (J10) has the analog read/write circuitry that drives the heads. (You can see a wire from the upper left corner of the board going to the heads.) The next board (J9) controls the spindle drive motor, lowers the heads onto the disk after loading, and detects sector marks. The inner board on the left (J8) counts the sectors on the disk. It also generates the 5V supply and has an oscillator to drive the head position transducer. ↩

5. The Disk drive maintenance manual includes schematics and a detailed description of the drive's operation. ↩

6. Modern disk drives position the heads based on a servo track written on the disk, a technology developed in 1971 that provided better positioning accuracy. The Diablo drive on the other hand, used older technology where position feedback was part of the drive. ↩

7. The head position rotary transducer uses a special transformer to generate the position signals. A 50 kHz carrier signal is fed into the transducer. This signal is modulated based on the head position to yield two signals, the quadrature signals 90° out of phase. The transducer has two parts: a rotary member that receives the carrier signal, and a stationary member that provides the two output signals. I haven't disassembled the transducer, but based on similar rotary transducers, I believe the transducer is built from zig-zag windings etched into circular printed circuit boards in the transducer. The zig-zags are closely spaced around the transducer disk, with their spacing matching one track's rotation of the transducer disk. The two output windings have the same spacing, but are offset one quarter of a zig-zag, i.e. 90°. As the transducer rotates, the input winding will line up alternately in phase and opposite phase with the output winding, yielding a positive and then negative output, once per track. The other output winding behaves similarly, but 90° out of phase. ↩

8. A mechanical mouse uses a similar quadrature technique to determine the direction of motion. A mechanical mouse typically uses optical encoders rather than the disk drive's transformer encoders. ↩

9. We used an oscilloscope to examine the seek circuitry on a working drive, and found very complex, almost chaotic signals showing the constant adjustments of the servo circuitry to keep the head aligned.

   

   A working disk drive shows the complex signals in the servo mechanism. The input signal (blue) triggers variations in the motor control signal (yellow). The motor voltage (pink) is constantly adjusted so the motor current (green) tracks the control signal.

   The signals from the transducer are processed, combined, and differentiated to generate spikes (blue) as the head moves. This input is filtered to form the motor control signal (yellow). The voltage driving the head motor (pink) is constantly adjusted so the velocity signal (green, from the current-sense resistor) is proportional to the control signal (yellow). ↩

Xerox Alto's 3 Mb/s Ethernet: Building a gateway with a BeagleBone

The Alto was a revolutionary computer designed at Xerox PARC in 1973. It introduced the GUI, high-resolution bitmapped displays, the optical mouse and laser printers to the world. But one of its most important contributions was Ethernet, the local area network that is still heavily used today. While modern Ethernets handle up to 100 gigabits per second, the Alto's Ethernet was much slower, just 3 megabits per second over coaxial cable. Even so, the Alto used the Ethernet for file servers, email, distributed games, network boot, and even voice over Ethernet.

The Alto's 3 Mb/s Ethernet isn't compatible with modern Ethernet, making it difficult to transfer data between an Alto and the outside world. To solve this, I built a gateway using the BeagleBone single-board computer to communicate with the Alto's Ethernet. In this article I discuss how the Alto's Ethernet works and how I implemented the gateway.

The Alto's Ethernet hardware

The Alto's Ethernet used a coaxial cable, rather than modern twisted-pair cable. A transceiver box was the interface between the Alto and the cable, converting the Alto's logic signals to the signals on the cable. In the photo below, you can see a transceiver clamped onto the coaxial cable; a "vampire tap" punctures the cable making contact with the central conductor.

Inside a transceiver box. The "vampire tap" on the right connects the transceiver to the Ethernet cable. The connector on the left goes to the Alto.

The Alto's Ethernet uses Manchester encoding of the bits. That is, a "1" is sent as "10" and a "0" is sent as "01". The reason to do this instead of just sending the raw bits is the Manchester encoding is self-clocking—there's a transition for every bit. If instead you just sent a raw stream of bits, e.g. "00000000000", the receiver would have a hard time knowing where the bits start and end. The figure below shows how 0110 is transmitted with Manchester encoding.

An example of Manchester encoding, for the bits 0110.

The Alto's network protocols

The Alto predates TCP/IP, instead using a protocol called Pup (the PARC Universal Packet).1 The protocol has many similarities with TCP/IP, since the Pup designers influenced the TCP design based on their experience. The basic Pup Internet Datagram is analogous to an IP packet. A packet contains a destination address for the packet consisting of a network id (1 byte), a host id on the network (1 byte), and a 4-byte socket. A machine's host id is specified by jumper wires on the backplane (below). Thus, packets can be routed through a large network to the right machine.

Long blue jumper wires on the Alto's backplane specify the system's Ethernet address.

Some simple network operations just exchanged Pup packets, for example requesting the time from a network server. But for more complex operations, Xerox built layers on top of Pup. For a long-lived connection between two machines, Xerox built RTP (Rendezvous/Termination Protocol), a protocol to establish connections between two ports. This is analogous to a TCP socket connection. Once the connection is established, the computers can communicate using Byte Steam Protocol (BSP). This protocol handles error correction and flow control, very similar to TCP.

On top of these protocols, Xerox implemented email, FTP (File Transfer Protocol), network boot, networked games such as Maze War, network error reporting, network disk backups and the first computer worm.

Map of the Xerox PARC network, showing important servers. May 1978. From Alto User's Primer.

Designing an Ethernet gateway

I decided to build a gateway that would allow the Alto to communicate with a modern system. The gateway would communicate with the Alto using its obsolete 3Mb/s Ethernet, but could also communicate with the outside world. This would let us network boot the Alto, transfer files to and from the Alto and backup disks. I expected this project to take a few weeks, but it ended up taking a year.

The first decision was what hardware platform to use. My plan was to use a microcontroller to generate the 3Mb/s signals by "bit banging", i.e. directly generating the 1's and 0's on the wire. (A regular processor couldn't handle this in real time, due to interrupts and task switching.) But a microcontroller wouldn't be suitable for running the network stack and communicating with the outside world; I'd need a "real computer" for that. Someone suggested the BeagleBone: a credit-card sized Linux computer with an ARM processor that also incorporated two microcontrollers. This would provide a compact, low-cost implementation, so I started investigating the BeagleBone.

The BeagleBone single-board Linux computer is small enough to fit in an Altoids tin. It has many general/purpose I/O pins, accessible through the connectors along the top and bottom of the board.

The BeagleBone's microcontrollers, called PRUs,2 are designed to run each instruction in a single 5ns cycle. Since the Ethernet pulses are 170ns wide, I figured I had plenty of time to send and receive signals. (It turned out that getting everything done in 170ns was challenging, but it worked out in the end.) In my gateway, the PRU reads the Ethernet signal coming from the Alto, and generate the Ethernet signal going to the Alto, as well as sending a network collision signal to the Alto.

In case you're wondering what PRU code looks like, the C code fragment below shows how a byte is encoded and sent. The PRU code outputs the right sequence of HIGH/LOW or LOW/HIGH pulses (the Manchester encoding), making sure each part is 170ns wide. Each bit of register R30 controls an output pin, so the bit for the Ethernet write pin is set and cleared as needed. wait_for_pwm_timer() provides the 170ns timing for each pulse. The full code is here.

To receive data from Ethernet, my PRU code doesn't do any decoding; it just measures the time between input transitions and sends these timings to the BeagleBone's ARM processor. Originally my code decoded the Manchester encoding into bits and assembled the bits into words, but 170ns wasn't enough time to do this reliably. Instead, since this decoding doesn't need to be done in real time, I moved the decoding to the more powerful ARM processor. The ARM processor also does the higher-level Ethernet protocol handling, such as generating and checking packet checksums.

Rather than deal with transceivers and coaxial cable, I connected the BeagleBone to the Alto's Ethernet port directly, omitting the transceiver that normally connects to this port. This port provides standard 5V TTL signals to the transceiver, but the BeagleBone inconveniently uses 3.3V signals. I needed a circuit to translate the voltage levels between the BeagleBone and the Alto. This should have been easy, but I ran into various problems along the way.

On the back of the Alto, Ethernet is accessed via a DB-25 connector that connects to a transceiver box.

I built a prototype voltage translation circuit on a breadboard, using a 74AHCT125 level shifter chip (schematic). However, I was getting no Ethernet signal at all from the Alto, so I started probing the Alto's board for a malfunction. I discovered that although the Alto's schematic showed pull-up resistors, these resistors didn't exist on the Alto's Ethernet board (see photo below). Without a resistor, the open-collector signal stayed at ground. Adding the resistors to my circuit fixed that problem.

One problem I encountered was termination resistors R8 and R9 appeared on the schematic but were missing from the board (above the word "ETHERNET").

The next step was to write the PRU microcontroller code to send and receive Ethernet packets. After a lot of testing with the Alto's Ethernet Diagnostic Program (EDP), I was able to echo packets back and forth between the BeagleBone gateway and the Alto.

The Ethernet Diagnostic Program can be used to test the Ethernet. It has a simple GUI.

To be useful, the gateway must support the network stack: Pup packets, network booting, Alto-style FTP and so forth. I started by rewriting the Alto's network code, making a direct port of the BCPL implementation3 to Python. This turned out to be a mess, because the original implementation depended heavily on the Alto's operating system, which provided multiple threads, so the application could run in one thread and the network code in other threads. I managed to get the stateless low-level Pup packets to work okay, but the higher-level Byte Stream Protocol was a tangle of busy-waiting loops and timeouts.

Fortunately the Living Computers Museum+Lab (LCM+L) came to rescue. Josh Dersch at the museum had written a C# implementation of the Alto's network stack, so I decided to discard my implementation of the network stack. This implementation, called IFS,4 supported the low-level protocols, as well as servers for network operations, FTP, and Copydisk. IFS was a new implementation rather than a direct port like my attempt, and that approach worked much better.

The Living Computers Museum built a 3 Mb/s Ethernet interface using an FPGA chip.

The LCM+L also built an Ethernet interface to the Alto. Theirs was much more complex than mine, consisting of an FPGA chip on a custom PCI card plugged into a PC. Unlike my interface, theirs communicated with a transceiver box, rather than connecting to the Alto directly. Even though the LCM+L's FPGA interface worked for us, I decided to complete my interface board since I liked the idea of a low-cost, compact Ethernet gateway. Their interface is more "realistic", since it connects to a real transceiver and network cable and can work with multiple Altos on a network. (The photo below shows the mini-network connecting the LCM+L interface and an Alto.) On the other hand not everyone has Ethernet transceivers lying around, so mine is more widely usable.

A very short 3 Mb/s Ethernet network, showing two transceivers attached to the coaxial cable. One transceiver is connected to the Alto and the other is connected to the LCM+L gateway card.

I made a simple printed circuit board to replace my breadboarded prototype. This was the first circuit board I had designed in about 35 years and technology has certainly changed. Back in the 1980s, I drew circuit board traces with a resist pen on a copper-clad board, and then etched the board in ferric chloride solution from Radio Shack. Now, I designed my board using Eagle, uploaded the file to OSH Park, and received a beautiful circuit board a few days later.

My 3 Mb/s Ethernet interface is a level-shifter card on a BeagleBone. Note the capacitor soldered directly to the driver IC to fix a ringing signal problem.

Running the LCM+L's IFS software on the BeagleBone took some work since the BeagleBone runs Linux and IFS was written in C# for Windows. By using the Mono platform, I could run most of the C# code on the BeagleBone. However, the LCM+L software stack communicated with the gateway by encapsulating Alto Ethernet packets in a modern Ethernet packet, using a .Net Pcap library to access the raw Ethernet. Unfortunately, this library didn't work with Mono, so I rewrote the IFS encapsulation to use UDP.

At this point the Alto and my BeagleBone implementation communicated most of the time, but there were a few tricky bugs to track down. The first problem was FTP would get sometimes bad packets. Eventually I discovered that I had mixed up byte and word counts in one place, and the large packets from FTP were overflowing the PRU's buffer.

The next problem was that when I used Copydisk to copy a disk pack from the Alto to the BeagleBone, a dropped packet caused the network to lock up after a couple minutes. This was puzzling since the sender should retransmit the lost packet after a timeout and everything should recover (like TCP). After tedious debugging, I found that if the Alto and the BeagleBone transmitted packets close together, a race condition in my code caused a dropped packet.56 Then IFS neglected to send a necessary ack when it received an unexpected packet, so the Alto kept resending the wrong packet. After fixing both my gateway code and the IFS ack, the network ran without problems.

Second version of the interface. Note the blue jumper wire to replace the missing trace.

My first version of the PCB had a few issues8 so I made a second version (above). I found that the driver chip didn't sink enough current for a long cable, so I replaced it with the chip used by the transceivers (a 7438 open collector NAND gate). I also decided to try KiCad instead of Eagle for the second board. Unfortunately, I made a mistake in KiCad7 and the new board was missing a trace, so I had to solder in a jumper (which you can seen in the photo above).

I probed the Alto's Ethernet card with an oscilloscope to check the signals it was receiving. An extender card from the LCM+L allowed us to access the card.

I tested the new board but it didn't work at all: the Alto wasn't receiving data or sending data. I realized that the new driver chip was a NAND gate so the signals were now inverted. I updated the PRU code to flip LOW and HIGH on the data out signal. The board still didn't work, so I probed the Alto's Ethernet board with an oscilloscope, comparing the signals from the old working board to the new failing board. Strangely, both signals were identical in the Alto and looked fine.

Eventually Marc asked me if there could be something wrong with the collision detection. I realized that the the new NAND driver was also inverting the collision detection signal. Thus, the Alto was seeing constant network collisions and refused to send or receive anything. After fixing this in my PRU software, the new board worked fine. I'm making a third revision of the board to fix the missing trace; hopefully this won't break something else.

An assortment of vintage Ethernet transceivers, along with the tap tools used to drill holes in the Ethernet cable for the vampire tap connection.

Conclusion

The Ethernet gateway project took much more time and encountered more problems than I expected. But it worked in the end, and several Alto owners are now using my gateway. I've open-sourced this interface (the board layout and the gateway software); it's available on github.

The Alto I've been restoring came from YCombinator; the restoration team includes Marc Verdiell, Carl Claunch and Luca Severini. My full set of Alto posts is here and Marc's extensive videos of the restoration are here. Thanks to the Living Computers Museum+Labs for the IFS software.

Follow me on Twitter or RSS to find out about my latest blog posts.

Notes and references

1. For more information on the Pup protocol, see Pup: An Internetwork Architecture and the Pup Specifications memo.  ↩

2. Programming the PRU microcontrollers has a difficult learning curve, and it's not nearly as well documented as, say, the Arduino. Based on my experience with the PRU, I wrote some programming tips here and here. ↩

3. The Alto's network code is online; you can look at one of the Pup files here. The code is written in BCPL, which was a predecssor to C. It is very similar to C, but with different syntactic choices. For instance, >> is not a shift but accesses a field in a structure. ↩

4. At PARC, IFS was an acronym for Interim File System, an Alto-based distributed file server. It was called interim because an improved system was being developed, but somehow a new system never replaced IFS. ↩

5. Most of the time, the BeagleBone and the Alto alternated sending packets. But every second, IFS sent a "breath of life" packet, the packet executed by the Alto to start a network boot. The breath of life packet was part of the original Alto—it simplified the network boot code since the Alto didn't need to request a network boot, but just listen for a second or two. Since the breath of life packet was asynchronous with respect to the other network traffic, eventually it would happen at a bad time for my code, triggering the dropped packet problem. ↩

6. The problem with the PRU and the "breath of life" packet was due to how I was sending signals between the PRU and the main BeagleBone processor. When sending a packet, the main processor would signal the PRU, and then wait for a signal back when the packet was sent. When a packet was received, the PRU would send a signal to the main processor. The problem was that the PRU sent the same signal in both cases. So a "packet received" signal at the wrong time could wake up the packet sending code, losing the received packet. Fundamentally, I was treating the signals between the main processor and the PRU as synchronous, when everything was really asynchronous. I rewrote the gateway code so ownership of the send and receive buffers was explicitly transferred between the main processor and the PRU. A signal just indicated that ownership had changed somehow; checking the buffer status showed what exactly had changed. See this discussion for details. ↩

7. My error in KiCad was I placed a net label just a bit too far from the wire on the schematic, so the wire wasn't connected to the net. ↩

8. My first PCB had a couple issues that I had to hack around. Ringing in the signal to the Alto corrupted it, so I added a damping capacitor. The driver didn't have enough current for an Ethernet cable longer than 10 feet, so I added a pull-down resistor. There were also a couple mechanical issues. The location of the Ethernet jack made it hard to remove the cable, and the board blocked one of the BeagleBone buttons. I also discovered that using a 74HC125 driver chip didn't work at all; just the 74AHCT125 worked. ↩

Xerox Alto zero-day: cracking disk password protection on a 45 year old system

We've been archiving a bunch of old Xerox Alto disk packs from the 1970s. A few of them turned out to be password-protected, so I needed to figure out how to get around the password protection. I've developed a way to disable password protection, as well as a program to find the password instantly. (This attack is called XeroDay, based on a suggestion by msla.)

The Xerox Alto. The disk drive is the black unit below the keyboard. The processor is behind the grill. The Mandelbrot set has nothing to do with this article.

The Alto was a revolutionary computer designed at Xerox PARC in 1973 to investigate personal computing. In the photo above, the Alto computer itself is in the lower cabinet. The Diablo disk drive (in 1970s orange, below the keyboard) takes a removable 14 inch disk pack that stores 2.5 megabytes of data. (A bunch of disk packs are visible behind the Alto and in the photo below.) I've been restoring a Xerox Alto, along with Marc Verdiell, Luca Severini and Carl Claunch. (The full set of Alto posts is here and Marc's videos are here.)

Some of the disks from Xerox PARC that we're archiving.

Now that we have the Alto running, one project is to archive a bunch of disks that have been sitting at Xerox PARC for decades, and find out if there are any interesting treasures among the disks. We can archive disks by running the Copydisk program on the Alto, and copying them over Ethernet to a PC server. (Ethernet was invented by Xerox for the Alto.) However, it's considerably faster to read the data directly off the disk drive, bypassing the computer. Carl created an FPGA-based disk controller (below) that connects to the Diablo disk drive, speeding up the archiving process.1

Diablo disk tool, built by Carl Claunch using an FPGA.

Before reading each disk, we open up the pack and carefully clean the surface. After storage for decades, these disks have some grime, dust, and the occasional bug (of the dead insect variety), so we need to clean them to reduce the chance of a head crash.

A Xerox Alto disk pack, opened for cleaning. The "flaws" on the surface are just reflections.

Most of the archived disks can be booted on the Alto or the ContrAlto simulator. But a few disks only booted to a password prompt (see below), and we couldn't use the disk without the password. So I decided to hack my way into the password-protected disks.

Booting a password-protected disk results in a request for the password.

The Alto documentation discusses password protection, explaining how a password can be associated with a disk. It only promises "a modest level of security", which turns out to be true. It also says if you forget the password, "you will need an expert to get access to anything on your disk." But could I break in without finding an expert?

Password protection is described in the Alto User's Handbook page 5.

A bit about passwords

Storing passwords in plain text is a very bad idea, since anyone who can access the file can see the password.9 Most systems use a solution invented by Roger Needham in 1967. Instead of storing the password, you hash the password through a cryptographic one-way function and store the hash. When the user inputs a password, you hash it through the same function and compare the hashes. If they match, the passwords match. And if anyone sees the hash, there's no easy way to get the password back.

One problem with hashed passwords is if two users have the same hash, then you know they have the same password. A solution (invented in Unix) is to hash some random bytes (called salt) along with the password to yield the stored hash. Since different users will have different salt, the hashes will be different even if the passwords are the same. (Of course you need to store the salt along with the hash in order to check passwords.)2 Like Unix, the Alto used salted and hashed passwords.

The Alto's hash algorithm

The source code for the Alto's password algorithm reveals how the password hashing is implemented.3 The Alto uses four words of salt with the password (two words based on the password creation time and two words based on the user name). The password hash is 4 words (64 bits) long. The Alto's password hash algorithm is pretty simple:4

Hash c = -a*x*x + b*y

where a is the time salt and b is the user name salt. x is a one-word value generated from the password string and y is a two-word value from the password string, both generated by concatenating characters from the password.5

Disabling the password

There's a way to disable the password on disk, gaining access to the file system.6 The Alto boots from disk by running the file sys.boot; this file decides if a password is required for boot, based on the 9-word password vector stored inside the file. The first word is a flag indicating if the disk is password protected. If the password flag is set, the boot program requires a password before proceeding. The next four words are the salt, and the final four words are the password hash itself.

The password protection can be disabled by clearing the flag word inside sys.boot, which is the 128th word in the second block of sys.boot. The tricky part is finding where this word is on disk. The file system stores a directory as the name of each file along with the disk address of the file's first block. By reading the directory as raw data and interpreting it, we can find the location of sys.boot. In the Alto file system, each disk block has pointers to the previous and next block. So once we've found the first block, we can follow the pointer to find the second block. (Basically I re-implemented a subset of the Alto file system using the raw disk.) Erasing the password flag in this block makes the disk bootable without the password.

After implementing this, I realized there's a short cut. The writers of the disk bootstrap code didn't want to re-implement the file system either, so the Alto simply copies the first block of sys.boot to the first disk sector. This makes it trivial to find the file, without needing to scan the directory. Once I have the first block, I can find the second block from the pointer, access the block, and clear the password flag, disabling the security.7

I made a simple Python program to update the disk image file and clear the password flag (link). After running this program, I was able to access the protected disks. The password-protected disks didn't have any super-secret treasures. However, one disk contained an implementation of APL in Mesa, which was interesting. (APL is a cool programming language known for its extremely terse notation and strange character set. Mesa is a high-level language developed at Xerox PARC; it influenced Java.) We haven't gotten Mesa running on the Alto yet, but this looks like an interesting thing to try out.

After defeating the password, I can view the disk contents. These files implement APL in Mesa.

Brute-forcing the password

While I could access the disk simply by clearing the password flag, I wondered what the original passwords were. I implemented the password algorithm in C (link), so I could rapidly test passwords. My hope was that testing against a list of top 100,000 passwords would find the passwords, since I didn't expect much in the way of 1970s password security practices. Surprisingly, there were no hits so the passwords weren't common words. My next step was brute-forcing the password by generating all strings of length 1, 2, 3 and so forth.8 Finally at length 8, I cracked the first password with "AATFDAFD". The second disk had password "HGFIHD" and the third had "AAJMAKAY". Apparently random strings were popular passwords back then.

I was a bit suspicious when I saw that both 8-character passwords started with "AA" so I investigated a bit more. It turned out that using "AB" in place of "AA" also worked, as did starting with anything in "{A-Z}{A-G}". Studying the algorithm more closely, I realized that when x and y get too long, the old character bits were just dropped. Thus, when you use a password longer than 6 characters, most of the bits from the first characters are lost. This is a pretty big flaw in the algorithm.

Finding the password with math

It takes half an hour or so to brute force the password; can we do better? Yes, by doing some algebra on the password formula yielding:

y = (c + a*x*x) / b

where x and y are generated from the password string, a and b are salt, and c is the stored password hash. Since x is only 16 bits, we can easily try all the values, finding ones for which the division works. When we find a solution for y, we can recover the original password by chopping x and y into 7-bit characters. Using this technique, the password can be recovered almost instantly. I implemented this in Python here.

Conclusion

The Xerox Alto's disk passwords can be trivially bypassed, and the password can be easily recovered. The password algorithm has multiple flaws that make it weaker than you'd expect (dropping password characters) and easily reversed. Since the system is almost 45 years old, they had to keep the code small and they weren't facing modern threats. After all, Xerox only promised "modest" security with the passwords, and that's what it provided.

Notes and references

1. For more details on the archiving process, see Carl's post on archiving. He writes about the FPGA disk tool here. ↩

2. Salting passwords also protects against password attacks using precomputed rainbow tables, but that wasn't a concern back then. ↩

3. The password code can be viewed at Password.bcpl. The code is in BCPL, which is a predecessor of C. The syntax has some trivial but confusing differences; I'll explain the most important. p!1 is just an array access, equivalent to p[1] in C. vec 4 allocates 4 words, essentially malloc(4). ps>>String.char↑i is equivalent to ps->String.char[i], accessing a character from the password structure. $a is 'a' and rem is %. Square brackets in BCPL are blocks, like curly braces in C. Also note that the code has inline assembly for the vector addition and multiplication functions. ↩

4. The negation in the hash function is broken; only the top two words are negated. The Password.bcpl code points out this bug, but notes that they couldn't fix it without invalidating all the existing passwords. ↩

5. The x value is generated by concatenating the 1st, 4th, ... characters of the password, while y consists of the other characters. The concatenation is done using 7-bit characters, for some reason. ↩

6. I thought I could boot off an unprotected disk and then use the Neptune graphical file browser to view protected files in the second drive. However, they thought of this and Neptune also checks for a password. In the screenshot below, Neptune shows the contents of the left disk, but requires a password to show the contents of the right disk. ↩

   The Neptune file browser checks for a password.

7. A few more details about the Alto file system for reference. Each disk sector consists of a 2-word header, an 8-word label, and a 256-word data record, each with a checksum. (The Alto's file system (like the Alto) uses 16-bit little-endian words.) The file system structures are defined in include file AltoFileSys.D. The header contains the physical disk address of the sector (sector number, track number, head number), which is used to validate that the sector returned from the drive is the correct sector. The label contains file system information: pointers to the next and previous sectors in the file, the number of characters used and the file id. This information can also be used to recover from corruption using the scavenger program, which is similar to Unix's fsck. See the Operating System Reference Manual; section 4.2 describes disk files. ↩

8. The Alto's password algorithm converted all lower case letters to upper case, which reduced the search space. I'm not sure if special characters are allowed or not. ↩

9. The code that calls the password routine has comments about making sure the cleartext password never gets stored on disk, and the in-memory password buffers are zeroed out so they don't get swapped to disk. So it's clear that the writers of the password code were thinking things through. ↩

Fixing the Ethernet board from a vintage Xerox Alto

A Xerox Alto system on the East coast had Ethernet problems, so the owner sent me the Ethernet board to diagnose. (After restoring our Alto, we've heard from a couple other Alto owners and try to help them out.) This blog post describes how I repaired the board and explains a bit about how the Alto's groundbreaking Ethernet worked.

The Alto was a revolutionary computer designed at Xerox PARC in 1973, introducing the GUI, high-resolution bitmapped displays and laser printers to the world. But one of its most important contributions was Ethernet, the local area network that is still heavily used today. While modern Ethernets handle up to 100 gigabits per second, the Alto's Ethernet was much slower, just 3 megabits per second over coaxial cable. Even so, the Alto used the Ethernet for file servers, email, network boot, distributed games and even voice over Ethernet.

An extender board made it easy to probe the Ethernet card in the Xerox Alto. The chassis is pulled out to access the boards. Above the chassis is the disk drive, which stores just 2.5 megabytes on a 14-inch disk pack.

The Alto's chassis slides out of the cabinet, allowing access to the circuit boards (see above). To test the Ethernet board, I plugged it into the Alto we've restored. The Ethernet board is hanging out the right side of the chassis because I used an extender board.

The board had a couple straightforward problems—a chip with a bent pin (which I straightened) and another chip with broken pins. (While some people recommend re-seating chips on old boards, this can create new problems.) The broken chip was a 74S86, a quadruple XOR gate that has been obsolete for years (if not decades). I replaced it with a 74LS86, a similar but non-obsolete chip.1

When I powered up the Alto, I didn't get anything from the Ethernet board: it was not sending or receiving successfully. This double failure was a bit surprising since malfunctions usually affect just one direction. But I quickly discovered a trivial problem: when I pulled out the Alto's cabinet to access the circuit boards, the Ethernet connector on the back came loose. After plugging the connector in, I saw that the Alto was sending packets successfully, but still wasn't receiving anything.

Oscilloscope trace from the Ethernet board. Yellow is input data, green is R-C filtered, blue is detected edges. The blue trace is a bit of a mess with runt pulses.

I started probing the Ethernet board's input circuit with the oscilloscope. The board was receiving the input okay, but a few gates later the signals looked kind of sketchy, as you can see above. The yellow trace is the input Ethernet signal. The board applies an R-C filter (green) and then the signal edges are detected (blue). While the input (yellow) is a clean signal, the green signals only go up to about 2 volts, not the expected 4-5 volts. Even worse, the blue waveform is irregular and has "runt" pulses—short irregular signals that don't go all the way up. With waveforms like this, the board wouldn't be able to process the input.

Manchester encoding and the decoding circuit

I'll take a detour to explain the Ethernet's Manchester encoding, before getting into the details of the circuitry.

The Alto's Ethernet encoded data over the coaxial cable using Manchester encoding.32 In this encoding, a 1 bit is sent as a 1 followed by a 0, while a 0 bit is sent as a 0 followed by a 1. The diagram below shows how the bits "0110" would be encoded. A key reason to use this encoding is that simplifies timing on the receiver, since it is self-clocking. Note that there is a transition in the middle of each cell that can be used as a clock. (The obvious scheme of just sending the bits directly has the problem that in a long string of 0's, it's hard to know exactly how many were transmitted.)

An example of Manchester encoding, for the bits 0110.

To decode the Ethernet signal arriving at the Alto, the first step is to extract the clock signal that indicates the timing of the bits. To extract the clock, the transitions in the middle of each bit cell must be detected. The trick is to ignore transitions at the edges of cells. The diagram below shows how the clock is extracted. First, edge pulses are generated for each transition edge of the input. Then, a clock pulse is generated for each edge pulse. The clock pulse about 75% of the width of the bit cell, and any second pulse that happens while the clock is still high is blocked. (The edge circled in red is an example of an ignored edge.) The result is the steady clock signal, synchronized to the input.4 This clock is used to latch the input signal value into a shift register, which converts the serial Ethernet stream into a word value usable by the Alto.

Extracting the clock signal from a Manchester-encoded Ethernet input. Edges are extracted from the input signal and used to trigger the clock. An edge too close to the previous (red) is dropped.

The schematic5 below shows the circuit that performs this clock extraction. There are a few tricks in this interesting semi-analog circuit. When the Ethernet input signal changes, the low-pass R-C (resistor capacitor) filter output will be slower to change. This will cause the edge detect XOR gate to go high briefly until the R-C signal catches up. Then the clock is generated by a "one-shot", a chip that generates a fixed-width pulse for each input pulse. The pulse width is set to about 75% of the bit cell width by another resistor and capacitor. The one-shot is wired to ignore any additional pulses that occur while the clock output is high.

Schematic of the clock extraction circuit in the Xerox Alto's Ethernet interface.

Since the timing in this circuit was controlled by multiple resistors and capacitors, I thought that a capacitor might have gone out of tolerance. But first I decided to do the simpler test of swapping the XOR chip with the one from our working Alto board. Surprisingly, this fixed the problem, and the Ethernet board now functioned flawlessly. Thus, while the 74LS86 has input characteristics similar to the 74S86, they weren't close enough, and substituting the more modern chip messed up the R-C filter's behavior. (If the gate draws too much input current, the capacitor won't charge as fast.) Marc dug through his collection of old chips and found another 74S86 so we could get both boards operating.

Oscilloscope trace showing the extracted clock (pink), edges of the Ethernet input (yellow) and the R-C filtered input (green).

The oscilloscope trace above shows the nice, steady clock (pink) after replacing the XOR chip. With the right XOR chip, the RC-filtered signal (green) rises to 4 volts, unlike before when it just went to 2 volts. The edge signal (yellow) is nice and even, without the "runt pulses" seen earlier.

An overview of the Ethernet board

I'll wrap up by explaining a bit about the Alto's Ethernet board (shown in the photo below). The large connector at the left plugs into the Alto's backplane, providing communication with the rest of the system. The small connector at the right connects to the Ethernet, via a transceiver. Some of the key components of the Ethernet interface are labeled on the diagram below. With this generation of chips, even a whole board of chips doesn't give you a lot of functionality so much of the Ethernet functionality is implemented in software and microcode.

The Xerox Alto's Ethernet board, showing major functional blocks.

Most of the chips are simple TTL logic, but the board uses a few interesting and obscure chips. Input and output data goes through a 16-word FIFO buffer built from four Intel 3101A RAM chips; each one holds just 16×4 bits. (The 3101 was Intel's first product.) Commands to the Ethernet board are decoded by three Intel 3205 decoder chips. Buffer chips interface between the card and the Alto's bus. Shift registers for the input and output convert between data words and serial data over the Ethernet. The CRC error checking is done by a specialized Fairchild 9401 CRC chip. Surprisingly, the Manchester encoding is implemented with a PROM-driven state machine, built from two Intel 3601 PROMs. Finally, the XOR chip is the chip that caused all the problems.5

The board has TTL inputs and outputs, so it doesn't connect directly to the Ethernet cable. Instead, a transceiver box is the interface between the Alto and the cable. In the photo below, you can see the transceiver clamped onto the coaxial cable. The transceiver converts the Alto's logic signals to the signals on the cable and injects the signals through an isolation transformer. In addition, the transceiver detects collisions if the Alto tries to send at the same time as another machine on the network.

The transceiver injects the Xerox Alto's signals into the Ethernet coaxial cable and reads signals from the cable.

You might wonder how the Alto can communicate with a modern system, since the Alto's 3 Mb/s Ethernet isn't compatible with modern Ethernet. I am building a gateway based on the BeagleBone (below) that uses the BeagleBone's microcontroller (the PRU) to "bit bang" the Ethernet signals. The BeagleBone also runs an implementation of IFS (Interim File System), the server side software to support the Alto. This is a C# implementation of IFS written by the Living Computers Museum.

I'm building an interface to the Alto's 3Mb/s Ethernet, using a BeagleBone. The cable on the right goes to the Alto's Ethernet port, while a standard Ethernet plugs into the left.

Conclusion

Bob Metcalfe invented Ethernet at Xerox PARC between 1973 and 1974, with a 3 megabit per second Ethernet for the Alto. Since then, Ethernet has become a ubiquitous standard, and has increased in speed more than 4 orders of magnitude. Unfortunately, as with many other technologies, Xerox failed to take advantage of Ethernet as a product. Instead, Bob Metcalfe left Xerox in 1979 and formed the company 3Com, which sold Ethernet products and became one of the most successful technology firms of the 1990s.

I tracked down a problem in an Alto Ethernet board—my replacement for a broken chip was almost the same, but incompatible enough to make the circuit fail. The problem showed up in the circuit that extracted the clock from the incoming Ethernet signal. This circuit illustrates some clever techniques used by the Alto designers to implement Ethernet with simple TTL chips.

The Ethernet card plugged into the Xerox Alto's chassis. The disk drive is above the chassis, with a white 14" disk pack visible inside.

The Alto I've been restoring came from YCombinator; the restoration team includes Marc Verdiell, Carl Claunch and Luca Severini. My full set of Alto posts is here and Marc's extensive videos of the restoration are here. Thanks to the Living Computers Museum+Labs for the extender card and the IFS software.

You can follow me on Twitter here for updates on the restoration.

Notes and references

1. The 7400 family of TTL chips has many subfamilies with different characteristics. The 74S components use Schottky diodes for faster performance than standard TTL. The 74LS (low-power Schottky) chips use much less power (1/8 in this case), but are slightly slower and draw slightly different input currents. ↩

2. Ethernet started with Manchester encoding and continued to use this encoding up to 10 Mb/s speed. However, Manchester encoding is somewhat inefficient since each bit can have two transitions, doubling the bandwidth requirement. 100 Mb/s Ethernet solved this by using MLT-3 encoding, which uses three voltage levels. To avoid the problem of losing the clock on a long string of 0's, each group of 4 bits is sent as a particular 5 bit pattern that eliminates long repetitive sequences. Ethernet gets progressively more complex as speed increases. At 100 Gb/s Ethernet, blocks of 64 bits are encoded with 66 bit patterns and data is split across multiple lanes. ↩

3. Manchester encoding was developed for magnetic drum storage for the University of Manchester's Mark 1 computer. The Mark 1 was an influential early computer from 1949, one of the earliest stored-program computers, also introducing index registers and Williams tube storage. ↩

4. One potential problem with the clock extraction is if you have a steady stream of bits (1111... or 0000...) the Ethernet signal will be a steady stream of transitions, and you can't tell where to start the clock. The solution is that each Ethernet transmission starts with a sync bit to get the clock started properly. ↩

5. The schematics for the Alto's Ethernet board are at Bitsavers. The Ethernet is described in more detail in the Alto Hardware Manual. ↩