iPad charger teardown: inside Apple's charger and a risky phony

This article is now available in Turkish iPad şarjının iç yapısı and Chinese [中文翻译版本] Apple iPad原装充电器拆解.
Apple sells their iPad charger for $19, while you can buy an iPad charger on eBay for about $3. From the outside, the chargers look the same. Is there a difference besides the price? In this article, I look inside real and counterfeit chargers and find that the genuine charger has much better construction, power quality, and most importantly safety. The counterfeit turns out to be a 5 watt charger in disguise, half the power of a genuine charger.

iPad
Counterfeit
A real Apple iPad charger (left) and a counterfeit charger (right

From the outside, the real charger (left) and counterfeit charger (right) are almost identical. If you look very closely, you can spot are a few differences in the text: The counterfeit removed "Designed by Apple in California. Assembled in China" and the manufacturer "Foxlink"[1], probably for legal reasons. (But strangely, the counterfeit still says "TM and © 2010 Apple Inc.") The counterfeit charger displays a bunch of certifications (such as UL) that it doesn't actually have. As you will see below, there is no way it could pass safety testing.

Opening up the chargers reveals big differences between them. The genuine charger on the left is crammed full of components, fitting as much as possible into the case. The counterfeit charger on the right is much simpler with fewer components and much more empty space. The Apple charger uses larger, higher-quality components (in particular the capacitors and the transformer); below you will see that these have a big effect on power quality and safety.

iPad
Counterfeit
The components inside a real iPad charger (left) and a counterfeit charger (right).

The components inside a real iPad charger (left) and a counterfeit charger (right).

One safety difference is obvious: the Apple charger has much more insulation. The upper (high-voltage) half is wrapped in yellow insulating tape. Some components are encased in shrink tubing, there are plastic insulators between some components, and some wires have extra insulation. The counterfeit charger only has minimal insulation.

The build quality of the Apple charger is much higher. In the counterfeit charger, some components are visibly crooked or askew. While this doesn't affect the circuit electrically, it indicates a lack of care in construction.

Flipping the boards over reveals that the circuitry of the genuine Apple charger is much more complex than the counterfeit. The Apple board is crammed with tiny surface-mounted components in every available spot. The counterfeit board has a lot of empty space, with just a few components. Note the reddish insulating tape in the lower center of the Apple board, another safety feature of the genuine charger.

iPad
Counterfeit
The circuit board of a real iPad charger (left) and a counterfeit charger (right).

The circuit board of a real iPad charger (left) and a counterfeit charger (right).

How the chargers work

Both the real and counterfeit chargers use similar flyback[2] switching power supply circuits. The switching power supply is the innovation that allows these chargers to be so compact, unlike the heavy "wall warts" powering older consumer electronics. The principle of a switching power supply is the power is switched on and off tens of thousands of times a second, allowing it to provide the exact amount of power required with very little power wasted as heat. In addition, the high frequencies allow the charger to use a small transformer, unlike the bulky transformers used for 60 Hz AC.

Since the counterfeit charger is much simpler, it is easier to understand how it works and I'll explain it first in reference to the picture below. The AC power enters through the white wires in the upper left. It passes through a fusible resistor, which acts as a safety fuse. Below this, the bridge rectifier contains four diodes which convert the AC into DC (at about 170 to 340 volts[3]). The input capacitor smooths out this power. The 4-pin control IC[4] monitors the charger and uses the switching transistor to turn the high-voltage DC on and off 41,000 times per second. This chopped DC is fed into the primary winding of the flyback transformer. The transformer converts this to the desired high-current 5 volts. The output diode produces DC, and the output capacitor smooths it out. Finally, the output voltage is available at the USB connector to power your iPad. A few components round out the circuit. A feedback winding on the transformer provides voltage feedback to the control IC. This winding also powers the IC; the IC power capacitor smooths out this power. Finally, the blue snubber[5] capacitor absorbs current spikes when the transistor is switched off.[6]

Counterfeit
Inside a counterfeit iPad charger

Inside a counterfeit iPad charger

The genuine iPad charger below operates on similar principles, although the circuit is more advanced. The AC input is on the lower right, and goes through a 2A fuse (in black insulation for safety). The primary has much more filtering than in the counterfeit charger with a filter coil (common mode choke), inductor, and two large electrolytic capacitors. This increases the cost, but improves the power quality. On the output side (left), the charger has two filter capacitors, including a high-quality aluminum polymer capacitor (with the magenta stripe). The Y capacitors help reduce interference.[7] The tiny NTC temperature sensor lets the charger shut down if it overheats. (I removed some of the charger's insulation to make the components visible in this photo.)

iPad
Inside a genuine iPad charger.

Inside a genuine iPad charger.

On the other side of the circuit board, things get complicated in the Apple charger. Starting with the AC input in the upper right, the charger includes additional input filters as well as spark gaps.[8] The latch release circuit[9] lets the charger reset quickly from faults. The control IC[10] provides advanced control of the charger under varying conditions. (This IC is much more complex than the control IC in the counterfeit charger.) The current sense resistor lets the IC monitor the current through the transformer and the line voltage resistors let the IC monitor the input voltage (as well as initially powering-up the IC[9]). The protection circuit uses the temperature sensor on the other side of the board to shut down if there is an over-voltage or over-temperature problem. [20]

iPad
The circuit board inside a genuine iPad charger showing the components.

The circuit board inside a genuine iPad charger showing the components.

The secondary side includes some special features for power quality. The Y-capacitor filter works with the Y capacitors to filter out noise. The output filter circuitry is more complex than in the counterfeit. Note that the real charger has a ground connection, unlike the counterfeit charger which has a plastic pin here.[11]

Both chargers use resistors to put special voltages on the USB data lines[12] to indicate the charger type, using Apple's proprietary system (details). (This is why iPads say "Charging is not supported with this accessory" with some chargers.) Through these resistors the genuine charger indicates that it is an Apple 2A charger, while the counterfeit indicates that it is an Apple 1A charger. This shows that the counterfeit is really a 5W charger packaged as a 10W charger.

When looking at these circuits up close, it's easy to forget just how small the components are. The picture below shows one of the surface-mount components (a 0-ohm resistor[13]) from the iPad charger. It is just to the left of Roosevelt's chin on the dime.

A zero-ohm resistor from the iPad charger, on top of a dime

A zero-ohm resistor

Safety, or lack thereof

Safety probably isn't something you think about when you plug in your charger, but it's important. Inside the charger is 170 volts or more with very little separating it from your iPad and you. If something goes wrong, the charger can burn up (below), injure you, or even kill you. Devices such as chargers have strict safety standards[14] - if you get a charger from a reputable manufacturer. If you buy a cheap counterfeit charger, these safety standards are ignored. You can't see the safety risks from the outside, but by taking the chargers apart, I can show you the dangers of the counterfeit.

Counterfeit iPhone
Counterfeit iPhone charger that burned up
A Counterfeit iPhone charger that burned up. Photo by Anool Mahidharia. Used with permission

Creepage and clearance

The UL regulations[14] require safe separation between the high voltage and the low voltage. This is measured by creepage - the distance between them along the circuit board, and clearance - the distance between them through air. The regulations are complex, but in general there should be at least 4mm between high-voltage circuitry and low-voltage circuitry.

iPad
The iPad charger provides safe creepage and clearance distances between the high-voltage side (bottom) and low-voltage side (top).

The iPad charger provides safe creepage and clearance distances between the primary high-voltage side (bottom) and secondary low-voltage side (top).

The image above shows how the genuine iPad charger's circuit board separates the high voltage (bottom) from the low voltage (top). The happy face on the right marks an empty region that provides a safety gap between the primary and secondary. (This is a contrast with the rest of the circuit board, which is crammed full of components.) This gap of 5.6mm provides a comfortable safety margin. The happy face on the left marks a slot in the board that separates the low voltage and high voltage. The photo below shows how an insulating fin is built into the case and through this slot to protect the USB connector. Additional reddish-brown insulating tape goes through this slot, and the whole high-voltage section is wrapped in yellow insulating tape. The result is multiple layers of protection.

iPad
The iPad charger case has a plastic fin that slides around the USB port to provide extra insulation.

The iPad charger case has a plastic fin that slides around the USB port to provide extra insulation.

The creepage distance on the counterfeit charger board below is scary - only 0.6 mm separation between low and high voltage. The sad face on the right shows where a low-voltage trace is nearly touching the high-voltage trace below. (The ruler on the right indicates millimeters.) The board isn't as bad as it could be: the happy face on the left marks a slot cut in the circuit board under the transformer to increase the creepage distance. But overall, this board is unsafe. If you use the charger in a humid bathroom and a drop of water condenses across the 0.6 mm gap, then zap!

Counterfeit
Dangerous creepage in a counterfeit iPad charger.

Dangerous creepage in a counterfeit iPad charger.

Safety in the transformer

For safety, the high-voltage and low-voltage sides of the charger must be electrically isolated.[15] But obviously the electrical power needs to get through somehow. The flyback transformer accomplishes this task by using magnetic fields to transfer the power without a dangerous direct connection. Because the transformer is a large and relatively expensive component, it is tempting to take safety and quality short cuts here. The genuine transformer (left) is considerably larger than the counterfeit (right), which is a hint of better quality and more power capacity. Disassembling the transformers shows that this is the case.

iPad
Counterfeit
The flyback transformers from an iPad charger (left) and a counterfeit charger (right). Dime and banana are for scale.

The flyback transformers from an iPad charger (left) and a counterfeit charger (right). Dime and banana are for scale.

The key safety requirement of the transformer is to separate the high-voltage windings from the low-voltage secondary winding, and the counterfeit charger fails here. The pictures below show the transformers after removing primary windings and insulating tape, revealing the secondary winding. The wires look similar at first glance, but the the genuine charger (left) has triple-insulated wire while the counterfeit (right) is uninsulated except for a thin varnish. The triple-insulated wire is an important safety feature that keeps the high voltage out even if there is a flaw in the insulating tape and in the wire's insulation. Also note the additional black and white insulation on the wires where they leave the transformer. In the counterfeit charger, the only thing separating the secondary winding from high voltage is the insulating tape. If there is a flaw in the tape or the wires shift too far, then zap!

iPad
Inside the transformer of an iPad charger, This is the triple-insulated secondary winding.
Counterfeit
The secondary winding does not have triple-insulated wiring. This is a major safety flaw in the counterfeit iPad charger.

The real charger provides much more power with much less noise

Lab measurements of the output from the chargers shows a couple problems with the counterfeit. First, the counterfeit turns out to provide at most 5.9W, not 10W. Second, the output voltage is extremely noisy and full of spikes.

The following voltage-vs-current graphs show the performance of the iPad charger (left) and counterfeit charger (right) under increasing load. The line for the real charger goes much farther to the right, showing that the real charger provides much more current. By my measurements, the real charger provides a maximum of 10.1 watts, while the counterfeit charger provides only 5.9 watts. The consequence is the real charger will charge your iPad almost twice as fast. (For details on these graphs, see my article testing a dozen chargers.) The other thing to note is the line for the Apple charger is smooth and thin, while the counterfeit charger's line is all over the place. This indicates that the power provided by the counterfeit charger is noisy and low quality.

iPad
Voltage vs current graph for iPad charger
Counterfeit
Voltage vs current graph for fake iPad charger

The next pair of graphs shows the power quality. The yellow line shows the voltage. The real charger has a stable yellow thin line, while the counterfeit charger's output has large voltage spikes. (I had to change the scale to get the output to fit on the screen, so the counterfeit charger is actually twice as bad as it appears here.) The bottom of the counterfeit charger's yellow line is wavy, due to 120 Hz ripple appearing in the output voltage.

iPad
High frequency oscilloscope trace from Apple iPad charger
Counterfeit
High frequency oscilloscope trace from counterfeit iPad charger

The orange line shows the frequency spectrum of the output: lower is better, and higher is exponentially worse. The counterfeit spectrum is much higher in general, with a large spike at the switching frequency. This shows that the counterfeit charger's power is worse across the frequency spectrum.

You might wonder if the power quality actually matters. The biggest impact it has is on touchscreen performance. The interference from bad power supplies is known to cause the touchscreen to behave erratically.[16] If your screen malfunctions when plugged into a charger, this is probably the cause.

Inside the real charger's transformer

There's more inside the transformer that you'd expect. This section does a full teardown of the transformer from the genuine charger.

iPad
A copper band surrounds the ferrite core in the flyback transformer from an iPad charger.
iPad
Removing the ferrite core and insulation reveals the double-stranded primary winding.

The first photo above shows that underneath the the first layer of yellow insulating tape, a layer of copper foil is attached to the transformer's ferrite core to ground it. Next, removing the ferrite core and more insulation reveals the double-stranded primary winding. The high-voltage input is fed into this winding.

iPad
After removing the triple-stranded bias winding and insulating tape, the secondary winding of the transformer is visible. Note the triple-insulated wires used for the secondary winding.
iPad
The next layer of insulation contains copper foil.

Underneath the primary winding and more insulating tape is the triple-stranded bias winding, which provides feedback and power to the control IC. (In the photo, this winding has been removed and is surrounding the transformer.) After removing more insulating tape, the secondary winding of the transformer is visible. As discussed in the safety section, the secondary winding has triple-insulated wires and extra insulation where the wires leave the transformer. The next layer of insulation (right) contains copper foil. This helps reduce interference.

iPad
The innermost layer of the iPad charger flyback transformer is the primary winding.

Finally, the innermost layer of the iPad charger flyback transformer is the second half of the primary winding (above). Splitting the primary winding into two layers is more expensive, but results in a better transformer due to better coupling of the magnetic fields.

In comparison, the transformer of the counterfeit charger is much lower quality. (I haven't included the pictures for reasons of space; click through to see them.) It simply has the bias winding (pic), secondary winding (pic), and primary winding (pic) , separated by insulating tape. Unlike the genuine transformer, the counterfeit saves cost by omitting the copper foil layers. The counterfeit also doesn't use the more expensive split, multi-stranded windings that the genuine charger uses. As discussed earlier, the secondary winding is plain copper wire, not triple-insulated wire, which is a significant safety flaw.

How does the iPad charger compare to the iPhone charger?

The iPad charger is considerably larger than the iPhone charger and provides twice the power. In my detailed iPhone charger teardown I looked at the internals of the iPhone charger. The iPhone charger (below) uses two circuit boards that combine to form a one inch cube, which is impressive engineering. The iPhone and iPad chargers are both flyback switching power supplies, but the feedback mechanisms are very different.[17] Overall, I like the iPhone charger more than the iPad charger from a design standpoint, mainly because of the harder engineering challenge of cramming everything into a much smaller space.

iPhone
Inside the iPhone charger: input inductor (green), Y capacitor (blue), flyback transformer (yellow), USB connector (silver). The primary circuit board is on top and the secondary board on the bottom.

Schematics

In my iPhone charger teardown, I drew up a schematic of the charger, but for the iPad chargers I didn't need to do this. The genuine iPad charger is almost identical[18] to the reference design schematic provided by iWatt. The counterfeit charger is almost identical to the schematic in the DB02A controller datasheet. You can see from the schematics that the genuine charger has a much more complex circuit than the counterfeit. (Click the thumbnails below to get to the datasheets.)

iPad
Thumbnail: Click for schematic of iPad charger based on iWatt 1691 controller.
Counterfeit
Thumbnail: Click for schematic of counterfeit iPad charger based on DB02A controller.

Is the Apple charger worth the price?

Apple's charger is expensive compared to other chargers, but is a high quality product. You should definitely stay away from the cheap counterfeit chargers, as they are low quality and dangerous. Non-Apple name brand chargers are generally good quality according to my tests, with some better than Apple. If you want to get an Apple charger without the high price, the best way I've found is to buy a used one on eBay from a US source. I've bought several for testing, and they have always been genuine.

I wrote earlier about Apple's huge profit margins on chargers. Apple has since dropped their charger prices from $29 to $19, which is more reasonable, but looking at the price of similar chargers from other manufacturers and the cost of components, I think Apple has a huge profit margin even at $19.[19]

In any case, the iPad charger is an impressive piece of engineering with a lot of interesting circuitry inside. The counterfeit charger is also impressive in its own way - it's amazing that a charger can be manufactured and sold for such a low price (if you don't care about safety and quality). Overall, you mostly get what you pay for; even if you can't tell from the outside, there are big differences inside the case.

Notes and references

[1] Foxlink (Taiwan), Foxconn (Taiwan), and Flextronics (Singapore) are all manufacturers for Apple with confusingly similar names. Foxconn is the company with controversy over employee treatment; this charger is made by Foxlink, a different company. Interestingly, the chairmen of both companies are brothers and the companies do a lot of business with each other. The companies state that they are entirely independent, though (statement, Foxlink annual report). Foxconn and Flextronics are the world's #1 and #3 largest electronics manufacturing companies according to the MMI top 50 for 2013, while Foxlink is smaller.

[2] The chargers uses a flyback design, where the transformer operates "backwards" from how you might expect. When a voltage pulse is sent into the transformer, the output diode blocks the output so there is no output - instead a magnetic field builds up in the transformer. The transformer core has a tiny air gap to help store this field. When the voltage input stops, the magnetic field collapses, transferring power to the output winding. Flyback power supplies are very common for low-wattage power supplies.

[3] You might wonder why the DC voltage inside the power supply is so much higher than the line voltage. The DC voltage is approximately sqrt(2) times the AC voltage, since the diode charges the capacitor to the peak of the AC signal. Thus, the input of 100 to 240 volts AC is converted to a DC voltage of 145 to 345 volts internally. This isn't enough to be officially high voltage but I'll call it high voltage for convenience. According to standards, anything under 50 volts AC or 120 V dc is considered extra-low voltage and is considered safe under normal conditions. But I'll refer to the 5V output as low voltage for convenience.

[4] The counterfeit charger uses a DB02A controller IC. This controller only has four pins and is in a TO-94 (SIP-4) package. (According to the official JEDEC standard, TO-94 is a bolt-like package for large SCRs. It's a puzzle why some companies use TO-94 to describe 4-pin inline packages.) According to the datasheet (Chinese), the chip is for 500mA-1000mA chargers, which explains why the counterfeit charger only produces 5 watts, instead of the 10 watts an iPad charger is supposed to produce. This controller is very inexpensive, available for ¥ 0.35 (about 6 cents).

I couldn't find any US chips similar to this chip, even after a lot of searching; it appears to be a Chinese design with datasheets only in Chinese, manufactured by "Fine Made" Shenzhen Fuman Electronics. Since the chip only has four pins, I expected it to be a trivial Ringing Choke Converter (RCC) circuit with just a couple transistors inside the chip - but I cracked it open with Vise-Grips and it turns out to be a fairly complex chip. I took a picture through a microscope of the IC die, which is about 1 mm across. One interesting feature is the many white pads around the outside of the die, which are used to blow fuses to trim various resistances in the chip. I wasn't expecting to see this level of quality and sophistication. The die has the label "N7113 802" at the right; I don't know what this indicates. Three of the four wires connect in the lower left, and the fourth in the lower right.

Die photo of the DB02A SMPS controller chip.

Die photo of the DB02A SMPS controller chip used in the counterfeit charger.

[5] When a diode or transistor switches, it creates a voltage spike, which can be controlled by a special snubber or clamp circuit. For a lot of information on snubbers and clamps, see Passive Lossless Snubbers for High Frequency PWM Conversion and Switchmode Power Supply Reference Manual.

[6] In the counterfeit charger, the switching transistor is a ALJ 13003 NPN power transistor (datasheet), apparently made by Shenzhen LongJing Microelectronics Co. This transistor is a version of Motorola's MJE 13003 switchmode transistor which was introduced in 1976 (MJE indicates power device in a plastic package). The bridge rectifier is a B6M (datasheet). The output diode is a SR260 Schottky barrier rectifier.

[7] The iPad charger uses special Y-capacitors to bridge the high-voltage and low-voltage sides of the charger. This capacitor helps reduce EMI interference, and is specially designed to avoid any safety hazard. It does, however, pass a tiny amount of electricity - if you feel a tingle from your charger, these capacitors are probably the cause. For more information on X and Y capacitors, see Kemet's presentation and Designing low leakage current power supplies.

[8] The iPad has two spark gaps next to inductor L1 (the input AC common mode choke). I couldn't find a lot of information on this sort of spark gap, but one example of it is an Infineon SMPS design, where similar spark gaps are designed to discharge accumulated charge for a 3KV lightning surge test.

[9] The Apple charger includes a "latch release circuit". If there is a fault, the control IC will shut down the charger until power is removed. However, after unplugging a charger, the input capacitors may store power for many seconds. (You may have seen LEDs remain illuminated for several seconds after unplugging devices.) The latch release circuit ensures that the charger will reset properly even if you plug it back in quickly. It does this by providing a separate diode bridge for the charger's power - this circuit has a much smaller capacitor, so it will power off quickly. (See the schematic for details.) This seems like over-engineering to me, adding extra circuitry for this rare case.

In normal use, by the way, the control IC is powered by the transformer's feedback winding. But if the control IC isn't running, the transformer won't work, leading to a chicken-and-egg situation. The solution is a startup power path where the control IC gets enough power from the AC input to start up, and then switches to the transformer.

[10] The genuine charger uses a complex control chip manufactured by iWatt, the 1691. This chip monitors the input line voltage, the current through the transformer, and the voltage feedback from the transformer. It controls the switching frequency and length of time the power is switched on, with different behavior under no load, low load, and high load, as well as constant monitoring for faults. A detailed presentation on the iW1691 is here. This chip sells for about 30 cents, but I expect Apple gets a better price.

[11] The real charger has a metal ground pin that connects to the power plug, while the counterfeit has a plastic pin. This is one difference between the chargers that is visible externally if you slide the power plug off the charger. Ironically, the US plug doesn't use the ground connection, so this is one safety issue that doesn't make any difference in practice.

[12] Apple uses a proprietary technique for the charger to indicate to the device what kind of charger it is. Different types of Apple chargers use resistances to put different voltages on the USB D+ and D- pins. For details on USB charging protocols, see my earlier references.

[13] While it would be nice to find superconductors inside the charger, unfortunately the zero-ohm resistor is a bit more than 0 ohms. While this resistor may seem pointless, it allows the manufacturers to substitute a resistor later if different transistors require it.

[14] The outside of the charger has the slightly mysterious text: "For use with information technology equipment". This indicates that the charger is covered by the safety standard UL 60950-1, which specifies the various isolation distances required. For a brief overview of isolation distances, see i-Spec Circuit Separation and some of my earlier references.

[15] Only a few special components can safely bridge the gap between the high voltage side of the charger and the low voltage side. The most obvious is the transformer. Y-capacitors can also bridge the primary and secondary side because they are designed not to pass dangerous currents, and not to short out if they fail. Optoisolators use a light signal to provide feedback between the circuits in an iPhone charger, but are not used in the iPad charger.

[16] For an explanation of why the noisy output from cheap chargers messes up touchscreens, see Noise Wars: Projected Capacitance Strikes Back. The article discusses how capacitive touchscreen ICs need to sense pico-Coulombs of charge, which is very difficult when AC noise is present. The article blames touchscreen problems on aftermarket low cost chargers.

[17] The biggest difference between the iPhone charger and the iPad charger is the feedback used to regulate the voltage. The iPhone charger measures the output voltage with a TL431 chip and sends a feedback signal to the control IC via an optoisolator. The iPad charger avoids these components by using primary-side regulation. Instead of measuring the actual output voltage, the iPad control IC looks at the voltage in the feedback winding, which should approximately match the output voltage.

[18] I noticed only a few significant differences between the iPad charger and iWatt's published 1691 charger reference design. This probably means iWatt did most of the design work for Apple.

Comparing the actual charger with the reference design shows a few filtering improvements. The charger has RC snubbers the input bridge rectifier (a rare feature also in the iPhone charger). The charger has an extra diode on the secondary for filtering, as well as a (zener?) diode in the switching transistor drive circuit. The iPad charger uses two Y-capacitors instead of one, and a R/C filter attached to the Y-capacitor on the secondary side. The charger connects line ground to secondary ground through a resistor. The reference design doesn't show the USB data resistors[12].

[19] Some people think that I'm ignoring Apple's cost of designing chargers when figuring their large profit margin. First, if you spend $2 million on design and manufacture 200 million chargers, then design adds only one cent to the cost per charger. Second, iWatt's designers deserve credit for the complex control chip and the reference design, which is most of the design work.

[20] For those interested in the components, the iPad charger's primary diodes (F6w) are 1.5A 60V Schottky Barrier Diodes (datasheet). The "T3" diodes are fast switching diodes (datasheet). The switching transistor is an Infineon SPA04N60C Cool MOS® 650V power transistor (datasheet). The bridge rectifier is a bridge: MB10S CD 0.5A bridge rectifier with high surge capacity (datasheet). The component in the protection circuit that looks like a transistor is a BAV70 dual high-speed switching diode (datasheet). The output diode is a SBR10U45SP5 10A super barrier rectifier (datasheet). The Y capacitors are 220pF 250V. The input capacitors are Samxon 10µFand 4.7µF 400v electrolytics. The output capacitors are a Koshin KLH 820µF 6.3V aluminum electrolytic, and a 820 µF 6.3V X-CON ULR aluminum polymer capacitor (which is more expensive than a regular electrolytic, but filters better because of its lower ESR).

The programming error that cost Mt Gox 2609 bitcoins

In October 2011, Mt Gox lost over 2609 bitcoins by accidentally creating transactions that can never be redeemed. (At the time, 2609 bitcoins were worth about $8000, but they would be worth almost $1.5 million dollars now.) The problem was discussed on the Bitcoin forum, but I think it's worth taking a closer look at it in light of Mt Gox's recent massive bitcoin loss and bankruptcy. This problem reveals how easy it is to lose bitcoins through software errors. It also reveals that Mt Gox has made careless and expensive programming errors before.

The problem

When you send bitcoins to an address, what happens inside the Bitcoin transaction is more complicated than you would expect. The transaction contains a tiny computer program in the Bitcoin Script language, and this program is executed to determine if the bitcoins can be spent. Normally the program requires a public key and signature in order to spend the bitcoins. The program checks that the public key matches the address you sent the bitcoins to, and the signature is valid, proving the spender of the bitcoins has the private key. If everything matches, the bitcoins can be spent.

You might wonder why Bitcoin uses such a complex system to validate transactions. The idea is that by providing a programming language, Bitcoin allows many different types of transactions, such as escrow transactions or more complex contracts.

Going into a bit more detail, a typical scriptPubKey program looks like:

OP_DUP
OP_HASH160
f2e63314c350094550c703fcdcd4850ad37d8310
OP_EQUALVERIFY
OP_CHECKSIG

This program is part of a transaction that sent bitcoins to the address 1P9LHy6K2c9cwbfSfdaaoYVAprqUYtcFnB (which in hex is the f2e633... value above). In order for that address to redeem the bitcoins, they must provide the public key for the address 1P9L... and provide the signature for the spending transaction (which proves they have the private key). Walking through the execution of the program, it first duplicates the provided public key, computes the 160-bit hash of it, and verifies that it is equal to the provided public key address. Then it checks the signature for validity. If all goes well, the bitcoins can be spent. If there is a problem, the transaction is rejected.

In Mt Gox's bad transactions, they made a small but costly error. The script is:

OP_DUP
OP_HASH160
0
OP_EQUALVERIFY
OP_CHECKSIG
Note that in place of the destination address hash, this transaction has the byte 0, representing OP_0, which pushes an empty array of bytes. Since it's impossible for the 160-bit hash to match an empty array, it's impossible for this script to complete successfully, and the bitcoins can never be spent. (For more information on how these scripts work, see my article Bitcoins the hard way).

Why does Bitcoin permit broken transactions?

You might wonder why Bitcoin permits transactions that can never be spent. Unfortunately, it would be very difficult to determine if a script can be satisfied or not. Bitcoin is designed to give people a lot of flexibility with the Script language, even though it's easy to shoot yourself in the foot. In addition, a complex algorithm to reject transactions would be very dangerous - if clients and miners disagree on the validity of a transaction, a blockchain fork will result, causing chaos.

(The computer scientists are probably yelling "halting problem" at this point. However, that doesn't apply because the Bitcoin Script language doesn't have loops so it's guaranteed to halt. A non-terminating script would be a disaster since miners and clients would lock up, so the language prohibits looping.)

In addition, it's common to lose bitcoins by using an address when you don't have the private key, and there's no way software could detect that. Strangely, people often do this on purpose. Many transactions deliberately send tiny amounts of bitcoins to bad addresses to hide text and images in the blockchain. Other people have deliberately lost over a million dollars in bitcoins in a Counterparty proof-of-burn.

Gold coins from the Fishpool Hoard. Unlike lost bitcoins, lost gold coins can sometimes be recovered.

Another interesting script bug

While analyzing coinbase transactions, I came across another interesting bug that lost bitcoins. Some transactions have the meaningless and unredeemable script:
OP_IFDUP
OP_IF
OP_2SWAP
OP_VERIFY
OP_2OVER
OP_DEPTH

That script turns out to be the ASCII text script. Instead of putting the redemption script into the transaction, the P2Pool miners accidentally put in the literal word "script". The associated bitcoins are lost forever due to this error.

Conclusion

Losing bitcoins due to programming errors is very easy. Mt Gox has lost thousands of bitcoins in the past this way, as have others. I don't know what happened to Mt Gox recently (although I know it wasn't OP_PUSHDATA2 malleability), but based on history it is worth keeping programming errors in mind.

If you want some scary reading describing other major Bitcoin losses, take a look at List of Major Bitcoin Heists, Thefts, Hacks, Scams, and Losses.

Bitcoin mining the hard way: the algorithms, protocols, and bytes

This article explains Bitcoin mining in details, right down to the hex data and network traffic. If you've ever wondered what really happens in Bitcoin mining, you've come to the right place. My previous article, Bitcoins the hard way described how I manually created a Bitcoin transaction and sent it into the system. In this article, I show what happens next: how a transaction gets mined into a block.

The purpose of mining

Bitcoin mining is often thought of as the way to create new bitcoins. But that's really just a secondary purpose. The primary importance of mining is to ensure that all participants have a consistent view of the Bitcoin data. Because Bitcoin is a distributed peer-to-peer system, there is no central database that keeps track of who owns bitcoins. Instead, the log of all transactions is distributed across the network.

The main problem with a distributed transaction log is how to avoid inconsistencies that could allow someone to spend the same bitcoins twice. The solution in Bitcoin is to mine the outstanding transactions into a block of transactions approximately every 10 minutes, which makes them official. Conflicting or invalid transactions aren't allowed into a block, so the double spend problem is avoided.

Although mining transactions into blocks avoid double-spending, it raises new problems: What stops people from randomly mining blocks? How do you decide who gets to mine a block? How does the network agree on which blocks are valid? Solving those problems is the key innovation of Bitcoin: mining is made very, very difficult, a technique called proof-of-work. It takes an insanely huge amount of computational effort to mine a block, but it is easy for peers on the network to verify that a block has been successfully mined.[1]

Each mined block references the previous block, forming an unbroken chain back to the first Bitcoin block. This blockchain ensures that everyone agrees on the transaction record. It also ensures that nobody can tamper with blocks in the chain since re-mining all the following blocks would be computationally infeasible.[2] As long as nobody has more than half the computational resources, mining remains competitive and nobody can control the blockchain.

As a side-effect, mining adds new bitcoins to the system. For each block mined, miners currently get 25 new bitcoins (currently worth about $15,000), which encourages miners to do the hard work of mining blocks. With the possibility of receiving $15,000 every 10 minutes, there is a lot of money in mining.

How mining works

Mining requires a task that is very difficult to perform, but easy to verify. Bitcoin mining uses cryptography, with a hash function called double SHA-256. A hash takes a chunk of data as input and shrinks it down into a smaller hash value (in this case 256 bits). With a cryptographic hash, there's no way to get a hash value you want without trying a whole lot of inputs. But once you find an input that gives the value you want, it's easy for anyone to verify the hash. Thus, cryptographic hashing becomes a good way to implement the Bitcoin "proof-of-work".

In more detail, to mine a block, you first collect the new transactions into a block. Then you hash the block to form a 256-bit block hash value. If the hash starts with enough zeros[3], the block has been successfully mined and is sent into the Bitcoin network and the hash becomes the identifier for the block. Most of the time the hash isn't successful, so you modify the block slightly and try again, over and over billions of times. About every 10 minutes someone will successfully mine a block, and the process starts over.

The diagram below shows the structure of a specific block, and how it is hashed. The yellow part is the block header, and it is followed by the transactions that go into the block. The first transaction is the special coinbase transaction that grants the mining reward to the miner. The remaining transactions are standard Bitcoin transactions moving bitcoins around. If the hash of the header starts with enough zeros[3], the block is successfully mined. For the block below, the hash is successful: 0000000000000000e067a478024addfecdc93628978aa52d91fabd4292982a50 and the block became block #286819 in the blockchain.

Structure of a Bitcoin block

Structure of a Bitcoin block

The block header contains a handful of fields that describe the block. The first field in the block is the protocol version. It is followed by the hash of the previous block in the blockchain, which ensures all the blocks form an unbroken sequence in the blockchain. (Inconveniently, the hash is reversed in the header.) The next field is the Merkle root,[4] a special hash of all the transactions in the block. This is also a key part of Bitcoin security, since it ensures that transactions cannot be changed once they are part of a block.[5] Next is a (moderately accurate) timestamp of the block, followed by the mining difficulty value bits.[3] Finally, the nonce is an arbitrary value that is incremented on each hash attempt to provide a new hash value. The tricky part of mining is finding a nonce that works.

Untitled

ASIC Bitcoin Miner
Photo by Mirko Tobias Schaefer, (CC BY 2.0)

A short program to mine a block

I wrote a Python program that mines the above block. The program itself is pretty simple - the hardest part of the code is computing the difficulty target from bits.[3] Otherwise it's just a loop over different nonce values. Each iteration puts the data into a structure, hashes it, and tests the result.

The following table shows the hash obtained for selected nonce values. The key point is that each nonce generates a basically-random hash value. Every so often a "lucky" nonce will generate a hash starting with some zeroes. To get a lot of zeroes, you need to try an exponentially large number of nonces. For this block, the "winning" nonce is 856192328.

noncehash
05c56c2883435b38aeba0e69fb2e0e3db3b22448d3e17b903d774dd5650796f76
128902a23a194dee94141d1b70102accd85fc2c1ead0901ba0e41ade90d38a08e
2729577af82250aaf9e44f70a72814cf56c16d430a878bf52fdaceeb7b4bd37f4
38491452381016cf80562ff489e492e00331de3553178c73c5169574000f1ed1c
3903fd5ff1048668cd3cde4f3fb5bde1ff306d26a4630f420c78df1e504e24f3c7
9900001e3a4583f4c6d81251e8d9901dbe0df74d7144300d7c03cab15eca04bd4bb
521170000642411733cd63264d3bedc046a5364ff3c77d2b37ca298ad8f1b5a9f05ba
181315200000c94a85b5c06c9b06ace1ba7c7f759e795715f399c9c1b1b7f5d387a319f
19745650000000cdccf49f13f5c3f14a2c12a56ae60e900c5e65bfe1cc24f038f0668a6c
2439898010000000ce99e2a00633ca958a16e17f30085a54f04667a5492db49bcae15d190
8561923280000000000000000e067a478024addfecdc93628978aa52d91fabd4292982a50

I should point out that I cheated by starting with a block that could be successfully mined. Most of the attempts to mine a block will fail entirely - none of the nonce values will succeed. In that case, you need to modify the block slightly and try again. The timestamp can be adjusted (which is why the timestamp in mined blocks is often wrong). New transactions can be added to the block, changing the Merkle hash. The coinbase transaction can be modified - this turns out to be very important for mining pools. Any of these changes will result in totally different hashes, so the nonce values can be tried again.

My Python program does about 42,000 hashes per second, which is a million times slower than the hardware used by real miners. My program would take about 11 million years on average to mine a block from scratch.

Mining is very hard

The difficulty of mining a block is astounding. At the current difficulty, the chance of a hash succeeding is a bit less than one in 1019. Finding a successful hash is harder than finding a particular grain of sand from all the grains of sand on Earth. To find a hash every ten minutes, the Bitcoin hash rate needs to be insanely large. Currently, the miners on the Bitcoin network are doing about 25 million gigahashes per second. That is, every second about 25,000,000,000,000,000 blocks gets hashed. I estimate (very roughly) that the total hardware used for Bitcoin mining cost tens of millions of dollars and uses as much power as the country of Cambodia.[6]

Note that finding a successful hash is an entirely arbitrary task that doesn't accomplish anything useful in itself. The only purpose of finding a small hash is to make mining difficult, which is fundamental to Bitcoin security. It seems to me that the effort put into Bitcoin mining has gone off the rails recently.

Mining is funded mostly by the 25 bitcoin reward per block, and slightly by the transaction fees (about 0.1 bitcoin per block). Since the mining reward currently works out to about $15,000 per block, that pays for a lot of hardware. Per transaction, miners are getting about $34 in mining reward and $0.10 in fees (stats).

FPGA Bitcoin mining setup with 41 Icarus

15 GH/s FPGA Bitcoin mining configuration with 41 Icarus. Photo by permission of Xiangfu Liu

Mining with a pool

Because mining is so difficult, it is typically done in mining pools, where a bunch of miners share the work and share the rewards. If you mine by yourself, you might successfully mine a block and get 25 bitcoin every few years. By mining as part of a pool, you could get a fraction of a bitcoin every day instead, which for most people is preferable.

Mining pools use an interesting technique to see how much work miners are doing. They send out a block to be mined, and get updates from a miner whenever a miner gets a partial solution. Each partial solution proves the miner is working hard on the problem and gives the miner a share in the final reward when someone succeeds in mining the block.

For instance, if Bitcoin mining requires a hash starting with 15 zeroes, the mining pool can ask for hashes starting with 10 zeroes, which is a million times easier. Depending on the power of their hardware, a miner might find such a solution every few seconds or a few times an hour. Eventually one of these solutions will start with not just 10 zeroes but 15 zeroes, successfully mining the block and winning the reward for the pool.[7] The reward is then split based on each miner's count of shares as a fraction of the total, and the pool operator takes a small percentage for overhead.[8]

Most of the time someone outside the pool will mine a block first. In that case, the pool operator sends out new data and the miners just start mining the new block. People in a pool can get edgy if a long time goes without a payout because of bad luck in mining.

Stratum: The communication between a pool and the miners

Next I'll look in detail at the communication between a miner and the mining pool. The communication between the pool and the miners is interesting. The pool must efficiently provide work to the miners and collect their results quickly. The pool must make sure miners aren't duplicating work. And the pool must make sure miners don't waste time working on a block that has already been mined.

An important issue for mining pools is how to support fast miners. The nonce field in the header is too small for fast miners since they will run through all the possible values faster than the pool can send blocks. The solution is to allow miners to update the coinbase transaction so they can put additional nonces there. This makes mining more complicated since after building the coinbase transaction the miner must recompute the Merkle hash tree and then try mining the block.

I'm going to look at the Stratum mining pool protocol that is used by many pools. (Some alternative protocols are the Getwork and Getblocktemplate protocols.) The following Python program uses the Stratum protocol to make a mining request to the GHash.IO mining pool and displays the results. (This program is a minimal demonstration; don't use this code for real mining.)

The information below is what the mining pool sends back over the network in response to the program above. Since the Stratum protocol uses JSON-RPC the results are readable ASCII rather than the binary packets used by most of Bitcoin. This provides all the data needed to start mining as part of the pool:
{"id":1,"result":[[["mining.set_difficulty","b4b6693b72a50c7116db18d6497cac52"],["mining.notify","ae6812eb4cd7735a302a8a9dd95cf71f"]],"4bc6af58",4],"error":null}

{"id":null,"params":[16],"method":"mining.set_difficulty"}

{"id":null,"params":["58af8d8c","975b9717f7d18ec1f2ad55e2559b5997b8da0e3317c803780000000100000000","01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4803636004062f503253482f04428b055308","2e522cfabe6d6da0bd01f57abe963d25879583eea5ea6f08f83e3327eba9806b14119718cbb1cf04000000000000000000000001fb673495000000001976a91480ad90d403581fa3bf46086a91b2d9d4125db6c188ac00000000",["ea9da84d55ebf07f47def6b9b35ab30fc18b6e980fc618f262724388f2e9c591","f8578e6b5900de614aabe563c9622a8f514e11d368caa78890ac2ed615a2300c","1632f2b53febb0a999784c4feb1655144793c4e662226aff64b71c6837430791","ad4328979dba3e30f11c2d94445731f461a25842523fcbfa53cd42b585e63fcd","a904a9a41d1c8f9e860ba2b07ba13187b41aa7246f341489a730c6dc6fb42701","dd7e026ac1fff0feac6bed6872b6964f5ea00bd8913a956e6b2eb7e22363dc5c","2c3b18d8edff29c013394c28888c6b50ed8733760a3d4d9082c3f1f5a43afa64"],"00000002","19015f53","53058b41",false],"method":"mining.notify"
{"id":2,"result":true,"error":null}
{"id":null,"params":[16],"method":"mining.set_difficulty"}
The first line is a response from the pool server with the subscription details. The first values are not too important. The value 4bc6af58 is the value extranonce1 that is used when building the block. Each client gets a unique value to ensure that all the mining clients generate unique blocks and don't duplicate work. The following value (4 bytes) is the length of the extranonce2_size value that the miner puts in the coinbase while mining.

The second line is a mining.set_difficulty message to our client. With a difficulty of 16, I can get a share every hour or two on my PC. In comparison, the Bitcoin mining difficulty is 3,129,573,174.52[3] - thus it's about 200 million times easier to get a share in this pool than to successfully mine a block independently. That's why people join pools.

The third line is a mining.notify notification to our client. This message defines that block for us to mine. There's a lot of data returned under "params", so I'll explain it field by field.

job_id58af8d8c
prevhash975b9717f7d18ec1f2ad55e2559b5997b8da0e3317c803780000000100000000
coinb10100000001000000000000000000000000000000000000000000000000000000
0000000000ffffffff4803636004062f503253482f04428b055308
coinb22e522cfabe6d6da0bd01f57abe963d25879583eea5ea6f08f83e3327eba9806b
14119718cbb1cf04000000000000000000000001fb673495000000001976a914
80ad90d403581fa3bf46086a91b2d9d4125db6c188ac00000000
merkle_branch["ea9da84d55ebf07f47def6b9b35ab30fc18b6e980fc618f262724388f2e9c591", ...]
version00000002
nbits19015f53
ntime53058b41
clean_jobsfalse

The job_id is used to identify this mining task if the miner reports back success.

Most of the fields are used in the block header. The prevhash is the hash of the previous block. Apparently mixing big-ending and little-endian isn't confusing enough so this hash value also has every block of 4 bytes reversed. The version is the block protocol version. The nbits indicates the difficulty[3] of the block. The timestamp ntime is not necessarily accurate.

The coinb1 and coinb2 fields allow the miner to build the coinbase transaction for the block. This transaction is formed by concatenating coinb1, the extranonce1 value obtained at the start, the extranonce2 that the miner has generated, and coinb2. The result is a transaction in Bitcoin protocol. The merkle_branch hash list lets the miner efficiently recompute the Merkle hash with the new coinbase transaction.

clean_jobs is used if the miner needs to restart the mining jobs.

After receiving this data, the miner can start generating coinbase transactions and mining blocks.

Butterfly Labs Jalapeño

Butterfly Labs Jalapeño ASIC miner, 7+ GH/s, by 0xF2, (CC BY-ND 2.0)

Creating a block for a pool

Once the miner has received the information from the pool, it is straightforward to form the coinbase transaction by joining the coinb1, extranonce1, extranonce2, and coinb2 to form a coinbase transaction. The diagram below shows how the combination of these four values forms a complete transaction, with the nonces in the middle of the coinbase script. (The block below is slightly different from the one described earlier.)

A coinbase transaction generated by the GHash.io mining pool

A coinbase transaction generated by the GHash.io mining pool

The structure of the coinbase transaction is similar to a regular transaction, but there are a few important differences. A normal transaction transfers bitcoins from inputs (usually source addresses) to outputs (usually destination addresses). A coinbase transaction is generating new bitcoins out of thin air, rather than doing a transfer, so the transaction is slightly different. The previous output hash and index are irrelevant for the coinbase transaction. the first script is the scriptSig which signs the transaction to prove ownership of the incoming bitcoins. In a coinbase transaction, this is irrelevant, so instead the field is called the coinbase and is mostly arbitrary data.[9] (Many miners hide messages in there.) The value field in the coinbase transaction is the 25 bitcoin mining reward plus any bitcoins left over from the other transactions (the left over bitcoins are treated as mining fees). Finally, both regular transactions and the coinbase transaction use the second script (scriptPubKey) to specify the recipients of the bitcoins.[10] For details on transactions, see my my previous article.

Once the coinbase transaction is created, the hash for this coinbase transaction is combined with the merkle_branch data from the pool to generate the Merkle hash[4] for the entire set of transactions. Because of the structure of the Merkle hash (explained below), this allows the hash for the entire set of transactions to be recomputed easily.

Finally, the block header is built from the new Merkle hash and the data provided by the pool, and the hash algorithm can iterate over the nonce values in the header, just like the Python program earlier. Once all the nonce values have been tried, the miner increments the extranonce2, generates a new coinbase transaction and continues.

A Bitcoin block header

A Bitcoin block header

Informing the mining pool of success

The difficulty[3] for a mining pool is set much lower than the Bitcoin mining difficulty (fewer leading zeros required), so it's much easier to get a share. When a block is hashed to the pool's difficulty, you send a simple JSON message to the mining pool to submit it:
{"method": "mining.submit", "params": ["kens.worker1", "58af8db7", "00000000", "53058d7b", "e8832204"], "id":4}
The parameters are the worker name, job id, extranonce2, time, and header nonce. This information is sufficient for the pool to build the matching coinbase transaction and header, and verify the block. If the hash meets the pool difficulty, you get a share. If the hash also meets the much, much harder Bitcoin difficulty, the block has been successfully mined. In this case the pool submits the block to the Bitcoin network and everyone with shares gets paid accordingly.

Mining for fun and profit

If you're curious about mining, it's surprisingly easy to try out mining yourself, although you'll be lucky to earn even a penny. Just create an account at a mining pool such as BTC Guild, download mining software such as cpuminer (minerd.exe), and run the software to start mining. For a pool with low difficulty, you should get shares in a few minutes; in a pool with a higher difficulty (such as GHash.IO), it may take you an hour or two to get a share, which is more frustrating.[3]

Example of Bitcoin mining

Unprofitable Bitcoin CPU mining on my PC

The screenshot above shows what mining looks like as you get shares and blocks get mined. I got lucky and it only took me a minute to successfully mine a share. A minute later someone successfully mined a block, so the pool tells everyone to start over. Another block was mined less than a minute after that - although blocks are 10 minutes apart on average, the times can vary widely. It took 12 minutes for my next share to be generated. After running for a while, I earned 0.00000043 BTC, which is a tiny fraction of a cent.

Bitcoin mining is an "arms race". Originally people could mine with the CPU on a regular PC, but that hasn't been practical for a while. Next mining was offloaded to GPUs. Now, mining is done with special-purpose ASIC hardware, which is rapidly increasing in speed. For-profit mining is very competitive, and you'll need to look elsewhere for information.

If you want to try out mining just for fun, you may prefer to mine a currency such as Dogecoin rather than Bitcoin. First, Dogecoin uses a different hash algorithm which doesn't work well with ASIC hardware, so you're not as disadvantaged compared to professional miners. Second, because dogecoins are worth much less than bitcoins, you'll end up with a much larger number of dogecoins, which seems more rewarding. For Dogecoin mining, I used the dogepool.pw pool somewhat arbitrarily. The process is almost the same as Bitcoin mining, except you use the scrypt algorithm instead of sha256d. There are many other alternative cryptocurrencies to choose from.

Notes and references

[1] Bitcoin mining seems like a NP (nondeterministic polynomial) problem since a solution can be quickly verified. However, there are a couple of issues with making this rigorous. First, since hashes are a fixed size, mining can be done in constant time (but with a very large constant of 2^256). Thus, you'd need to consider an extended mining scheme where the difficulty can go to infinity. Second, mining would need to be turned into a decision problem - e.g. instead of finding a nonce, the problem would be "Is there a successful nonce less than k". (Note that if you can solve that problem, you can rapidly find the nonce with binary search.)

With these changes, the mining problem is in NP. The next question is if it is NP-complete. That is, can an arbitrary NP-complete problem be turned into a mining problem? I believe that is currently unknown.

[2] You might wonder what happens if two miners succeed in mining a block at approximately the same time. Has the problem of conflicting transactions has just been replaced by the problem of conflicting blocks?) The rule is that only the longest chain of valid blocks is used, and the other branch is ignored. Thus, when a miner extends the chain with one of the two parallel blocks, the other block becomes an orphan block and is ignored.

Orphan blocks are fairly common, roughly one a day. For this reason, the (somewhat arbitrary) recommendation is to wait for six confirmations (about one hour) before considering a transaction solidly confirmed.

[3] I've been describing a successful hash as starting with enough zeros, but there's an official definition of difficulty. A valid block must have a hash below a target value. (Since the target starts with a bunch of zeros, so will the valid hash.)

There are two different hard-to-understand ways of representing the target. The first, bits is a mantissa/exponent representation of the target in 32 bits. The second, difficulty is the ratio between a base target and the current target. A difficulty of N is N times as difficult as this base target. The base target is 0x00000000FFFF0000000000000000000000000000000000000000000000000000, which corresponds to approximately 1 in 232 or 1 in 4.2 billion hashes succeeding.

Difficulty changes approximately every two weeks to keep the block hash rate around 1 every 10 minutes. The https://blockchain.info/stats difficulty value is 3,129,573,174.52, corresponding to a target of 00000000000000015f5300000000000000000000000000000000000000000000. Multiplying my PC's performance by the current difficulty shows it would take my PC about 35,000 years to mine a block.

The pool difficulty is important when using a mining pool. My PC can do about 12 million hashes/sec running cpuminer, so at a difficulty of 1 my PC could find a block every 6 minutes. The BTC Guild pool uses a difficulty of 2, so I get a share about every 12 minutes. GHash.IO has a minimum difficulty of 16 on the other hand, so I only get a share every hour or two on the average. (My overall earnings would be similar either way, since the shares per block scale inversely with the difficulty.)

[4] Instead of hashing all the transactions into the block directly, the transactions are first hashed together to yield a Merkle root. The Merkle root is the root of a binary Merkle tree. The idea is to start with all the transaction hashes. Pairs of hashes are hashed together to yield new hashes. The process is repeated on the new list of hashes and continues recursively until a single hash is obtained. This final root hash is the value used when computing the block. (See Wikipedia for more details.)

In the Merkle tree, each transaction is hashed. Then pairs of hashes are hashed together. Then pairs of the new hashes are hashed together, and so on, until a single hash remains. This allows the hash of a single transaction to be verified efficiently without recomputing all the hashes. One place this comes in useful is generating a new coinbase transaction for a mining pool.

The (patented) idea of a Merkle tree is if you need to modify or verify a single transaction, you don't need to recompute everything, but can just recompute the affected pairs. Personally, I think the Merkle tree is a pointless optimization for Bitcoin and for reasonable transaction numbers it would be faster to do a single large hash, rather than multiple hashes up the Merkle tree.

Here's some demonstration code to compute the Merkle root for the block I'm discussing. The 99 transaction hashes are hard-coded for convenience. The resulting Merkle root is 871714dcbae6c8193a2bb9b2a69fe1c0440399f38d94b3a0f1b447275a29978a

[5] There are a few ways that third parties can modify transactions without invalidating the signature on the transaction. This is known as transaction malleability. These modifications change the hash of the transaction. Since the hash is part of the block, a transaction has a fixed hash and cannot be modified by malleability once it has been mined into a block. (Unless the whole block is orphaned, of course.)

[6] It's hard to estimate the cost of mining because the hardware is changing so rapidly and it's unclear what is actually in use, but I'll do a rough calculation. Looking at the Bitcoin mining hardware and Mining hardware comparison pages, the HashBlaster looks like the most efficient currently available at 375 MH/s/$ and 1818 MH/s/W. The Bitcoin network is 25 billion MH/s, which works out to about $70 million hardware cost and 15 MW. (This is about the total power consumption of Cambodia.) At $0.15/kWH, that would be about $50,000/day on electricity ($300 per block or $0.70 per transaction). Since mining generates about $140,000 per day, spending $50,000 per day on electricity seems like the right ballpark. Other estimates are at Hacker News.

[7] You might wonder why a miner doesn't cheat. If they successfully mine a block, why not submit it themselves so they can claim the full mining reward, rather than splitting it? The main reason is the coinbase transaction has the pool's address, not the miner's address. If the miner submits the block bypassing the pool, the reward still goes to the pool. And if the miner changes the address, the hash is no longer valid.

[8] There are several different reward systems used by mining pools. For instance, a pool can pay out the exact amount earned from a block or an average amount. Or a pool can pay a fixed amount per share. A pool can weight shares by time to avoid miners switching between pools mid-block. These different systems can balance risk between the miners and the pool operator and adjust the variance of payments. For details, see the Bitcoin wiki here or here.

[9] I've figured out a lot of the structure of the coinbase script above. First it contains the block height (0x046063 or 286819), which is required for version 2). Next is the string '/P2SH/' which indicates the miner supports Pay To Script Hash). This is followed by a timestamp. Next is 8 bytes of the two nonces. This is followed by apparently-random data and then the text "Happy NY! Yours GHash.IO".

[10] The typical coinbase script format has changed over time. Originally, the output scripts were all pay-to-pubkey, with the script: public_key OP_CHECKSIG. This script puts the public key itself in the script. However, now about 95% of coinbase transactions use the standard pay-to-pubkey-hash script: OP_DUP OP_HASH160 addr OP_EQUALVERIFY OP_CHECKSIG. This script only includes the public key hash (the address) and requires the redeemer to provide the public key. To see the difference, compare the output scripts in this transaction and this transaction.

Hidden surprises in the Bitcoin blockchain and how they are stored: Nelson Mandela, Wikileaks, photos, and Python software

Every Bitcoin transaction is stored in the distributed database known as the Bitcoin blockchain. However, people have found ways to hack the Bitcoin protocol to store more than just transactions. I've searched through the blockchain and found many strange and interesting things - from images to source code in JavaScript, Python, and Basic. If you're running a Bitcoin client, you probably have all this data stored on your system.[1]

Nelson Mandela tribute

The Bitcoin blockchain contains this image of Nelson Mandela and the tribute text. Someone encoded this data into fake addresses in Bitcoin transactions, causing it to be stored in the Bitcoin system.

Image of Nelson Mandela found in the Bitcoin blockchain.

Nelson Mandela (1918-2013)
"I am fundamentally an optimist. Whether that comes from nature or nurture, I cannot say. Part of being optimistic is keeping one’s head pointed toward the sun, one’s feet moving forward. There were many dark moments when my faith in humanity was sorely tested, but I would not and could not give myself up to despair. That way lays defeat and death."
"I learned that courage was not the absence of fear, but the triumph over it. The brave man is not he who does not feel afraid, but he who conquers that fear."
"Difficulties break some men but make others. No axe is sharp enough to cut the soul of a sinner who keeps on trying, one armed with the hope that he will rise even in the end."
"It always seems impossible until it’s done."
"When a man has done what he considers to be his duty to his people and his country, he can rest in peace."
"Real leaders must be ready to sacrifice all for the freedom of their
"Everyone can rise above their circumstances and achieve success if they are dedicated to and passionate about what they do."
"Education is the most powerful weapon which you can use to change the world."
"For to be free is not merely to cast off one’s chains, but to live in a way that respects and enhances the freedom of others."
"There is no passion to be found playing small – in settling for a life that is less than the one you are capable of living."
“There is nothing like returning to a place that remains unchanged to find the ways in which you yourself have altered.” -Nelson Mandela

The data is stored in the blockchain by encoding hex values into the addresses. Below is an excerpt of one of the transactions storing the Mandela information. In this transaction, tiny amounts of bitcoins are being sent to fake addresses such as 15gHNr4TCKmhHDEG31L2XFNvpnEcnPSQvd. This address is stored in the blockchain as hex 334E656C736F6E2D4D616E64656C612E6A70673F. If you convert those hex bytes to Unicode, you get the string 3Nelson-Mandela.jpg?, representing the image filename. Similarly, the following addresses encode the data for the image. Thus, text, images, and other content can be stored in Bitcoin by using the right fake addresses.

Secret message in the first Bitcoin block

It is well known that the Genesis block, the very first block of data in Bitcoin contained a "secret" message. This message was stored in the coinbase[2], a part of a Bitcoin block that is filled in by the miner who mines a Bitcoin block. Along with the standard data, the original transaction also contains the message: 'The Times 03/Jan/2009 Chancellor on brink of second bailout for banks'[3]. Presumably this is a political commentary on Bitcoin compared to the insolvency of "real" banks.

Bitcoin logo

People rapidly figured out how to encode arbitrary content into the Bitcoin blockchain by using hex data in place of Bitcoin addresses.[4] One of the first uses of this technique was to store the Bitcoin logo in the blockchain. I extracted the following image from the blockchain, where it was hidden among normal transactions.[5]

Image found in the Bitcoin blockchain: Bitcoin logo

The Bitcoin logo, hidden in the blockchain.

Prayers from miners

Early on, the miner Eligius started putting Catholic prayers in English and Latin in the coinbase field of blocks they mined. Here are some samples:
Benedictus Sanguis eius pretiosissimus.
Benedictus Iesus in sanctissimo altaris Sacramento.
Ave Maria, gratia plena, Dominus tecum. Benedicta tu in mulieribus, ...
...and life everlasting, through the merits of Jesus Christ, my Lord and Redeemer.
O Heart of Jesus, burning with love for us, inflame our hearts with love for Thee.
Jesus, meek and humble of heart, make my heart like unto thine!
These prayers turned out to be surprisingly controversial, leading to insults being exchanged through the blockchain: "Oh, and god isn't real, sucka. Stop polluting the blockchain with your nonsense.", "FFS Luke-Jr leave the blockchain alone!", and a rickroll in response: "Militant atheists, http://bit.ly/naNhG2 -- happy now?".[6]

The codebase technique has since been used by many other miners as advertising. Typical messages are: Hi from 50BTC.com, For Pierce and Paul, Mined at GIVE-ME-COINS.com, EclipseMC: Aluminum Falcon?, Happy NY! Yours GHash.IO, Mined By ASICMiner, BTC Guild, Made in China, BitMinter, /bitparking, hi from poolserverj, /ozcoin/stratum/, /slush/.[7]

XSS demo

I've found JavaScript code in the blockchain that demonstrates a potential XSS attack. A common security hole on websites is cross-site scripting (XSS)[8], where an attacker can inject hostile JavaScript into a web page viewed by the victim. Surprisingly, such an attack was possible with Bitcoin. The transaction's output script was set to the hex corresponding to:
<script>window.alert("If this were an actual exploit, your mywallet would be empty.")</script>
Apparently some Bitcoin websites would fail to escape the tags, causing the script to run if you viewed the page. The above script just created a harmless dialog box, but a more malicious transaction could potentially steal the user's bitcoins stored on the website.

Len Sassaman Tribute

A tribute to cryptographer Len Sassaman was put in the Bitcoin blockchain a couple weeks after his death by Dan Kaminsky.[9]
---BEGIN TRIBUTE---
#./BitLen
:::::::::::::::::::
:::::::.::.::.:.:::
:.: :.' ' ' ' ' : :
:.:'' ,,xiW,"4x, ''
:  ,dWWWXXXXi,4WX,
' dWWWXXX7"     `X,
 lWWWXX7   __   _ X
:WWWXX7 ,xXX7' "^^X
lWWWX7, _.+,, _.+.,
:WWW7,. `^"-" ,^-'
 WW",X:        X,
 "7^^Xl.    _(_x7'
 l ( :X:       __ _
 `. " XX  ,xxWWWWX7
  )X- "" 4X" .___.
,W X     :Xi  _,,_
WW X      4XiyXWWXd
"" ,,      4XWWWWXX
, R7X,       "^447^
R, "4RXk,      _, ,
TWk  "4RXXi,   X',x
lTWk,  "4RRR7' 4 XH
:lWWWk,  ^"     `4
::TTXWWi,_  Xll :..
=-=-=-=-=-=-=-=-=-=
LEN "rabbi" SASSAMA
     1980-2011
Len was our friend.
A brilliant mind,
a kind soul, and
a devious schemer;
husband to Meredith
brother to Calvin,
son to Jim and
Dana Hartshorn,
coauthor and
cofounder and
Shmoo and so much
more.  We dedicate
this silly hack to
Len, who would have
found it absolutely
hilarious.
--Dan Kaminsky,
Travis Goodspeed
P.S.  My apologies,
BitCoin people.  He
also would have
LOL'd at BitCoin's
new dependency upon
   ASCII BERNANKE
:'::.:::::.:::.::.:
: :.: ' ' ' ' : :':
:.:     _.__    '.:
:   _,^"   "^x,   :
'  x7'        `4,
 XX7            4XX
 XX              XX
 Xl ,xxx,   ,xxx,XX
( ' _,+o, | ,o+,"
 4   "-^' X "^-'" 7
 l,     ( ))     ,X
 :Xx,_ ,xXXXxx,_,XX
  4XXiX'-___-`XXXX'
   4XXi,_   _iXX7'
  , `4XXXXXXXXX^ _,
  Xx,  ""^^^XX7,xX
W,"4WWx,_ _,XxWWX7'
Xwi, "4WW7""4WW7',W
TXXWw, ^7 Xk 47 ,WH
:TXXXWw,_ "), ,wWT:
::TTXXWWW lXl WWT:
----END TRIBUTE----

A creature simulator in Basic

I found a simple character-based simulator in Basic. The idea is 5 creatures wander around the screen eating food blocks and breeding or dying. Unfortunately the code has a bunch of bugs and doesn't work.[10]

The original Bitcoin paper

In this transaction the Bitcoin blockchain contains the PDF for the original Bitcoin paper.

Thumbnail of the original Bitcoin paper.

Thumbnail of the original Bitcoin paper.

Rickrolls

Rickrolling is a popular internet prank, and Bitcoin is not immune. One rickroll was described above as part of the prayer dispute.[6] The lyrics to Never Gonna Give You Up! are found in a second rickroll.[11]

A third rickroll has the song metadata and lyrics encoded in Base-64.[12]

Catagory: Poetry
Title: Never Gonna Give You Up
Performer: Rick Astley
Writer: Mike Stock, Matt Aitken, Pete Waterman
Label: RCA Records
Released: 27, July, 1987

We're no strangers to love
You know the rules and so do I
A full commitment's what I'm thinking of
You wouldn't get this from any other guy
I just wanna tell you how I'm feeling
Gotta make you understand

Never gonna give you up,
Never gonna let you down
Never gonna run around and desert you
...

Photographs in a messaging system

Recently someone has built a message/storage system on top of Bitcoin that allows a growing sequence of messages, text, and images to be stored in the blockchain.[13]

Among other things, this system contains text from the Bhagavad Gita, 1000 digits of pi, multiple JPG and PNG images, a Shel Silverstein poem, a Rumi poem, and quotes from a random party. Here are some of the images stored in the blockchain using this system:

EMBIICompressedLogo.png: Image found in the Bitcoin blockchain. KruseEMBII.jpg: Image found in the Bitcoin blockchain. EhrichWeAreStarStuff.jpg: Image found in the Bitcoin blockchain. DriveHugPuddle.jpg: Image found in the Bitcoin blockchain. ILoveYouMore.jpg: Image found in the Bitcoin blockchain.

Some images found in the Bitcoin blockchain.

Wikileaks cablegate data

A 2.5 megabyte Wikileak files ('cablegate-201012041811.7z') was embedded in the Bitcoin blockchain.[14] The data is followed by a message explaining how to access it.[15]
Wikileaks Cablegate Backup

cablegate-201012041811.7z

Download the following transactions with Satoshi Nakamoto's download tool which
can be found in transaction 6c53cd987119ef797d5adccd76241247988a0a5ef783572a9972e7371c5fb0cc

Free speech and free enterprise! Thank you Satoshi!

5c593b7b71063a01f4128c98e36fb407b00a87454e67b39ad5f8820ebc1b2ad5
221d900b5ac701028f9dfab7dfba326f608308386d45c05432e721b7c122cba7
... 128 lines of transaction ids deleted ...
Downloading the data from the blockchain is inconvenient since the download tool needs to be used on the 130 chunks of 20 KB separately. (It's much easier to download the file from the internet.)

Cablegate data stored in Bitcoin

The blockchain contains the source code for Python tools to insert data into the blockchain and to download it.[16] In a weird self-referential twist, the downloader can be used to download itself. The uploader/downloader puts data into the destination address, but extends the previous technique by using Bitcoin escrow / multi-sig to put three addresses in each destination. It also uses a checksum to make storage more reliable.

Here's the code in the blockchain to insert data into the blockchain. While it says it was written by Satoshi Nakamoto (the pseudonymous author of Bitcoin), that's probably not true.

And here's the code to extract data from the blockchain.
The download tool is slightly buggy - the crc32 has a signed-vs-unsigned problem which suggests it wasn't used extensively.

Leaked firmware key and illegal primes

This transaction has a link about a leaked private key, followed by 1K of hex bytes as text, which supposedly is the private key for some AMI firmware.

The change from that transaction was used for this transaction, which references the Wikipedia page on illegal primes, followed by two supposedly-illegal primes from that page.

The change from that transaction was then used for the Wikileaks Cablegate messages, implying the same person was behind all these messages. It looks like someone was trying to store a variety of dodgy stuff in the Bitcoin blockchain, either to cause trouble or to make some sort of political point.

Email from Satoshi Nakamoto

The following email message allegedly from Bitcoin inventor Satoshi Nakamoto appears in the blockchain.[17] (It's almost certainly not really from him.) It seems to be referring to the removal of some Script opcodes from the Bitcoin server earlier and making the corresponding change to the Electrum server. My guess is this message is someone pointing out a bug fix for Electrum in a joking way.
From a3a61fef43309b9fb23225df7910b03afc5465b9 Mon Sep 17 00:00:00 2001
From: Satoshi Nakamoto <[email protected]>
Date: Mon, 12 Aug 2013 02:28:02 -0200
Subject:[PATCH] Remove (SINGLE|DOUBLE)BYTE

I removed this from Bitcoin in f1e1fb4bdef878c8fc1564fa418d44e7541a7e83
in Sept 7 2010, almost three years ago. Be warned that I have not
actually tested this patch.
---
 backends/bitcoind/deserialize.py |    8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/backends/bitcoind/deserialize.py b/backends/bitcoind/deserialize.py
index 6620583..89b9b1b 100644
--- a/backends/bitcoind/deserialize.py
+++ b/backends/bitcoind/deserialize.py
@@ -280,10 +280,8 @@ opcodes = Enumeration("Opcodes", [
     "OP_WITHIN", "OP_RIPEMD160", "OP_SHA1", "OP_SHA256", "OP_HASH160",
     "OP_HASH256", "OP_CODESEPARATOR", "OP_CHECKSIG", "OP_CHECKSIGVERIFY", "OP_CHECKMULTISIG",
     "OP_CHECKMULTISIGVERIFY",
-    ("OP_SINGLEBYTE_END", 0xF0),
-    ("OP_DOUBLEBYTE_BEGIN", 0xF000),
     "OP_PUBKEY", "OP_PUBKEYHASH",
-    ("OP_INVALIDOPCODE", 0xFFFF),
+    ("OP_INVALIDOPCODE", 0xFF),
 ])


@@ -293,10 +291,6 @@ def script_GetOp(bytes):
         vch = None
         opcode = ord(bytes[i])
         i += 1
-        if opcode >= opcodes.OP_SINGLEBYTE_END and i < len(bytes):
-            opcode <<= 8
-            opcode |= ord(bytes[i])
-            i += 1

         if opcode <= opcodes.OP_PUSHDATA4:
             nSize = opcode
--
1.7.9.4

Text in Bitcoin addresses

Bitcoin addresses are 34 characters long, so it is possible to put something interesting in the text address, although there are limitations.

The first option for putting text into an address is to test millions or billions of private keys by brute force in the hope of randomly getting a few characters you want in the public address. This generates a "vanity" address which is a valid working Bitcoin address. An example is Bitcoin Armory, which uses the donation address 1ArmoryXcfq7TnCSuZa9fQjRYwJ4bkRKfv. Note that only six desirable characters were found, and the rest are random. You can use the vanitygen command-line tool or a website like bitcoinvanity to generate these addresses.

Many people have recently received tiny spam payments from vanity addresses with the prefixes 1Enjoy... and 1Sochi... addresses. These payments don't get confirmed by miners and the purpose of them is puzzling.

The second option is to use whatever ASCII address you want (starting with a 1 and ending with a six-character checksum). Since there is no known private key for this address, any bitcoins sent to this address are lost forever. Despite this, some addresses have received significant amounts: 1BitcoinEaterAddressDontSendf59kuE. has received over 1.6 bitcoins (over $1000). 1111111111111111111114oLvT2 (hex 0) has received almost 3 bitcoins.

A very strange activity is the large-scale deliberate "burning" of bitcoins by sending them to 1CounterpartyXXXXXXXXXXXXXXXUWLpVr, where nobody can ever use them. Amazingly, this address has received over 2,130 bitcoins (about $1.5 million dollars worth) that are now lost forever. The motivation is that Counterparty is issuing their own crypto-currency (XCP) in exchange for destroyed bitcoins. The idea is that "proof-of-burn" is a more fair way of distributing currency than mining.

Mysterious encrypted data in the blockchain

There are many mysterious things in the blockchain that I couldn't figure out, that appear to be encrypted data.

Between June and September 2011, there were thousands of tiny mystery transactions from a few addresses to hundreds of thousands of random addresses sorted in decreasing order. These transactions are for 1 to 45 Satoshis, and have never been redeemed. As far as I can tell, the data is totally random. But maybe there is a secret message in the addresses or in the amounts. In any case, someone went to a lot of work to do this, so there must be some meaning. [20]

One interesting thing is that the change address from the cablegate description was then used for three 86 kilobyte GPG-encoded files.[18] From the "magic numbers" at the beginning of these files I know that these are GPG files encrypted using CAST5, but what is in these files is a mystery. Without the passphrase, they can't be decrypted.

By following the change addresses, we can see that after submitting the "Satoshi" uploader and downloader, the same person submitted the Bitcoin PDF. The same person then submitted five mysterious files.[19] These files appear entirely random, so they may contain encrypted data.

Valentine's day messages

There are a bunch of Valentine's day messages in the blockchain from a couple days ago. I assume someone set up a service to do this.

How to put your own message in the blockchain

It's pretty easy to put your own 20-character message into the blockchain. The following steps explain how.
  1. Take your 20-character string and convert it to hex. E.g. in Python:
    '//righto.com/bc'.encode('hex')
    
  2. Convert the resulting hex string to an address. An easy way is online: https://blockchain.info/q/hashtoaddress/your hex value yields 1AXJnNiDijKUnY9UJZkV5Ggdgh36aWDBYj.
  3. Send bitcoins to that address and your message will show up in the blockchain when your transaction gets mined. Important: those bitcoins will be lost forever, so send a very small amount, like 10 cents. My test message can be seen at the end of blk00113 here.

Summary

People have found a variety of ways to store strange things in the Bitcoin blockchain. I have touched on some of them here, but undoubtedly there are many other hidden treasures.

The notes to this article provides hashes for the interesting transactions, in case anyone wants to investigate further.

ASCII image of Bernanke from the Bitcoin blockchain.

ASCII image of Bernanke from the Bitcoin blockchain.

Notes and references

[1] Clients store the 16-gigabyte blockchain in the data directory. On Windows, this is C:\Users\userid\AppData\Roaming\Bitcoin. The blocks are stored in a sequence of 128 megabyte files blknnnnnn.dat. Syncing these files is why a full Bitcoin client takes hours to start up.

An easy way to see the ASCII contents of the blockchain is to visit bitcoinstrings.com.

[2] In the Bitcoin protocol, every mined block has a transaction that creates new bitcoins. Part of that transaction is an arbitrary coinbase field of up to 100 bytes in the Script language. Normally the coinbase field has data such as the block number, timestamp, difficulty, and an arbitrary nonce number.

The full coinbase in the genesis block is:

PUSHDATA: 04
bits value (mining difficulty): FFFF001D
PUSHDATA: 01
nonce value: 04
PUSHDATA: 45
'The Times 03/Jan/2009 Chancellor on brink of second bailout for banks': 5468652054696D65732030332F4A616E2F32303039204368616E63656C6C6F72206F6E206272696E6B206F66207365636F6E64206261696C6F757420666F722062616E6B73

[3] The message in the Genesis block is slightly different from the actual newspaper article: Chancellor Alistair Darling on brink of second bailout for banks.

[4] A brief overview of Bitcoin addresses will make this technique easier to understand. Normally, you start with a random 256-bit private key, which is necessary to redeem Bitcoins. From this, you generate a public key, which is hashed to a 160-bit address. This address is displayed in ASCII using a technique called Base58Check encoding. This ASCII address, such as 1LLLfmFp8yQ3fsDn7zKVBHMmnMVvbYaAE6, is the address used for transferring Bitcoins. But inside the transaction, the address is stored as the 160-bit (20 byte) hex value.

In normal use, you have no control over the 20-byte hex value used as an address. The trick for storing data in the transaction is to replace the address with 20 bytes of data that you want to store. For instance, the string This is my test data turns into the hex data '54686973206973206d7920746573742064617461'. If you send some bitcoins to that address, the bitcoins are lost forever (since you don't have the private key matching that address), but your message is now recorded in the Bitcoin blockchain.

See my earlier article for details on how Bitcoin addresses are generated.

[5] The Bitcoin logo was hidden in two transactions: ceb1a7fb57ef8b75ac59b56dd859d5cb3ab5c31168aa55eb3819cd5ddbd3d806 and 9173744691ac25f3cd94f35d4fc0e0a2b9d1ab17b4fe562acc07660552f95518.

If you look at the first ScriptPubKey of the first transaction, the address is 3d79626567696e206c696e653d3132382073697a, which turns into the ASCII text =ybegin line=128 siz. If you do this for all the addresses, you get an ecoded file. This file turns out to be encoded in the obscure yEnc encoding, designed in 2001 for transmitting binaries on Usenet. I hacked together some code to extract and decode the file, resulting in the bitcoin.jpg file shown above. There was some discussion of this logo in 2011, but I don't know if anyone has actually extracted the image until now.

[6] The prayers can be found in blk00003 and blk00004. Eligius is appropriately named after Saint Eligius the patron saint of goldsmiths and coin collectors. The Rickroll is here.

[7] For a while, the mysterious message /P2SH/ appeared in the coinbase field over and over. This string is an indication that the miner supports the pay-to-script-hash Bitcoin feature. The purpose of this was to ensure that more than 50% of the miners supported the feature before it was rolled out.

[8] The XSS attack demo is in transaction 59bd7b2cff5da929581fc9fef31a2fba14508f1477e366befb1eb42a8810a000. The JavaScript for the attack was put in the transaction's output script. The blockchain.info website displays the contents of the output script, but apparently didn't escape it as HTML. Thus, the contents <script> would not be displayed as text, but would be executed as part of the page. The demo only popped up an alert box, rather than running malicious JavaScript. The creator of the attack describes it on Reddit.

[9] A talk presents some details on the tribute (here). The data is in transaction 930a2114cdaa86e1fac46d15c74e81c09eee1d4150ff9d48e76cb0697d8e1d72. This tribute cost 1 BTC, 0.01 BTC per line.

[10] The Basic code is in block 3a1c1cc760bffad4041cbfde56fbb5e29ea58fda416e9f4c4615becd65576fe7, and it is stored in "uploader" format, with a donation to Satoshi's genesis block address 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa.

Unfortunately the code is a mess with GOSUBs without RETURNs, broken loops, half-implemented ideas, and unused variables, so the code doesn't work, which is disappointing. It's a mystery why someone would put this BASIC code into the blockchain.

[11] The Rick Astley lyrics are in transaction d29c9c0e8e4d2a9790922af73f0b8d51f0bd4bb19940d9cf910ead8fbe85bc9b. This data is included using the OP_RETURN technique, which was later supported as a non-hacky way to put data into the blockchain.

[12] The third rickroll has the data encoded in a structured format, maybe from some music database. The data format is base-64 metadatabase-64 lyrics The transaction is 0b4efe49ea1454020c4d51a163a93f726a20cd75ad50bb9ed0f4623c141a8008.

[13] The messaging system references "AtomSea & EMBII", who I assume are the creators. The chain started with address 12KPNWdQ3sesPzMGHLMHrWbSkZvaeKZgHt with 0.269 BTC on 2013-12-01 23:54:35 Each output is 0.000055 bitcoins, just over the current network minimum of .0000546 bitcoin. The next transaction in the chain can be found by looking at each change address, which pays for the next block. The chain ended when it ran out of bitcoins, at address 1DQwj8BDLWy9BMzX8uUcDYze3hx8q7uBy4.

In total, the data chain has 85KB of data including images, random quotes, and HTML. The system embeds filenames, lengths, and the data. There are also a lot of transaction ids stored in the data, presumably serving as an index.

[14] The 2.5 megabyte Cablegate file was stored in 130 separate transactions each holding 20,000 bytes of data, transactions 5c593b7b71063a01f4128c98e36fb407b00a87454e67b39ad5f8820ebc1b2ad5 to 2663cfa9cf4c03c609c593c3e91fede7029123dd42d25639d38a6cf50ab4cd44#o6". Each transaction includes a trivial 0.00000001 bitcoin donation to the Wikileaks donation address 1HB5XMLmzFVj8ALj6mfBsbifRoD4miY36v. This data is stored in checksummed download tool format.

[15] The cablegate description is in 691dd277dc0e90a462a3d652a1171686de49cf19067cd33c7df0392833fb986a, and is stored in "uploader" format. It's a bit circular that this message describes where to find the download tool, but the message itself needs the download tool to be read. Fortunately it's not too hard to read the message without the tool.

[16] The uploader is in transaction 4b72a223007eab8a951d43edc171befeabc7b5dca4213770c88e09ba5b936e17". The downloader is in transaction 6c53cd987119ef797d5adccd76241247988a0a5ef783572a9972e7371c5fb0cc.

In a cute touch, these transactions both donate 0.00000001 bitcoins to address 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa, which is Satoshi Nakamoto's address from the Genesis Block.

[17] This transaction, 77822fd6663c665104119cb7635352756dfc50da76a92d417ec1a12c518fad69 has an unusual scriptPubKey: OP_IF OP_INVALIDOPCODE 4effffffff 1443 bytes of data OP_ENDIF.

[18] The encrypted GPG files are in transactions 7379ab5047b143c0b6cfe5d8d79ad240b4b4f8cced55aa26f86d1d3d370c0d4c#o448, d3c1cb2cdbf07c25e3c5f513de5ee36081a7c590e621f1f1eab62e8d4b50b635#o448, and cce82f3bde0537f82a55f3b8458cb50d632977f85c81dad3e1983a3348638f5c.

[19] To "follow the money", the PDF transaction put change into address 1HT8vpTV1wj2ck6jgW7my6vCtJQv14Cdp. This address funded the embedding of a 10KB mystery file in this transaction. The change from that was used for another file here, followed by this, this, and this. These uploaded file transactions all included 0.001 BTC donations to 1JVQw1siukrxGFTZykXFDtcf6SExJVuTVE, the 50BTC.com address.

[20] Some addresses associated with the mystery transactions are: 18qr2srETSvQq4kP7yBYRqQ4LzmjhtRmcD, 1MaZAHzEFfinRJ2dwK6YtNDfvWMBkiAxDr, 1AgwESN7RKNZtaqzbqu6kPg3RS6C2qCgHi, 1AZUPm5PC5QguquNsBg7HhWUYz5dfm2nU9, and 1J1aR7ayNp9sma8QVyyWGF87PzDU1vp5BD.

The Bitcoin malleability attack graphed hour by hour

I have a new Bitcoin article: Hidden surprises in the Bitcoin blockchain
The Bitcoin network was subject to a strange attack this week. Up to 25% of the recorded transactions were modified using a technique called transaction malleability. By examining the Bitcoin blockchain, I've created an hour-by-hour look at the attack.

For details on how transaction malleability works, see my article Bitcoin transaction malleability: looking at the bytes. As a quick summary, the attacker takes a new Bitcoin transaction, modifies it in a trivial way that changes the transaction hash, and sends it back into the Bitcoin system. The modified transaction functions exactly the same (transferring the bitcoins between the same addresses), but results in two slightly different versions of the transaction in the system. However, if client software or exchange software depends on the transaction hash, temporarily having two different hashes for the transaction can cause a variety of problems.

The reason malleability is possible is that inside a Bitcoin transaction is a tiny program that provides the signature data. This script pushes the (hex) 48 byte signature by using the instruction 48. An attacker can change the script to use the OP_PUSHDATA2 instruction (4d) followed by a two-byte length (48 00). The modified transaction is still valid, since the script has exactly the same action.

Tracking the malleability attack

I created the graph below, which shows the hourly progress of the attack: the blue line is the total number of Bitcoin transactions, and the green line is the number of transactions that were modified by the malleability attack.

Graph of Bitcoin transactions suffering from malleability attack, Feb 2014.

Graph of Bitcoin transactions suffering from malleability attack, Feb 2014.

The attack started off affecting a fairly small number of transactions on Feb 9. The malleability attack itself appears to have started in block 284980 (Feb 9, 8:12 PST), which contains 36 modified transactions. Since the number of affected transactions in this block and following blocks was small, I wonder if this was a test phase for the attack.

The attack really took off the morning of February 10. At the peak, up to 25% of Bitcoin transactions were modified.

The attack ended fairly abruptly the morning of Feb 11. I made a bunch of transactions that evening, hoping to see a modified one, but I was disappointed that they all went through untouched.

A few modified transactions continued to trickle in for the next few days, with some even today (Feb 14). Some of these are older transactions that were mined very slowly because they didn't include any fees. For example, this transaction was modified on Feb 10, but not mined until Feb 14.

History of OP_PUSHDATA2 usage

I wanted to find out if there were any precursors to the malleability attack, or any similar attacks earlier. I scanned the entire blockchain looking for transactions using the OP_PUSHDATA2 opcode, which is used in the malleability attack. (As an aside, the Bitcoin data is a pain to parse for several reasons.)

Up until the attack, OP_PUSHDATA2 was very rare. I saw OP_PUSHDATA2 used in July 2013 here for a strange - but not modified (malleated?) - transaction. OP_PUSHDATA2 was used again in November 5 (here)when someone used OP_PUSHDATA to include a joke signature in the transaction: I should not run the washing machine while listening to WZBC. I managed to convince myself that the machine was slowly failing -- that a rythmic, squeaking noise it had been making had gotten a little worse. Ten minutes later, though, the machine had paused. But the noise was still there. All that text is stored inside the Bitcoin transaction. There are a bunch of ways to "hide" text messages in the blockchain, and this transaction used an unusual one.

On Feb 4, this transaction used OP_PUSHDATA2 in a strange broken transaction that wasted 0.001 BTC. Interestingly, the sibling transaction wasted 0.03201 BTC in a broken MULTISIG transaction with the "correct horse battery staple" public key. I conclude that someone was trying out strange things on Feb 4, including the rare OP_PUSHDATA2 instruction. Was this debugging for the malleability attack a few days later or was this unrelated experimentation?

Some conclusions

There has been some speculation that the malleability attack directed the modified transactions to a specific miner. However, I looked at the blocks containing these transactions, and they come from a variety of well-known miners. So there's nothing miner-specific about this attack. The attackers don't have their own mining pool.

There's a 100-millisecond sleep in the Bitcoin peer's message processing loop. There has been speculation that the attacker could beat regular peers by avoiding this loop: regular peers would wait 100ms to pass along messages, while the attacker could get a transaction, modify it, and send it to a miner immediately. This seems plausible to me.

One puzzle is that Mt.Gox announced their difficulties on Feb 7, and then explained Feb 10 that they were stopping withdrawals due to a malleability attack. Since the OP_PUSHDATA2 attack didn't start until Feb 9, this attack can't be responsible for the Feb 7 problems. One possibility is there was a different type of malleability attack that affected Mt.Gox. It would be interesting to get the hash for one of the affected transactions from before Feb 7, to see what was going on.

Around the same time as the malleability attack, many people received tiny payments from 1Enjoy and 1Sochi addresses. I believe all these payments were rejected by miners as junk and remain unconfirmed. As far as I know, there is no connection between these tiny spam payments and the malleability attack, but the timing is suspicious.