The high-frequency trading debate has been polarising opinion for years now, and with little impact on the march of the technologies which are enabling it. Here at MoneyScience, we try not to take a view on the ethics or cultural impact of HFT - we like the evolution of technology as a rule, but dislike speculation when it comes at the expense of markets which would otherwise provide a socially meaningful role. Progress is generally good, we feel - but to paraphrase Spiderman, 'with great power comes great responsibility' - and financial markets as a rule haven't done a great job in recent history of demonstrating they can handle it. We may have the technology to trade ultra-fast, but whether we have the scientific or economic infrastructure to understand and control it is the core of the debate.

The Knight Capital debacle has prompted the usual soul-searching in the general populace - though not, one assumes, in the HFT or Algo-trading communities where the view seems to be that human error was the problem - not the robots  themselves. As anyone with a GCSE in Computer Science knows, Garbage in, Garbage out: this is not a narrative which permits the explanation that the system itself might be garbage.

Anyhow, in this extended article, Wired get in on the act with an interesting overview of the HFT space which adds some historical context as well as identifying some of the technologies which will take HFT into the future. Definitely worth a read.

Editor’s note: One of the most interesting things about the catastrophe at Knight Capital Group—the trading firm that lost $440 million this week—is the speed of the collapse. News reports describe the bulk of the bad trades happening in less than an hour, a computer-driven descent that has the financial community once again asking if their pursuit of profit has lead to software agents that are fast, dumb, and out of control. We’re posting this story in advance of its publication in Wired’s September issue because it examines how Wall St. got to the point where flash failures come with increasing frequency, and how much farther traders seem willing to go in pursuit of ever-greater speed.

Wall Street used to bet on companies that build things. Now it just bets on technologies that make faster and faster trades.

Extract

...Not all trading takes place in New York. By historical accident, derivatives such as futures and options are mostly traded on the Chicago Mercantile Exchange, 720 miles away. So a few years ago, a company called Spread Networks began quietly buying up rights-of-way for a route that would lop about 140 miles off the shortest fiber-optic cable distance between the Chicago Merc and the communications hub of Carteret, New Jersey, the primary data center for Nasdaq. Existing networks tend to follow railroad lines and were designed to serve population centers, not to provide a point-to-point link for traders. Instead of dipping south toward Philadelphia, Spread’s route heads northwest through central Pennsylvania and then due west to Cleveland. Latency is typically measured in round-trip times (i.e., an order and a confirmation); the shortest cable route before Spread lit up its network in 2010 clocked a round-trip time of 14.5 milliseconds, according to Spread executives, but capacity was inadequate, so most customers had to settle for 15.9 milliseconds. Spread cut that to as little as 13.1 milliseconds for its premium “dark fiber” service, a connection that doesn’t have to be shared with other customers. Prices are a closely guarded secret in this world, although the consensus estimate among traders is “plenty.”

Spread quickly began signing up customers, but by the spring of 2012, there was a faster competitor on the horizon. Because of some complicated physics, the speed of light through any medium is inversely proportionate to the medium’s index of refraction—so signals travel about 200,000 kilo­meters per second through fiber-optic cable, compared with 300,000 through the atmosphere. The fastest communication between New York and Chicago would be line-of-sight through the air, which requires a chain of microwave relay towers. Tradeworx is building such a network, as is McKay Brothers, a California firm that hopes its system will be the fastest, with a round-trip latency of less than 9 milliseconds. Its route, cofounder Bob Meade boasts, uses the smallest possible number of towers, 20, and deviates from a perfect geodesic (more poetically known as a “great circle,” the shortest line between two points along a planet’s surface) by just 4 miles. It includes roughly another 2 miles of wiring in the towers themselves, connecting the dish antennas hundreds of feet in the air with amplifiers on the ground...