Trading in Milliseconds: When Correlations Break Down

May 2nd, 2014 at 7:20 pm

My temporal mind has been whipsawed lately, as having just finished Piketty’s excellent tome on wealth inequality over the centuries—a book where a decade is a flicker of time—I’m turning to Michael Lewis’ book on high-speed traders cashing in on arbitrage opportunities measured in milliseconds.

In case you’re wondering the latter is all about, I’m thoroughly enjoying this paper by Budish et al from which the figure below is drawn.  I think it provides a great way of seeing what’s going on here.

The figure tracks the prices of two securities, labeled ES and SPY, both of which track the S&P 500, and it tracks them over four diminishing time horizons.  Clearly, over a typical trading day these two securities move in lockstep, as you’d expect.  When equity prices are thusly correlated, there’s no obvious chance to make some money by selling one high and buying the other low.  That tight correlation is still there on an hourly basis and even over the course of a minute.

But once you get to milliseconds, the correlation breaks down.  Somewhere between 1:51 and 1:52, more precisely between the 39th and 40th second therein—even more precisely, I’d say around the 558th millisecond—you could sell ES high and buy SPY low.

Perhaps it’s not so surprising that correlations break down at millisecond intervals—kinda like the first time you looked at a leaf under a microscope, it looked nothing like a leaf.  But the fact that technology allows high-frequency traders to take advantage of such arbitrage opportunities is what’s germane, and pretty amazing, here, as Lewis writes about so compellingly (and I’ve just started his book).

OK, so what’s the problem?  Well, there are many.  First, this type of trading ain’t exactly a model of your grandad’s stock market, the purpose of which was to allocate excess savings to its most productive use.  It’s investing lots in hardware, software, and human innovation by smarter than average people to uncover millisecond price differences that have nothing to do with added economic efficiency in any way that at least I can recognize.

Moreover, according to Budish et al:

We calculate that there are on average about 800 such arbitrage opportunities per day in ES-SPY, worth on the order of $75 million per year. And, of course, ES-SPY is just the tip of the iceberg [I’m not sure why there aren’t many more than 800–JB]. While we hesitate to put a precise estimate on the total prize at stake in the arms race, back-of-the-envelope extrapolation from our ES-SPY estimates to the universe of trading opportunities very similar to ES-SPY – let alone to trading opportunities that exploit more subtle pricing relationships – suggests that the annual sums at stake are in the billions.

These authors point out ways in which such HFT actually reduces market liquidity (relative to a world without HFT) by exposing “liquidity providers”—traders selling stocks—to high-speed front-runners who can “pick off” stale quotes, like the elevated ES price in the millisecond panel above, before the sellers can adjust.

I’m working on something longer about this that I think my readers will find quite interesting.  It’s not just technology that allows this to occur.  It’s also lax regulation, and there are good, smart people thinking and acting on ways to stop it.

More to come.


Source: Budish et al.

Print Friendly, PDF & Email

6 comments in reply to "Trading in Milliseconds: When Correlations Break Down"

  1. Charles Idelson says:

    High frequency trading the problem? Yep, exactly why we need a financial transaction tax on Wall Street speculation. HR 1579, the Inclusive Prosperity Act, introduced by Rep. Keith Ellison, would impose a very modest tax on trades of stocks, bonds, derivatives and other financial instruments that would damp down such HFT trades, reducing volatility on the market that does nothing but harm our economy — and could raise hundreds of billions of dollars of revenue for critical human needs, such as jobs, healthcare, student debt, and fighting poverty, HIV/AIDS and the climate crisis.

    11 European Union countries are in the process of implementing a similar tax. Why should we be left behind. Learn more at

  2. rootless_e says:

    “First, this type of trading ain’t exactly a model of your grandad’s stock market, the purpose of which was to allocate excess savings to its most productive use.”

    If you start from a grandly counter-factual premise, like the one above, your conclusions will not make any sense. When exactly was the stock market engaged in this wonderful exercise of allocating excess savings to the most productive use?

  3. Smith says:

    1. More interested in Piketty’s book than Michael Lewis’ book. We understand Flash Boys is probably more fun to read, but high speed trading and front running involve billions to tens of billions per year (estimated from news stories, New York Times Magazine and 60 Minutes), whereas Picketty addresses trillions (increased share of income from 1980 to 2012, 5% to 10% of $17 trillion, give or take, and depending on interpretation of income).
    2. Isn’t Flash Boys but a symptom of the part of Piketty’s thesis that wealth begets wealth? Cost of entry into starting high speed trading firm is prohibitive.
    3. While Piketty has everyone’s attention isn’t this the best time to capitalize on the opportunity to press for reform and restoration of the New Deal?
    4. To show just how misguided (to use the gentlest of terms) liberal debate often becomes, Steve Rattner suggests ending corporate taxes in today’s Times. Nearly every argument made for doing this is outrageously false, unworkable, and dangerous to the premise it’s even possible to begin restoring previous levels of equality. Most notable, it’s diametrically opposed to Piketty’s call not to seek the lowest common denominator in setting economic policy, but just the opposite, work to globalize efforts fighting inequality.
    5. Things I want to know about Piketty’s book:
    a. How are labor issues addressed ( new book coming out sure not to gather as much attention ) My own take is that it’s essential to restore New Deal protections, with some updates for modern economy, and not rely on redistribution.
    b. While wealth tax gets all the attention, merely restoring confiscatory rates for high incomes, estate taxes, and corporate rates, eliminating special treatment of dividends and capital gains, and closing loopholes, may be sufficient. Key political idea in selling such a program is that it’s a conservative restoration, same way Reagan and Tea Party framed their agenda on the right. Would Piketty data indicate this is insufficient? By how much?
    c. Does Piketty address role of debt, U.S. bankruptcy law of 2005 removed moral hazard of banks making risky loans, likewise several changes to student loan programs insulating banks encourage reckless lending and higher tuition for everyone. Also Supreme Court ruling in 1978 effectively eliminated 200+ years of state usury law. Debt causes inequality.
    d. Political issues, what can motivate people in an era of cheap fast food, wide screen TVs and cell phones (bread and circuses) to vote. Why are there so few New Dealers? How does Piketty address? War doesn’t just destroy capital, it discredits the losing party, and even kills off the party leaders. In the U.S. the revolutionary war caused an exodus of loyalists, the civil war diminished the South’s role in setting the national agenda, Hoover’s role as president at the outset of the Great Depression’s discredited Republicans, rescued only after 20 years by a war hero (Eisenhower).
    e. Role of education, college is the new high school, but not free in the U.S., aid dependent on parental support or filing for aid, or interim independent living between high school and college. Can economics and/or economic history be taught in high school. (the French manage to teach philosphy) Piketty take on education?
    f. In an era of 80% permanent Bush tax cuts, Heritage Foundation’s RomneyCare, Andrew Cuomo’s encouragement of gambling (a poor tax) and cutting estate tax, defeat in Senate of minimum wage, are there 2014 midterm bullet points that can be supported with Piketty data?
    g. Throw in Comcast TimeWarner anti-anti-trust response from Obama, and Obama’s ending net neutrality instead of establishing the internet as a public utility, monopolies and oligopolies and too big to fail banks play a crucial role in rising inequality. Is this reflected in Piketty data? Rents from innovation aided by Obama’s corporate tilt in IP law, destroying over 200 years of patent law, changing first to invent, now it’s first to file.

  4. Tom Cantlon says:

    An analogy you’re welcome to use if you find it useful. It would be like someone standing between you and the cashier. Anytime they see that the price tag on your can of beans is higher than what the cash register expects (the stock boy hadn’t gotten that far with new price tags) they step in the middle and take the difference. They pay the cashier the 95cents the store is now charging and then take your 98cents. Except not just anyone can set themselves up to stand between customers and cashiers. It takes having many billions of dollars of capital to back up those brief moments of holding the product. So only people who are already extremely fat and happy can get into this no-lose skimming position.

  5. Chester Molshensky says:

    Incorrect analysis. ES is a future and SPY is a managed ETF that pays dividends. There is a basis risk that you are not talking about.