Order and Execution Events

Quite recently we had a customer come to us with a question regarding the Execution events received on the API.  The question was: "How do a I recognise the last execution event for a given order event?"  This is a little bit tricky in the Java API as it does not behaviour exactly the same way that FIX does.  E.g. If an individual order with a quantity of 30 aggressively matches multiple price points (quantity of 10 at each) on the exchange, a fix user would expect would expect:

MsgType(35)=8, ExecType(150)=New(0), CumQty(14)=0
MsgType(35)=8, ExecType(150)=Trade(F), CumQty(14)=10
MsgType(35)=8, ExecType(150)=Trade(F), CumQty(14)=20
MsgType(35)=8, ExecType(150)=Trade(F), CumQty(14)=30

As our API is based upon our XML protocol the information that we can return on the API is restricted to what we receive in those events. For a similar scenario over our XML protocol only emits a single order event at the end of the matching cycle, therefore the data output would be:

So the Java API does not have the information about the individual filled quantities as a result of each execution only the final state of the order. This can make it a little tricky to find which execution event represents the end of the matching cycle. The information seen by a user of the Java API for the same scenario would be:

getQuantity() = 10, getOrder.getFilledQuantity() = 30
getQuantity() = 10, getOrder.getFilledQuantity() = 30
getQuantity() = 10, getOrder.getFilledQuantity() = 30

However, it is possible to use some of the additional events that are available on the Java API to derive the same behaviour. By listening to both Execution and Order events we can track the cumulative quantities as we go. It does require a little bit more state management by the client, but the logic is fairly straight forward.

We're hiring!

If you like what we do and you're interested in working for LMAX, take a look at our Stack Overflow Careers advert.

Weekly Events Update - Tuesday 26th July

Apologies for the late post, unfortunately sometimes actual development gets in the way.

Last week

The Disruptor was nominated for the 2011 Java Innovation Awards.

We made a change to the application which means that FX trading is now only shut down for 5 minutes after 10pm, instead of 30 minutes.

A team of LMAX cyclists completed the London to Cambridge Bike Ride, raising money for Breakthrough Breast Cancer, and no-one (from our team) died.

This week

Mike will be giving a lightning talk at the LJC/JBUG event tonight, Tuesday. Trisha and Martin will also be turning up.

Future events

5th October - TradeTech Architecture. Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

13th - 14th October - Goto Amsterdam. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be tackling the folklore surrounding high performance computing.

31st October - 2nd November - JAX London.  Mike Barker and Trisha Gee will be presenting "Understanding the Disruptor, a Beginners Guide to Hardcore Concurrency".

15th November - StackOverflow Dev Days 2011. Mike Barker will be presenting on Performance & Technology Folklore. 

Weekly Events Update - Monday 18th July

This week

Some of the development team will pop in to the London Java Community monthly social on Tuesday. It's always a great night to talk tech (or not) and meet new people.

On Wednesday, Dave Farley will be giving a presentation at SkillsMatter on how we do Continuous Delivery.

Future events

5th October - TradeTech Architecture. Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

13th - 14th October - Goto Amsterdam. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be tackling the folklore surrounding high performance computing.

15th November - StackOverflow Dev Days 2011. Mike Barker will be presenting on Performance & Technology Folklore.

Nominated for three UK Agile Awards

We're excited to announce that we've been nominated for an Agile award in three categories:

• Best Agile Team
• Most Agile Aware Organisation
• Most Engaged Management Team
  

Using agile techniques is important to the way we deliver business value as fast as possible, so it's nice to get a little recognition in this area.

Weekly Events Update - Monday 4th July

Last week

We made a number of tweaks to the Disruptor, with some improvements to the code and some changes to the website, and there are now a number of blog posts explaining sections of it.

There has also been work on two .NET versions of it: one on github and one on Google code.  We're pretty excited about them and interested to see the results.

This week

On Wednesday some of us are attending "The Future of TX Trading Debate" hosted by EQUINIX.

Thursday is Oracle's global introduction to Java 7, with a live webcast.

Future events

5th October - TradeTech Architecture. Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

13th - 14th October - Goto Amsterdam. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be tackling the folklore surrounding high performance computing.

Technology Events - State of play on Monday 27th June

An update on upcoming LMAX Technology events:

This week:
On Tuesday a number of us will be at the UCL Algorithmic Trading Conference.  Martin Thompson, CTO, will be presenting on Risk Management.

Future events:
5th October - TradeTech Architecture. Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

13th - 14th October - Goto Amsterdam. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be tackling the folklore surrounding high performance computing.

Disruptor technical paper available now

An article describing the ground-breaking Disruptor pattern developed here at LMAX is now available on the Disruptor website. The paper covers the concepts outlined in presentations like How to Do 100K+ TPS at Less than 1ms Latency.

Abstract
LMAX was established to create a very high performance financial exchange. As part of our work to accomplish this goal we have evaluated several approaches to the design of such a system, but as we began to measure these we ran into some fundamental limits with conventional approaches.

Many applications depend on queues to exchange data between processing stages. Our performance testing showed that the latency costs, when using queues in this way, were in the same order of magnitude as the cost of IO operations to disk (RAID or SSD based disk system) – dramatically slow. If there are multiple queues in an end-to-end operation, this will add hundreds of microseconds to the overall latency. There is clearly room for optimisation.

Further investigation and a focus on the computer science made us realise that the conflation of concerns inherent in conventional approaches, (e.g. queues and processing nodes) leads to contention in multi-threaded implementations, suggesting that there may be a better approach.

Thinking about how modern CPUs work, something we like to call “mechanical sympathy”, using good design practices with a strong focus on teasing apart the concerns, we came up with a data structure and a pattern of use that we have called the Disruptor.

Testing has shown that the mean latency using the Disruptor for a three-stage pipeline is 3 orders of magnitude lower than an equivalent queue-based approach. In addition, the Disruptor handles approximately 8 times more throughput for the
same configuration.

These performance improvements represent a step change in the thinking around concurrent programming. This new pattern is an ideal foundation for any asynchronous event processing architecture where high-throughput and low-latency
is required.

At LMAX we have built an order matching engine, real-time risk management, and a highly available in-memory transaction processing system all on this pattern to great success. Each of these systems has set new performance standards that, as far as we can tell, are unsurpassed.

However this is not a specialist solution that is only of relevance in the Finance industry. The Disruptor is a general-purpose mechanism that solves a complex problem in concurrent programming in a way that maximizes performance, and that is simple to implement. Although some of the concepts may seem unusual it has been our experience that systems built to this pattern are significantly simpler to implement than comparable mechanisms.

The Disruptor has significantly less write contention, a lower concurrency overhead and is more cache friendly than comparable approaches, all of which results in greater throughput with less jitter at lower latency. On processors at moderate clock rates we have seen over 25 million messages per second and latencies lower than 50 nanoseconds. This performance is a significant improvement compared to any other implementation that we have seen. This is very close to the theoretical limit of a modern processor to exchange data between cores.

Update on Technology Events

An update on upcoming LMAX Technology events:

This week:

• On Tuesday, some of the development team will be wandering along to the monthly London Java Community pub night. This month's theme is Development War Stories.
• On Wednesday, Martin Thompson, CTO, will be talking about our real-time risk management at 4pm (UK time) via webinar.

Future events:
5th October - TradeTech Architecture. Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

13th - 14th October - Goto Amsterdam. Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be tackling the folklore surrounding high performance computing.


The public calendar (viewable on the right hand side of this blog) is up to date with all the events we're attending or would like to attend.

Upcoming Technology Events

Since going live last year, LMAX technologists have been attending and presenting at a number of events.  We've decided to write a weekly blog post summarising what's on the horizon to give people more of an opportunity to attend - after all, we want you guys to come and see us.

The calendar is fairly fluid, and sometimes confirmation doesn't arrive until the last minute, so things may change week by week.  However, a regular update will at least give some visibility over what's in the pipeline.


22nd June, 4pm - Ultra High Performance Risk Management: Informatica Webinar Featuring Martin Thompson, CTO, LMAX.

5th October - TradeTech Architecture.  Martin Thompson, CTO, will be on one of the panels.

11th October - Goto Aarhus.  Martin Thompson (CTO) and Dave Farley (Head of Software Development) will be presenting the LMAX Disruptor pattern: "100K transactions per second at less than 1ms latency".

Upcoming LMAX Technology events

Senior LMAX techies will be presenting at a couple of events this week:

• Tomorrow, Mike Barker will be at the STAC Performance Summit, on the panel discussion: The Future of Messaging Middleware.  He'll be using his experience as Lead Developer here to answer questions on how developers really use messaging.
• On Friday, Dave Farley, Head of Software Development, will be presenting on Continuous Delivery at the ThoughtWorks Manchester Open Day.  Continuous Delivery is key to how LMAX can adapt quickly to change and release updates to the platform frequently and in a timely fashion.