Middleware is a term often thrown around when talking about technology, but what exactly does it mean? There is no precise definition because it is used to cover a range of functions. A useful way to think of it is as a kind of infrastructure, or set of services, that software applications rely on to get jobs done. 

One of the things about infrastructure is that it tends to disappear into the background and only becomes visible when it stops working. But as the recent crisis has shown, infrastructure is far more important than is generally recognized. 

With infrastructure being so critical to the NYFIX Marketplace, Itiviti developed its own as part of the continuous investments made in ensuring NYFIX is the leading global order routing network. We value innovation and draw on leading open source technology to create a state-of-the-art solution. Now that our middleware is proving its robustness in the demanding operations of our network, we are contributing our code back to the open-source community

A matrix for FIX messages

NYFIX Marketplace is sometimes referred to as a hub-and-spoke system. From the outside, it can look like that, with NYFIX as the 'hub' and counterparties as the 'spokes' (diagram 1). Internally, however, NYFIX Marketplace is, in fact, a matrix, with each counterparty connected directly to its counterparties. FIX messages flow directly between counterparties, with some messages also sent to database servers for safe storage or 'drop-copied' to multiple destinations (diagram 2).

 nyfix diagrams sidebyside

Keeping track of all message senders and receivers can get complicated, especially as the size of the network increases. With NYFIX Marketplace, which is the world's largest FIX network, there are many thousands of interconnections to manage. This is where middleware comes in. It allows applications to 'publish' messages from customers or counterparties onto the network without needing to know their actual destinations. Instead, the destinations – applications servicing customers or counterparties – 'subscribe' to and receive messages relevant to them. 

This publish-subscribe architecture greatly simplifies the problem of delivering millions of messages between thousands of counterparties. It also makes the NYFIX network more scalable and reliable by avoiding bottlenecks and single points of failure. 

Publish-subscribe middleware is also commonly referred to as a 'service bus'.

Building our own

Initially, we relied on middleware from other software vendors, but with the middleware being such an integral part of our operation for the NYFIX marketplace and our need for more control, we decided to develop our own and started work on a project a few years ago.  After assessing our requirements and the tools available, we decided to combine two leading open-source software projects and add further functionality where we needed it. 

For the messaging interface, we chose OpenMAMA, an application programming interface (API) that works with any middleware, and for several years has been incorporated into commercial products and in-house systems. Middleware also requires a network transport layer – technology that deals with things like connection communication, reliability, and flow control. We turned to ZeroMQ, an open-source messaging library that is widely used in Linux-based operating systems and by major technology and R&D organizations and included in many Linux distributions. It offered the features, reliability, and performance we required for a network transport layer. ZeroMQ is essentially a toolkit, which has the advantage of not imposing architecture on what you build. On the flip side, its flexibility carries with it a steep learning curve, and developing applications with it can involve a lot of effort.

 middleware doron quote1

By combining OpenMAMA and ZeroMQ with the functionality we developed ourselves, we created a production- quality, high-performance message-oriented middleware that we call OZ. It provides many of the most popular features of this type of middleware, such as publish-subscribe and request/reply (inbox) messaging, dynamic service discovery ('yellow pages' for finding services) and broker-less architecture (no intermediary between message sender and receiver) for reduced latency and optimum throughput.

Sharing OZ

We rolled out the new middleware earlier this year across our Europe, Asia and North America data centers. From March 2020, OZ was live across all three regions, processing over 50 million FIX messages a day with no issues. This includes periods of significant market volatility during the coronavirus pandemic.

 middleware frank quote

Itiviti believes in sharing ideas and experiences – within our teams, as well as with clients, partners and our wider community. With OZ, we have gone even further. We are sharing the actual code that we use to run our business. 

Our OZ innovation would not have been possible without the contributions of many others, including the OpenMAMA and ZeroMQ communities. In the process of developing our solution as part of our evolving technology stack, we contributed back to those communities. And by open-sourcing OZ, we hope it will establish its own community that will improve the software further.

Bill

Written by

Bill Torpey

NYFIX R&D - Managing Director

Bill has over forty years of experience designing and building reliable, high-performance software in the financial technology industry, with particular specialties in in-memory database and messaging middleware technology. Bill joined NYFIX in 2007 when the company he founded was acquired by NYFIX -- that software continues to power the NYFIX Marketplace today.

Share this insight