Timestamps are not clocks

Europe’s MiFID 2 regulations contain requirements for “clock synchronization” (RTS 25). ESMA’s use of that term mirrors language in the directive itself, but an unfortunate consequence is that some investment firms and trading venues seem to be interpreting it too literally. That is, they view their sole task to be synchronizing their clocks and are paying too little heed to the accuracy of software timestamps obtained from those clocks. Organizations that take this approach are assuming a big compliance risk. The intent of MiFID 2 is clearly to regulate the accuracy of timestamps. Clock synchronization is just a means to that end, and a partial means at that. Let me explain.

The point of RTS 25 is to make it more feasible for national regulators to reconstruct the sequence of events in markets where trades are interspersed by tiny fractions of a second. The regulators want every timestamp that an organization associates with a reportable event to diverge from universal time (UTC) by no more than very small, specific amounts. The fact that the drafters of MiFID 2 expressed this requirement as “synchronisation of business clocks” isn’t surprising. They are not technologists. In daily life, there’s not much difference between clock synchronization and timestamp accuracy. If your watch is synchronized to another clock, then so are your readings of the time. That is, when speaking in the magnitudes of ordinary life, there is essentially no delay between the “timestamp” your brain takes and the time on the watch.

But the same is not true of software timestamps when we care about accuracy in millionths or thousandths of a second. Even if the system clock underneath that software is perfectly synchronized to UTC, timestamps taken from the clock can far exceed the RTS 25 tolerances for error.

This is because a modern software system has many independent actors within it, which compete for hardware resources. As a result, sometimes timestamping can be delayed while the system handles other tasks. These delays are largest and most frequent when the system is busiest—for example, when markets are most active (and perhaps most interesting to regulators).

On the scale of MiFID 2, software-timestamping delays can be large and unpredictable. Our data show that in a well-controlled system, the maximum delay can be kept to microseconds or tens of microseconds. But in poorly controlled systems, delays can be hundreds of microseconds or even milliseconds. And from our conversations with regulated firms, it seems that often the systems in scope for RTS 25 fall into the poorly-controlled category. The resulting timestamp errors can easily put these systems out of compliance with RTS 25, which requires accuracy to within 100 microseconds or 1 millisecond of UTC (depending on certain criteria).

Thus, focusing exclusively on clock sync is like preparing your child for an entrance exam by simply ensuring she has sharp pencils. Necessary? Yes. Sufficient? Hardly.

To understand the compliance risk this presents, consider that the timestamping rules exist in the context of other regulations and laws. For example, in a market-abuse investigation, the burden of proof is typically on the firm being investigated. If a firm suspected of using HFT algorithms to front-run client orders produces evidence that relies on application timestamps that don’t comply with RTS 25, the firm may have a hard time proving that a given client order actually arrived after an HFT order was sent.

IT engineers usually understand that software timestamping is fraught with challenges, but they sometimes express resignation. This can be for organizational reasons, as in: “IT can’t help it if an application developer writes code in a non-compliant way”. That’s a fair point and underscores the need for application owners to be part of the compliance process for MiFID 2 timestamping.

But sometimes this sentiment reflects a view that the noise in software timestamps is beyond control. That is not true. While it isn’t possible to squeeze out all unpredictability, firms can draw on well-understood disciplines to achieve timestamps with the stability required by RTS 25. Options for remediation include finely tuning the environment, upgrading it, or modifying the application.

At a minimum, organizations must understand the reliability of software timestamps for each application. If remediating a given set of applications is unattractive, there are hardware-timestamping alternatives such as wire capture. Wire capture will make sense in many situations, though the regulations implicitly exclude it in some cases.

Drawing attention to software timestamps is not to minimize the importance of clock synchronization itself. Getting a reliable source of UTC in all locations, distributing that time to all applications and devices in a fault tolerant way, and ensuring that clocks are staying in sync are non-trivial challenges that present engineers with a host of decisions to make. But once in place, time-sync infrastructure poses a far lower risk of non-compliance than that posed by application timestamping. Infrastructure is also a more contained problem. There are fewer components than the hundreds or thousands of application instances that feed off them. And infrastructure components are relatively easy to upgrade, whereas the lead time to remediate and regression test applications and their platforms is much longer.

To help accelerate the compliance process for both infrastructure and applications, the STAC-TS Working Group is developing standards and tools to assess the accuracy of timestamps in various computing environments. This will enable firms to identify systems that don’t comply, as well as provide design-level evidence of compliance for systems that do.

I look forward to discussing these topics at the MiFID 2 workshop we’re hosting on May 9, where a collection of experts will help business and technical people plan and budget by reaching a common understanding of the regulatory requirements, their implications, and strategies for both achieving and proving compliance. It’s going to be a busy summer.