By Fergal Toomey - Chief Scientist and Co-Founder at Corvil
In my last blog entry I discussed the prevalence of ‘microbursts’ in market data feeds – short, occasional bursts of activity in the feed during which data rates reach levels much higher than average. We saw that during busy periods the LatencyStats.com feeds can send many high-rate microbursts in succession, each burst lasting much less than one second. If the data rate during a microburst exceeds the capacity of a network link or a feed handler, the excess data will be forced to queue. This adds latency to the feed messages, and in the worst case messages may be discarded if a system runs out of buffer space to hold waiting data.