Random erroneous updates for GOOG.O and others

cory.schmidt.1

We have noticed that we receive erroneous updates randomly during the course of a trading day. For example, with 5-minute GOOG.O we received the following update:

It's almost 4 points below the previous trade. Here's what it looks like in our chart:

Here's the relevant code to create the log file I referenced above:

// Create the platform session.
using ISession session = Sessions.GetSession();

// Open the session
session.Open();

HistoricalPricing.Sessions[] sessionArray = new HistoricalPricing.Sessions[]
{
    HistoricalPricing.Sessions.normal, 
};

// Create a Historical Pricing stream - specifying the desired 'intraday' interval
string symbol = "GOOG.O";
var stream = Summaries.Definition(symbol)
                      .Interval(Summaries.Interval.PT5M)
                      .Fields("DATE_TIME", "OPEN_PRC", "HIGH_1", "LOW_1", "TRDPRC_1", "ACVOL_UNS")
                      .Sessions(sessionArray)
                      .Count(1250)
                      .GetStream();

// Open a file for output
StreamWriter outputStream = new StreamWriter($"C:\\Users\\coryschmidt\\{symbol}Stream.txt");

stream.OnInsert((data, st) =>
    {
        outputStream.WriteLine($"INSERT: {DateTime.Now}");
        Common.OutputTableToStream(data.Table, outputStream);
    })
    .OnUpdate((data, st) =>
    {
        outputStream.WriteLine($"UPDATE: {DateTime.Now}");
        Common.OutputTableToStream(data.Table, outputStream);
    })
    .OnStatus((status, st) => outputStream.WriteLine($"STATUS: {status}"))
    .OnError((error, st) => outputStream.WriteLine($"ERROR: {error}"))
    .Open();

If I re-request the historical data the erroneous Low disappears.

cristian.baciuwahl

Hello @cory.schmidt.1

Historical data and streaming updates are retrieved through different services.

If the defect with the LOW value would happen all the time, I would presume that maybe there is a problem with the metadata or derivative rules, because those rules are loaded once and used for all incoming updates. If it happens randomly and most of the times the field has the expected value, I am lead to believe that incorrect data might be coming from the backend services, making this a Time Series content issue.

Best regards,

Baciu Wahl Cristian

nick.zincone

Hi @cory.schmidt.1

When you observe this, is it only for GOOG.O? Also, do you notice a specific trend, for example I can see a value of LOW_1 included with the update - does this anomaly occur coincidentally with a specific value like LOW_1?

cory.schmidt.1

We have also seen it with other instruments like NVDA.O and META.O and with HIGH_1 as well. We have only observed it with intraday intervals, not Daily. In one test we had 3 charts open: a 1-minute, a 5-minute, and a Daily of NVDA.O and the anomaly showed up in the 1-min and 5-min charts but not the Daily.

cristian.baciuwahl

Hello
@cory.schmidt.1

I have provided a new package version (1.0.9) with enhanced tracing for the summary stream flow.
I've sent you the details and link over email, as usual.

Could you please turn on trace logs and when you detect this again, send us the logs ?

This will help in our ongoing investigation.

Thank you,

Cristian