Tick data from dot net API not matching Tick Data from Reuters RTD

rohit.agarwal.rg

I am using below .NET code

_request1 = _timeSeries.SetupDataRequest("DIJF22")

.WithView("TRDPRC_1")

.WithAllFields()

.WithInterval(CommonInterval.Tick)

.WithNumberOfPoints(60000)

.OnDataReceived(DataReceivedCallback)

.CreateAndSend();

And I am using below excel formula

=RHistory("DIJF22","TRDPRC_1.Timestamp;TRDPRC_1.Value;TRDPRC_1.Volume","INTERVAL:TICK NBROWS:60000","FRQ:10M","CH:Fd",B1)

The results from both of them are not matching at all.

Please check

Jirapongse

You can try the TSDB_RAW view.

            var _request1 = timeSeries.SetupDataRequest("DIJF22")  
                .WithView("TSDB_RAW")
                .WithAllFields()
                .WithInterval(CommonInterval.Trades)
                .WithNumberOfPoints(60000)             
                .OnDataReceived(OnDataReceived)
                .WithTimeZone(TimezoneType.GMT)
                .CreateAndSend();

        private void OnDataReceived(DataChunk chunk)
        {
            Console.WriteLine("{0}", chunk.Ric);
            foreach (IData record in chunk.Records.Reverse())
            {
                foreach (string field in record.Keys)
                {
                   if (field == "TSDB_TRDPRC_1" || field=="TSDB_TRDVOL_1" || field=="TSDB_VhExchgTime")
                     Console.Write("{0}, {1}, ", field, record[field]);
                }
                Console.WriteLine("");
             
            }
        }

zoya faberov

Hello @rohit.agarwal.rg,

I think the cause for the not matching of the results is in parametrization. Only if every parameter matches between excel formula and code, will the results be consistent, and myself I don't always find these mismatches in parametrization easy to catch.

In this case, I think it's update frequency that differs. In Excel you have 10 minutes frequency, and in .Net code- every tick.

Do the results look right if you change it to be the same?

rohit.agarwal.rg

Thanks Zoya for getting back. The 10 min in excel formula is actually the frequency in which the data will be refreshed in the sheet. The interval as can be seen in excel formula ("INTERVAL:TICK NBROWS:60000") is Tick only.

Secondly, I checked more and can see that if I am using Interval as TAS in excel then I am getting the results same as what I am getting using Interval as TICK in .net

I think that in .Net you are returning the TAS data even when the interval is set as TICK.

Thanks in advance.

zoya faberov

Hi @rohit.agarwal.rg,

Tick interval did not apply for me via API for this expression.

However, 10M as interval with 60 points, to see more clearly what is returned

In Excel

=RHistory("DIJF22","TRDPRC_1.Timestamp;TRDPRC_1.Volume;TRDPRC_1.Close","TIMEZONE:UTC NBROWS:60 ADJUSTED:NO INTERVAL:10M",,"SORT:ASC TSREPEAT:NO CH:Fd",L3)

And 10M in code:

 request = timeSeries.SetupDataRequest(TimeSeriesRequestExample.instr)
       .WithView("TRDPRC_1")
       .WithAllFields()
       .WithInterval(CommonInterval.Intraday10Minutes)
       .WithNumberOfPoints(60)
       .OnDataReceived(DataReceivedCallback)
       .CreateAndSend();

The results appear to match (within the available options):

zoya faberov

If the parameters match, which can be challenging, then the result should match too.

rohit.agarwal.rg

Thanks Zoya for your response.

I am sure that API is working for intervals other than TICK.

But I need the TICK data in .net API same as what I am getting via excel for below formula

=RHistory("DIJF22","TRDPRC_1.Timestamp;TRDPRC_1.Value;TRDPRC_1.Volume","INTERVAL:TICK NBROWS:60000","","CH:Fd",B1)

Please test why .net is not returning the same and if there is any way to get the same then please share me request code for .net

zoya faberov

Hello @rohit.agarwal.rg,

Absolutely.

.Net code should return the same Timeseries result as Excel interval Tick, as long as parameters are the same.

Here is an easy way to see and to reproduce.

My Excel

My .Net request output:

My code:

request = timeSeries.SetupDataRequest(TimeSeriesRequestExample.instr)
       .WithView("TRDPRC_1")
       .WithAllFields()
       .WithInterval(CommonInterval.Tick)
       .WithNumberOfPoints(6)
       .OnDataReceived(DataReceivedCallback)
       .CreateAndSend();

Let us know if it works the same for you. If you are not getting the output, this is because the output for Interval Tick does not cast as Bar, as is shown in .Net TimeSeries example. To see it correctly I do:

if (chunk.Records.ToArray().Length != 0)
            {
                Console.WriteLine("Instr: Timestamp Value Volume");
                foreach (IData dat in chunk.Records.ToArray())
                {
                    if (dat.ToTickRecord().Timestamp.HasValue)
                    {
                        Console.WriteLine(
                            "Tick {0}: {1} {2} {3} ",
                            TimeSeriesRequestExample.instr,
                            dat.ToTickRecord().Timestamp,
                             dat.ToTickRecord().Value,
                              dat.ToTickRecord().Volume

                        );
                    };
                }
            }

Let us know how this works for you

rohit.agarwal.rg

Thanks Zoya for comparing the tick data using the excel and console. Can you please export the data in a file instead to console and write all 60000 data

if (chunk.Records.ToArray().Length != 0)

{

StreamWriter sw1 = new StreamWriter("a.csv");

foreach (IData dat in chunk.Records.ToArray())

{

if (dat.ToTickRecord().Timestamp.HasValue)

{

sw1.WriteLine(

"Tick {0}: {1} {2} {3} ",

"DIJF22",

dat.ToTickRecord().Timestamp,

dat.ToTickRecord().Value,

dat.ToTickRecord().Volume

);

};

}

sw1.Close();

}

I have compared it and has much difference to the excel data. Example, for 23 Jan, i can see around 5k records in excel but only around 1k records in file retrieved in .net api.

zoya faberov

Hello @rohit.agarwal.rg,

I think what you are saying is that on a small result set the difference does not manifest, while on 60K it does.

You have examined, via Formula Builder for Excel, the parameters that are going into Excel, and you confirm that you are applying exact same parameters in code as you do in Excel.

In order to eliminate any guessing on my part, and for me to use to do exact same test as you run, please:

• Include your 60k formula for Excel
• Your 60k .Net request code excerpt.
• Zip and attach both result sets you obtain.

I will try to review both, to try to identify the cause for the difference that you observe, and if I don't see any difference, run the same to try to reproduce.

rohit.agarwal.rg

DIJF22 TICK DATA FROM EXCEL & .NET.zip

rohit.agarwal.rg

TickDataToFileWinformsApp.zip

I have attached the data i got from .net and excel in previous answer and this annswer contains the c# project with the code to generate the file

You can see that .net export is having less than half of entries for todays date 27th jan 2020

zoya faberov

Hi @rohit.agarwal.rg,

I must be missing something.

I have looked at your Excel sheet. It requests 6000 points and retrieves 6000 points.

I have looked at your project. It request 6000 points and retrieves 6000 points?

Is it different for you? How many points do you retrieve?

rohit.agarwal.rg

I am requesting 6000 data points in excel and 6000 in .net. I am getting 6000 in excel and 6000 in .net. So far so good.

But the data are not same. There are less than 1000 data points for 27th jan in the file that was retrieved from .net and there are around 2000 data points for 27th jan in the file that was retrieved from excel RTD.

zoya faberov

Hi @rohit.agarwal.rg,

I retrieve over 2000 on 27th with my code.

I think I see what is different.

I think the difference you see has to do with parametrization, what is happening is Excel is using UTC timezone by default, and in code you explicitly use GMT in code, and also reverse the output.

To get the same result from .Net code, as you see in Excel, try comment out in code:

 private void Request()
        {
            string contract = "DIJF22";
            _request1 = _timeSeries.SetupDataRequest(contract)
                .WithView("TRDPRC_1")
                .WithAllFields()
                .WithInterval(CommonInterval.Tick)
                .WithNumberOfPoints(6000)
                .OnDataReceived(DataReceivedCallback)
//                .WithTimeZone(TimezoneType.GMT)
                
                .CreateAndSend();
        }

And in DataReceivedCalback try removing reverse:

foreach (IData dat in chunk.Records.ToArray())
                {
                    if (dat.ToTickRecord().Timestamp.HasValue)
                    {
                        Console.WriteLine(
                            "Tick {0}: {1} {2} {3} {4}",
                            TimeSeriesRequestExample.instr,
                            dat.ToTickRecord().Timestamp,
                             dat.ToTickRecord().Value,
                              dat.ToTickRecord().Volume,
                              count

                        );
                    };
                    count++;
                }

rohit.agarwal.rg

(NEW WITHOUT TIMEZONE)DIJF22 TICK DATA FROM EXCEL & .NET.zip

I have removed the timezone from my c# code and from the excel and reexported both files and attached them.

You can see both of them start at same timestamp with same data but further rows are completely different

zoya faberov

@rohit.agarwal.rg

Volumes appear to be the same for me as well

I will take a look at your attach,

Please try running the attached modified example code I used to generate this output?

Do you see the same?

Usage Example Time series API.modif.20200127.zip

rohit.agarwal.rg

I am not sure why are you comparing the data of 24th when i have shared you twice screenshots of discrepancies in 27th data (today's data)

Can you please export entire data for today 27th using both .net and excel in files and attach here

zoya faberov

Hi @rohit.agarwal.rg,

Your two results do differ I see that.

This time I have started out with your Excel and code, to see what we need to match.

Here is what I did

On DIJF22 TICK DATA FROM EXCEL RTD - Copy.xlsx

I replaced GMT with UTC, waited for an update and paused.

=RHistory("DIJF22","TRDPRC_1.Timestamp;TRDPRC_1.Value;TRDPRC_1.Volume","NBROWS:6000 ADJUSTED:NO TIMEZONE:UTC  INTERVAL:TICK",,"CH:Fd",B2)

Then I run your .Net request code, only with GMT commented, immediately.

They matched perfectly. If I would have taken a little more time, I may have had a couple of additional ticks, but the rest should have still matched.

Are you able to reproduce the same matching results, for all 6000 points, if you proceed the exact same way?

rohit.agarwal.rg

Please generate the export in a file for .net and one in Excel for entire 27th and share it here like I had done

zoya faberov

Hi @rohit.agarwal.rg

Please find attached:

DIJF22 TICK DATA FROM EXCEL RTD - Copy.zip

data.zip

Does this help?

rohit.agarwal.rg

Thanks for sharing the file, i will recheck everything and get back tomorrow

rohit.agarwal.rg

I have again tried with your solution but unfortunately it is not working in my machine, maybe because my machine time is not UTC so it is not able to take UTC by default when i am running the code. But the solution provided by jirapongse.phuriphanvichai is giving exact same result as excel. Thanks for the support.

rohit.agarwal.rg

Thanks jirapongse.phuriphanvichai

This is returning exact same result as excel and solved my problem.

Thanks again