EMA - skip to latest data when consumer is slow

otto_christianto
otto_christianto
in EMA

Hi, Im using EMA to stream some forex data (about 80 in total), we discover that it seems our consumer is too slow.

For example, when streaming JPY= the received data timestamp eventually will differ far from our server time after some minutes.

image001.png

From what I understand this is due to huge amount of buffered data that yet to be consumed by the consumer. When consumer finally get it, the data is already in the buffer for several minutes.

Is there any way to 'skip' the buffered data and get the latest data instead?

Tagged:

Best Answer

  • Gurpreet
    Gurpreet
    Answer ✓

    Hello @otto_christianto,

    Its hard for an application to fall back by 9 minutes on processing - unless the app code is sleeping or locked waiting for some network resources. Its not a case of slow-consumer, otherwise ADS would have disconnected this application long time ago.

    I can't fathom, how the buffered events can be processed this late by the application code. I would recommend that you check if NTP sync etc is not throwing off the local clock, and do the application code review.

    Since the app and the buffered events are under the app's control, you can optimize it and process the updates at a faster rate. Real time streaming data for 80 FX instruments is not a lot of data.

Answers

  • wasin.w
    wasin.w admin

    Hello @otto_christianto

    Can you replicate the issue on demand? If so, I suggest you enable the API's XML trace messages, replicate the issue and share the request-response messages on this post. The XML trace messages contain incoming/outgoing messages between the API and server including the timestamp.

    This information will let you check the time that the API sent a request message and the time that the API received data from the Server.

    You can enable the XML Trace by setting <XmlTraceToStdout value="1"/> in the EmaConfig.xml file like the following example:

    <Consumer>
        <Name value="Consumer_1"/>
        <Channel value="Channel_1"/>
        <Dictionary value="Dictionary_2"/>
        <XmlTraceToStdout value="1"/>
    </Consumer>

    Another question is can you replicate the issue with the EMA Java example (like ex100_MP_Streaming or ex370_MP_Batch) from the same machine and connects to the same RTDS server?


Categories