RFA API behavior when application thread is blocked during market Data processing

mktdata
mktdata
in RFA

Hello, we are using the RFA 8.2 C++ API to consume market data. In our application, the Driver Thread utilizes the RFA libraries to request data and receive callbacks from TREP through a TCP connection. If the Driver Thread becomes blocked and is unable to process incoming data, will the RFA libraries continue to receive and buffer data until the Driver Thread can resume processing, or will they stop receiving data from TREP if the Driver Thread is blocked? What is the expected behavior in this situation?

Tagged:

Answers

  • Gurpreet
    Gurpreet

    Hello @mktdata,

    The application behavior will depend on the data load on the application. There are internal buffers within the SDK and RTMDS which will start filling up, if the app thread does not empty the received messages fast enough. If this continues for too long, these buffers will eventually overflow, and the infrastructure will disconnect the offending consumer app.

    This scenario is called a slow consumer.

    If however, the data update rate is low, and the driver thread is blocked rarely, the inbuilt buffers should be able to handle this temporary disruption.

  • mktdata
    mktdata
    Hello
    @Gurpreet , how big these buffers can grow in size at max before consumer app is disconnected? I am not looking for exact number but just indicative figures. Will that grow in GB's?
  • Gurpreet
    Gurpreet

    The buffers in the RFA and the RTMDS are configurable. You can read about these parameters in the RFA configuration and logging guide, packaged with the SDK. Your market data admin should be able to configure the buffers on the infrastructure side.

    Important thing to note is that - if the consumer is really slow and unable to process messages fast enough, then no amount of buffers can fix this situation. Having very large buffers will increase the latency in the application update rate, and the consumer will disconnect after a bit of time anyways.

  • Jirapongse
    Jirapongse

    @mktdata

    Thank you for reaching out to us.

    It depends on the thread model that you are using. Please check the section 17.4 Configuring the RFA Consumer for Performance in the RFA developer guide.

    If the application uses Single Threaded With an Event Queue (Client Model), a RFA thread will push events in to an event queue while an application thread is busy or blocked.

Categories