cleanup() halts on rfa::common::EventQueue::destroy()

Unknown
Unknown
in RFA

After RFA C++ modified example StarterConsumer_BatchView has been running for a period of time, cleanup() halts on rfa::common::EventQueue::destroy()

How long does it take to destroy EventQueue? Is there any timeout?

Is there anything I can do to trace the destroy progress?

Tagged:

Best Answer

  • Steven McCoy
    Steven McCoy
    Answer ✓

    EventQueue::destroy() must not be called until all active streams have been completed. This is why the cleanup() function is organised as follows:

    // prevent new events from being added to the queue
    _pEventQueue->deactivate();
    // destroy all event consumers
    ...
    // finally destroy the event queue
    _pEventQueue->destroy();

    If all the event consumers have not terminated, or events are being used in multiple threads without synchronisation then the EventQueue::destroy() will exhibit undefined behaviour.

    It can be beneficial to wrap RFA objects with smart pointers and check the reference count before attempting to destroy.

    assert (event_queue_.use_count() <= 1);
    event_queue_.reset();

    I also like to use a leak tracker around objects with RFA references to provide useful feedback on the shutdown sequence.

Answers

  • Unknown
    Unknown

    This situation is not happening all the time, my test case is the same but it sometimes unexpectedly halts. Will it take time to destroy if the streaming runs for a period of time? e.g. If the streaming has run for a day, it will take 15 minutes to destroy in order to fully exit the program.

  • Jirapongse
    Jirapongse admin

    This may relate to the number of remaining events in the
    event queue and the size of memory used by the application.

    From our test, we have modified StarterConsumer to be
    a slow consumer. Therefore, the number
    of remaining events in the event queue was around 79000 events and the size of
    memory was growing to around 6 GB.

    image

    When cleaning up, it took around 8 minutes to exit the
    process.

    .....

    Dispatch Return: 79878

    Dispatch Return: 79877

    Dispatch Return: 79876

    Dispatch Return: 79875

    ********************** INFO ************************ INFO
    **********************

    2016 Apr 13 14:44:44.934 ST GMT+07:00 2634 26FC 332 INFO

    StarterConsumer Aborted
    (Control C) after attempting to dispatch msgs for 14

    2 seconds

    Exiting!.

    2016 Apr 13 14:44:53.866
    ST GMT+07:00 2634 26FC 629 TRACE
    ::cleanup().

    _pEventQueue->deactivate() is
    called.....##### 7:44:54.198 #####

    _pEventQueue->deactivate() return.....##### 7:44:54.216 #####

    _pEventQueue->destroy(); is called.....##### 7:45:22.601 #####

    _pEventQueue->destroy();return........##### 7:48:26.315 #####

    Context::uninitialize() is
    called...##### 7:48:27.529 #####

    Context::uninitialize() returns...#####
    7:52: 8.271 #####

    ********************** INFO ************************ INFO **********************

    2016 Apr 13 14:52:08.272
    ST GMT+07:00 2634 26FC 138 INFO

    Exiting StarterConsumer -
    Closing log file: StarterConsumer_9780.log.

    Press any key to continue . . .

    ......

    You may need to verify the total memory used by the application. If it is very high, the application may be a slow consumer and it will take time to clean up.

  • Steven McCoy
    Steven McCoy

    @Haseuser destroy will only take time if the application has been a slow consumer and buffered many thousands of messages that each need to be inspected. By monitoring the application memory usage one should be able to determine whether the application has been building up a backlog due to insufficient processing performance.

Categories