directory retrieval failed

vsudhini

Trying to get index pricing from production server (ads1.trep-s.prod-inx.cityindex.co.uk:14002) fails with below "directory retrieval failed" error. This error occurs from when running in our kubernetes AKS cluster. Connecting to pre-production servers works fine. Also, connecting to prod server from development machine running on windows work fine.

emaLog_1.log:

ClientName: ChannelCallbackClient

Severity: Info Text: Received ChannelUp event on channel Channel_Index

Instance Name Consumer_Index_2

Component Version ads3.5.0.L1.linux.rrg 64-bit

loggerMsgEnd

WARN|: loggerMsg

ClientName: DirectoryCallbackClient

Severity: Warning Text: Received RDMDirectory status message without the state

loggerMsgEnd

WARN|: loggerMsg

ClientName: ChannelCallbackClient

Severity: Warning Text: Received ChannelDownReconnecting event on channel Channel_Index

Instance Name Consumer_Index_2

Reactor 60811181

Channel 10429724

Error Id SUCCESS

Internal sysError 0

Error Location Reactor.ProcessReactorEventImpl

Error text Ping error for channel: Lost contact with connection... Client channel has connection attempts left

loggerMsgEnd

WARN|: loggerMsg

ClientName: LoginCallbackClient

Severity: Warning Text: RDMLogin stream state was changed to suspect with status message

username <not set>

usernameType <not set>

State: Open/Suspect/None - text: ""

loggerMsgEnd

ERROR|: loggerMsg

ClientName: Consumer_Index_2

Severity: Error Text: directory retrieval failed (timed out after waiting 60000 milliseconds) for ads1.trep-s.prod-inx.cityindex.co.uk:14002)

loggerMsgEnd

Jirapongse

@vsudhini

I checked the PCAP file and found a lot of TCP Retransmission messages from 36900 -> 14002. That message could be the source directory request message. This indicates that the source directory request message couldn't be sent to the server. This could be a network issue.

wasin.w

Hello @vsudhini

It seems the API failed to get the Directory Refresh message from the ADS server on time (the default time is 45000 milliseconds).

You can configure the DirectoryRequestTimeOut property of the EmaConfig.xml file to change the Directory Request message timeout.

vsudhini

I tried setting the value to
0, now the application waits forever

Jirapongse

@vsudhini

You may need to enable XML trace on the consumer to verify the incomming and outgoing messages. For example:

        <Consumer>         
            <Name value="Consumer_1"/>         
            <Channel value="Channel_1"/>     
            <Dictionary value="Dictionary_1"/>
            <XmlTraceToStdout value="1"/>
        </Consumer>

vsudhini

trace.txt attached trace

vsudhini

I have network capture pcap file too, if you need

vsudhini

index-trace-full.txt full trace

vsudhini

index-trace-full.txt

Jirapongse

@vsudhini

I checked the full trace file and found the following behavior.

The API received the source directory refresh message at 18:20:33. I am not sure why it sent a new login request at 18:20:34. After that, it was unable to retrieve any source directory refresh messages from the server.

Please check the version of EMA used by the application. If the application uses the old version, please try to upgrade the application to use the latest version. You can download the APIs from the Software Downloads. The APIs are in the MDS - API product family.

Yes, please share the PCAP file. I will check which side reset the connections.

However, normally Developer Community Forum is for how-to/general questions but your question seems to be more complex and requires investigation e.g. review source code, configuration, try to reproduce the problem etc. I recommend you raise a query through the premium support service. If you are a named user of that service you should see a link "Contact Premium Support" when you are logged into the developer portal.

vsudhini

0003-pcap.zip attached pcap file. We are using LSEG.Ema.Core (3.2.0.1). That is the latest dll that is available on nuget. Will try the premium support. Thanks for the help so far.