[EMA Consumer JAVA RTO] connection to aws endpoint fails with a valid credential

DimAngelNX

Hello team,

I am facing an issue when connecting to RTO using EMA, below is my code:

          Map elementMap = EmaFactory.createMap();        ElementList elementList = EmaFactory.createElementList();        ElementList innerElementList = EmaFactory.createElementList();         innerElementList.add(EmaFactory.createElementEntry().ascii("Channel", "Channel_1"));        elementMap.add(EmaFactory.createMapEntry().keyAscii("Consumer_1", MapEntry.MapAction.ADD, innerElementList));        innerElementList.clear();         elementList.add(EmaFactory.createElementEntry().map("ConsumerList", elementMap));        elementMap.clear();         Map configDb = EmaFactory.createMap();        configDb.add(EmaFactory.createMapEntry().keyAscii("ConsumerGroup", MapEntry.MapAction.ADD, elementList));        elementList.clear();         innerElementList.add(EmaFactory.createElementEntry().ascii("ChannelType", "ChannelType::RSSL_WEBSOCKET"));  // discoveryClient is of ServiceEndpointDiscoveryClient type implementing onSuccess()// This client was registered using the same clientId & secret as bellow and the endpoint // was successfully discovered        innerElementList.add(EmaFactory.createElementEntry().ascii("Host", discoveryClient.getHost()));        innerElementList.add(EmaFactory.createElementEntry().ascii("Port", discoveryClient.getPort()));        innerElementList.add(EmaFactory.createElementEntry().intValue("EnableSessionManagement", 1));         elementMap.add(EmaFactory.createMapEntry().keyAscii("Channel_1", MapEntry.MapAction.ADD, innerElementList));        innerElementList.clear();         elementList.add(EmaFactory.createElementEntry().map("ChannelList", elementMap));        elementMap.clear();         configDb.add(EmaFactory.createMapEntry().keyAscii("ChannelGroup", MapEntry.MapAction.ADD, elementList));        elementList.clear();         try {
                OmmConsumerConfig subscriptionConfig = EmaFactory.createOmmConsumerConfig(); // And finally create the consummer            OmmConsumer subscriptionRTOConsumer = EmaFactory.createOmmConsumer(subscriptionConfig.consumerName("Consumer_1")                    .clientId(XXXX)                    .clientSecret(XXXXXXX)                    .tokenServiceUrlV2("api.refinitiv.com/auth/oauth2/v2/token")                    .config(configDb)); 

On the consumer creation, I have many login retry errors like this:

  2023-12-20 13:19:17:566 ERROR main access.OmmConsumerImpl:675 - loggerMsg    ClientName: ChannelCallbackClient    Severity: Error    Text:    Received ChannelDown event on channel Channel_1    Instance Name Consumer_1_1    RsslReactor @2dca0d64   RsslChannel @7b84fcf8   Error Id 0    Internal sysError 0    Error Location Reactor.processWorkerEvent    Error text Error - exceeded initialization timeout (5 s)loggerMsgEnd

And the final error:

  2023-12-20 13:26:17:798 ERROR main refinitiv.RefinitivRTOSource:189 - Cannot start connector for client Id [XXXXX], invalid usage: login failed (timed out after waiting 45000 milliseconds) for eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443)

Any help would be appreciated,

Many thanks

wasin.w

Hello @DimAngelNX

The "login failed (timed out after waiting 45000 milliseconds) for eu-central-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443)" login error message indicates that the API cannot establish the WebSocket connection and sends the WebSocket Login request message to the eu-central-1-aws-3-sm.optimized-pricing-api.refinitiv.net WebSocket server on port 443. The error message is not related to the Authentication token.

The service discovery uses the HTTP connection (GET /streaming/pricing/v1/ HTTP/1.1) connects to the RDP API gateway (https://api.refinitiv.com), not the WebSocket connection.

Like @Jirapongse suggestion, the WebSocket connection may be blocked by a firewall or network policy.

Jirapongse

@DimAngelNX

Thank you for reaching out to us.

You need to compare the code with the ex450_MP_QueryServiceDiscovery example.

You can run it with the following parameters.

$>gradlew.bat runconsumer450 --args="-clientId <ClientID> -clientSecret <ClientSecret> -itemName EUR="

According to the log, it looks like that the application uses the WebSocket connection (TCP 443).

2023-12-20 13:26:17:798 ERROR main refinitiv.RefinitivRTOSource:189 - Cannot start connector for client Id [XXXXX], invalid usage: login failed (timed out after waiting 45000 milliseconds) for eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443)

RTO supports both the encrypted WebSocket (443) and encrypted TCP (14002) connection types.

Please check the connection type that you want to connect.

wasin.w

Hello @DimAngelNX

The configDB code is missing some statements for the RTO-WebSocket connection as follows:

• You need to specify Dictionary node to DictionaryType::FileDictionary type for loading field dictionary information from local files (the WebSocket connection cannot load dictionary information from the server)
• The ChannelType must be "ChannelType::RSSL_ENCRYPTED"
• You need to add "EncryptedProtocolType" node with "EncryptedProtocolType::RSSL_WEBSOCKET" value
• You need to add ""WsProtocols"" node with "tr_json2" value

Example Source Code (see the italic bold lines):

Map elementMap = EmaFactory.createMap();
ElementList elementList = EmaFactory.createElementList();
ElementList innerElementList = EmaFactory.createElementList();

innerElementList.add(EmaFactory.createElementEntry().ascii("Channel", "Channel_1"));

innerElementList.add(EmaFactory.createElementEntry().ascii("Dictionary", "Dictionary_1"));

elementMap.add(EmaFactory.createMapEntry().keyAscii("Consumer_1", MapEntry.MapAction.ADD, innerElementList));
innerElementList.clear();

elementList.add(EmaFactory.createElementEntry().map("ConsumerList", elementMap));
elementMap.clear();
Map configDb = EmaFactory.createMap();
configDb.add(EmaFactory.createMapEntry().keyAscii("ConsumerGroup", MapEntry.MapAction.ADD, elementList));
elementList.clear();
innerElementList.add(EmaFactory.createElementEntry().ascii("ChannelType", "ChannelType::RSSL_ENCRYPTED"));
innerElementList.add(EmaFactory.createElementEntry().ascii("EncryptedProtocolType", "EncryptedProtocolType::RSSL_WEBSOCKET"));
innerElementList.add(EmaFactory.createElementEntry().ascii("WsProtocols", "tr_json2"));

// discoveryClient is of ServiceEndpointDiscoveryClient type implementing onSuccess()
// This client was registered using the same clientId & secret as bellow and the endpoint
// was successfully discovered
innerElementList.add(EmaFactory.createElementEntry().ascii("Host", {host}));
innerElementList.add(EmaFactory.createElementEntry().ascii("Port", {port}));
innerElementList.add(EmaFactory.createElementEntry().intValue("EnableSessionManagement", 1));

elementMap.add(EmaFactory.createMapEntry().keyAscii("Channel_1", MapEntry.MapAction.ADD, innerElementList));
innerElementList.clear();

elementList.add(EmaFactory.createElementEntry().map("ChannelList", elementMap));
elementMap.clear();

configDb.add(EmaFactory.createMapEntry().keyAscii("ChannelGroup", MapEntry.MapAction.ADD, elementList));
elementList.clear();

innerElementList.add( EmaFactory.createElementEntry().ascii( "DictionaryType", "DictionaryType::FileDictionary" ));
innerElementList.add( EmaFactory.createElementEntry().ascii( "RdmFieldDictionaryFileName", "C:\\etc\\RDMFieldDictionary" )); <-- must be match your local folder
innerElementList.add( EmaFactory.createElementEntry().ascii( "EnumTypeDefFileName", "C:\\etc\\enumtype.def" )); <-- must be match your local folder
elementMap.add( EmaFactory.createMapEntry().keyAscii( "Dictionary_1", MapEntry.MapAction.ADD, innerElementList ));
innerElementList.clear();

elementList.add( EmaFactory.createElementEntry().map( "DictionaryList", elementMap ));
elementMap.clear();
configDb.add( EmaFactory.createMapEntry().keyAscii( "DictionaryGroup", MapEntry.MapAction.ADD, elementList ));
elementList.clear();
...

You can check the ex450_MP_QueryServiceDiscovery example as suggested by my colleauge.

DimAngelNX

Hello @wasin.w ,

Thank you for your reply.

I've already tried to use encrypted channel type like this:

innerElementList.add(EmaFactory.createElementEntry().ascii("ChannelType", "ChannelType::RSSL_ENCRYPTED"));
innerElementList.add(EmaFactory.createElementEntry().ascii("EncryptedProtocolType", "EncryptedProtocolType::RSSL_WEBSOCKET"));

But I have an assertion error comming from EMA when registering the OmmConsumerClient:

java.lang.AssertionError: KeystorePasswd must be non-null
    at com.refinitiv.eta.transport.EncryptionOptionsImpl.KeystorePasswd(EncryptionOptionsImpl.java:145)
    at com.refinitiv.ema.access.ChannelCallbackClient.readEncryptedChannelConfig(ChannelCallbackClient.java:1566)
    at com.refinitiv.ema.access.ChannelCallbackClient.tunnelingConfiguration(ChannelCallbackClient.java:1399)
    at com.refinitiv.ema.access.ChannelCallbackClient.channelConfigToReactorConnectInfo(ChannelCallbackClient.java:904)
    at com.refinitiv.ema.access.ChannelCallbackClient.initializeReactor(ChannelCallbackClient.java:1045)
    at com.refinitiv.ema.access.ChannelCallbackClient.initializeConsumerRole(ChannelCallbackClient.java:1304)
    at com.refinitiv.ema.access.OmmConsumerImpl.handleAdminDomains(OmmConsumerImpl.java:632)
    at com.refinitiv.ema.access.OmmBaseImpl.initialize(OmmBaseImpl.java:346)
    at com.refinitiv.ema.access.OmmConsumerImpl.<init>(OmmConsumerImpl.java:47)
    at com.refinitiv.ema.access.EmaFactory.createOmmConsumer(EmaFactory.java:237)

Any suggestions?

Jirapongse

@DimAngelNX

I got the same error when enable assertions (-ea) in JVM

Exception in thread "main" java.lang.AssertionError: KeystorePasswd must be non-null
    at com.refinitiv.eta.transport.EncryptionOptionsImpl.KeystorePasswd(EncryptionOptionsImpl.java:145)
    at com.refinitiv.ema.access.ChannelCallbackClient.readEncryptedChannelConfig(ChannelCallbackClient.java:1566)
    at com.refinitiv.ema.access.ChannelCallbackClient.tunnelingConfiguration(ChannelCallbackClient.java:1399)
    at com.refinitiv.ema.access.ChannelCallbackClient.channelConfigToReactorConnectInfo(ChannelCallbackClient.java:904)
    at com.refinitiv.ema.access.ChannelCallbackClient.initializeReactor(ChannelCallbackClient.java:1045)
    at com.refinitiv.ema.access.ChannelCallbackClient.initializeConsumerRole(ChannelCallbackClient.java:1304)
    at com.refinitiv.ema.access.OmmConsumerImpl.handleAdminDomains(OmmConsumerImpl.java:632)
    at com.refinitiv.ema.access.OmmBaseImpl.initialize(OmmBaseImpl.java:346)
    at com.refinitiv.ema.access.OmmConsumerImpl.<init>(OmmConsumerImpl.java:47)
    at com.refinitiv.ema.access.EmaFactory.createOmmConsumer(EmaFactory.java:237)
    at com.refinitiv.ema.examples.training.consumer.series400.ex450_MP_QueryServiceDiscovery.Consumer.main(Consumer.java:474)

It works fine if I disable assertions.

If you are a Refinitiv Developer Connect (RDC) named user, you can submit this issue via Contact Premium Support. Otherwise, you can raise this issue on GitHub.

DimAngelNX

Hello @Jirapongse ,

Thank you for your reply, disabling assertions fixed this particular issue.

But after considering all comments above, my code still fails with the following errors:

2024-01-02 09:55:53:489  WARN main access.OmmConsumerImpl:619 - loggerMsg
    ClientName: ChannelCallbackClient
    Severity: Warning
    Text:    Received ChannelDownReconnecting event on channel Channel_2
RsslReactor @377c68c6
RsslChannel @538cd0f2
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Error - exceeded initialization timeout (5 s)
loggerMsgEnd


2024-01-02 09:56:01:316 ERROR main access.OmmConsumerImpl:1717 - loggerMsg
    ClientName: Consumer_2_1
    Severity: Error
    Text:    login failed (timed out after waiting 45000 milliseconds) for eu-central-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443)
loggerMsgEnd


2024-01-02 09:56:01:319 ERROR main access.OmmConsumerImpl:675 - loggerMsg
    ClientName: ChannelCallbackClient
    Severity: Error
    Text:    Received ChannelDown event on channel Channel_2
Instance Name Consumer_2_1
RsslReactor @377c68c6
RsslChannel @22df874e
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Error - exceeded initialization timeout (5 s)
loggerMsgEnd

I suspect this is a token issue, althought use V2 authentication with

<EnableSessionManagement value="1"/>

Any suggestions?

Thank you

Jirapongse

@DimAngelNX

The error indicates the the API was unable to establish a connection to the server (eu-central-1-aws-3-sm.optimized-pricing-api.refinitiv.net:443). The connection may be blocked by a firewall or network policy. You need to contact your IT support team to verify the connection.

Otherwise, you can try other servers.

DimAngelNX

@Jirapongse

My service discovery client successfully manages to connect whit the clientId and secret and discovers the aws hosts, additionnally I checked the connection with my IT team and it is OK, so I suppose no problem on connection side.

But as the log posted above is saying login failed, I suppose there is a problem with auth token?

DimAngelNX

Hello @Jirapongse , @wasin.w ,

After analysis fro mthe network team here is what I have on teh network log:

Connection terminated before the Security Gateway was able to make a decision: Insufficient data passed.

Any suggestions on what could be the problem?

Thanks a lot

Jirapongse

According to the network log, it should not relate to the API.

You need to the contact the firewall support team to verify this error.