VAConsumer application curl issue

andy.sciascia

Hi. I'm updating our VA Consumer application to connect to RTO. I'm following the example VAConsumer code in the RTSDK - here.

I can run the RTSDK VAConsumer example and connect and get data so I know I have connectivity and the right credentials etc.

But my application keeps failing with this log

Failed to request authentication token information. Text: </home/build/workspace/Refinitiv-Poller-Build-EL6/Elektron-SDK/Cpp-C/Eta/Impl/Reactor/Util/rsslRestClientImpl.c:1521> Error: Failed to set CURL options with text: Failed initialization)

I set up my application to create a Rest log and I get this ( I've hidden credential info but it's the same as I use when I run the example )

REST log redirected to file.


--- REST REQUEST ---


<!-- Time: 13:28:22:252 -->
URL: https://api.refinitiv.com/auth/oauth2/v1/token
HTTP method POST
HTTP header data:
 Accept : application/json
 Content-Type : application/x-www-form-urlencoded
HTTP body data: grant_type=password&username=GE-A-XXXXXXXX&client_id=XXXXXXXXXXX&scope=trapi.streaming.pricing.read&takeExclusiveSignOnControl=true
Request timeout: 90


--- REST RESPONSE ---


<!-- Time: 13:28:22:252 -->
HTTP header data:
HTTP body data: (null)
Protocol version: (null)
HTTP status code: 0

I've set up curl using libcurl.so that comes with the RTDSDK

initOpts.jitOpts.libcurlName = "/home/andys/clones/Real-Time-SDK/buildDir/install/lib64/libcurl.so";

I know this libcurl.so should work because to get the example app to work I set LD_LIBRARY_PATH to include the path to that library to get it to work.

Any ideas what could be going wrong ? I've gone through the example line by line and I'm pretty sure I have the same setup but I must be doing something differently.

andy.sciascia

I've finally fixed this by building my own libcurl.so that links to OpenSSL 1.1.1m and making sure that it is found first in the load path.

It seems odd that the library provides a libcurl.so that links to OpenSSL 1.0 when the library itself specifically tries to load OpenSSL 1.1 if that is available in ripcInitializeSSL().

nick.zincone

Hi @andy.sciascia

Instead of using your updated VAConsumer, have you tried running the packaged VAConsumer for a lengthy amount of time? For example, I used the following:

> VAConsumer -encryptedSocket us-east-2-aws-3-med.optimized-pricing-api.refinitiv.net:14002 ELEKTRON_DD mp:IBM.N 
             -uname GE-A-XXXXXXXX-9 
             -passwd PASSWD 
             -clientId f612345678901234567890 
             -sessionMgnt 
             -runtime 3600

The example ran for an hour without issue.

For your application, can you provide more details such as when the error occurred? I want to determine if this error occurs about 10 minutes after you start? As you can see above, I've extended the -runtime to 1 hour from the default of 5 minutes to ensure the test I did goes through a proper token refresh.

andy.sciascia

Hi. I should have been clearer. My application never connects. The issue happens straight away.

The REST log indicates that my app attempted to get information from the Discovery server - right ? Is there any reason that the response would be NULL ? Is there anything missing in the POST ?

I've never run the example app for long but I've started it several times and it always connects and gets data.

I'll try that and update later on.

nick.zincone

Hi @andy.sciascia,

Ok - thanks for the clarification. Just to confirm, you can run the VAConsumer within the package but after you modified it, it fails? Or does the example version you linked to fails? I'm not clear on whether you updated it or not. I downloaded the release from the developer community and compiled the VAConsumer within that package.

I compared the POST, and you can too, with the packaged example and it is identical. I would start with the packaged version within the download package and build from there.

andy.sciascia

Hi. I built the VAConsumer using Refinitiv instructions.

My application is based on the old Consumer application. I'm updating it to handle RTO. I've compared my application line by line to the VAConsumer to make sure I'm setting up the same connection information and options. I've even run gdb on both applications to make sure that the information set up in these structures is identical when the connection call is made.

I'm specifically interested in why curl would return the error I listed above in the log.

Failed to set CURL options with text: Failed initialization 

This is a different error to the one that appears if JIT can't find curl. I can get VAConsumer to give me that error by not setting up LD_LIBRARY_PATH

Error rsslReactorConnect(): Failed to initialize RsslRestClient. Text: </opt/refinitiv/Real-Time-SDK/Cpp-C/Eta/Impl/Util/rsslCurlJIT.c:77> Error: 0012 Libcurl intialization failed.  Curl library: libcurl.so not found.

I tried breaking my app by giving it a bad libcurl location in the JIT options but I still get that "Failed initialisation" error. I'm beginning to suspect that my app is picking up libcurl.so from somewhere else.

Is the libcurl.so in the RTSDK special in any way ? Should consumer applications work even with a system libcurl ?

nick.zincone

Hi @andy.sciascia

Can you do a:

> which curl
> curl --version

Is curl in your path?

The installdb directory should contain external libs (including libcurl) and you should ensure your LD_LIBRARY_PATH is updated. You may need to update your PATH to point to the installdb dir.

andy.sciascia

I've made some progress. The application was picking up the system libcurl.so. I copied the Refinitiv one into /usr/lib64 and I finally got past that issue - onto the next one.

This seems to be a Refinitv bug. In my code and in the example we use rsslInitializeEx() to set the JIT libcurl location but it doesn't work.

I ran the VAConsumer example with the option

-libcurlName wrong 

to try to force it to fail with a bad libcurl location but it still worked if LD_LIBRARY_PATH was set.

I think when the Reactor create function is called it calls rsslInitializeEx() with default options which undoes the JIT libcurl.

Can you try running your VAConsumer and specify -libcurlName and see what happens. If you specify a bad location it should report it.

I'm now getting this

2022/02/23-22:34:15.573100 +0000: (upasrc-load-0): ERROR: ETA: Connection error, will try again shortly: (Failed to request authentication token information. Text: </home/build/workspace/Refinitiv-Poller-Build-EL6/Elektron-SDK/Cpp-C/Eta/Impl/Reactor/Util/rsslRestClientImpl.c:1595> Error: Failed to perform the request with text: Couldn't use specified SSL cipher)

I installed OpenSSL 1.1.1 because I had an old OpenSSL previously. On the system I run the VAConsumer with it's working with 1.0.2k

andy.sciascia

I've raised an issue about the libcurl problem.

My application is now using the Refinitiv libcurl.so because I've put moved the old library and put the Refinitiv one in its place.

I'm now getting the Cipher issue I mentioned in the last comment.

Is there a specific version of OpenSSL that is required ? I'm using 1.1.1m with my application.

nick.zincone

Hi @andy.sciascia

Thanks for the update.

I did try a similar command to specify nonsense for my libcurl using -libcurlName flag and it appears VAConsumer silently ignored it and reverted to the version found in LD_LIBRARY_PATH.

As for SSL, according to the readme, they "recommend that all installed versions of OpenSSL are patched to the latest version available.". If you have not viewed this readme, it will provide details on all the external dependencies.

andy.sciascia

Hi. I've raised an issue on GitHub for the JIT library options. Looks like a library issue. Re OpenSSL. I'm using 1.1.1m which is later that the 1.1 version in that
README.md. I think that the library is loading the SSL libraries from some other location although I have removed all instances of them except the new 1.1.1 versions. Is that SSL error something that would be associated with loading the wrong version ?

andy.sciascia

Just double checked on my machine and I've - removed every
libcrypto.so and
librssl.so from my VM except the 1.1.1m versions - made sure they were 1.1.1m by running a strings check - set up my LD_LIBRARY_PATH to point only at the directory they are in And I still get the SSL error.

nick.zincone

Hi @andy.sciascia

I noticed you posted a question within the GitHub site regarding libcurl. I would also suggest you do the same for openssl. I just want to confirm if you are still able to "run the RTSDK VAConsumer example and connect and get data"? Or is this specific to your application that is based on the old Consumer application?

I bring this up because if the former works, then to move forward, it might make sense to start with the one that is working and strip out the stuff you don't need. Obviously, I understand we are interested as to why it doesn't work.

andy.sciascia

Hi. I'm running the VAConsumer example on the Oracle 7 Docker that is documented by Refnitiv. I'm trying to get my application running on Cent OS 6 and Cent OS 7 and I'm getting the same curl issue. The issue is with ciphers which are not handled by the application. I think
libcurl.so is using SSL libraries itself and there is some problem there. On both Cent OS 6 and 7 the SSL libraries are later versions than required by Refinitiv. I am going to try what you suggested to double check my code but I have stepped through and I know its failing the authentication curl call to the Discovery server with the Cipher issue and when that call happens I examined the request arguments and they are all correct.

andy.sciascia

Also, just to confirm that my versions are correct. On EL7 I'm using the libraries shown below - so I can see that Refinitiv libcurl is using OpenSSL 1.0.2k. Also, I'm positive I'm using Refinitiv curl because if I unset the LD_LIBRARY_PATH the binary logs that it can't load
libcurl.so. ``` echo $LD_LIBRARY_PATH /home/build/clones/Real-Time-SDK/build/install/lib64:/lib64 [build@jenkins-cent7-001: ~/clones/cdev/datasrcs/sslsrc/acctests](PDSRT-216)$ find $LD_LIBRARY_PATH -name
libcurl.so /home/build/clones/Real-Time-SDK/build/install/lib64/
libcurl.so [build@jenkins-cent7-001: ~/clones/cdev/datasrcs/sslsrc/acctests](PDSRT-216)$ ldd /home/build/clones/Real-Time-SDK/build/install/lib64/
libcurl.so linux-vdso.so.1 => (0x00007ffd64dec000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f68f81ae000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f68f7f92000) libssl.so.10 => /lib64/libssl.so.10 (0x00007f68f7d20000) libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f68f78bd000) libz.so.1 => /home/build/clones/Real-Time-SDK/build/install/lib64/libz.so.1 (0x00007f68f76a6000) libc.so.6 => /lib64/libc.so.6 (0x00007f68f72d8000) /lib64/ld-linux-x86-64.so.2 (0x00007f68f8613000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f68f708b000) libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f68f6da2000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f68f6b9e000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f68f696b000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f68f675b000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f68f6557000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f68f633d000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f68f6116000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f68f5eb4000) [build@jenkins-cent7-001: ~/clones/cdev/datasrcs/sslsrc/acctests](PDSRT-216)$ readlink -e /lib64/libcrypto.so.10 /usr/lib64/libcrypto.so.1.0.2k [build@jenkins-cent7-001: ~/clones/cdev/datasrcs/sslsrc/acctests](PDSRT-216)$ readlink -e /lib64/libssl.so.10 /usr/lib64/libssl.so.1.0.2k ```

andy.sciascia

Sorry about formatting of last comment. I can't reliably log into this site using Chrome. I can log in reliably using IE and I can't edit comments.

andy.sciascia

I have just ported my application code into the VAConsumer example and it works. I changed my logging to print to the terminal and I can see that it connects

ETA: Transport Library Version: PACKAGE eta3.6.4.L1 rrg VERSION eta3.6.4.0 Tue Feb 15 17:33:06 CST 2022 ETA: Connect callback called 
ETA: Connecting to eu-west-1-aws-3-sm.optimized-pricing-api.refinitiv.net:14002 ETA Reactor: Got to authTokenEventCallback 
ETA Reactor: Channel open Callback, start dictionary request from service ELEKTRON_DD 
ETA Reactor: Requesting [00003] TREP channel ((nil)) domain (RSSL_DMT_DICTIONARY) 
ETA Reactor: Requesting [00004] TREP channel ((nil)) domain (RSSL_DMT_DICTIONARY) 
ETA: Connection return code 0 ETA: Name: RWFFld,Domain: RSSL_DMT_DICTIONARY, stream: [00003] Msg Class: RDM_DC_MT_STATUS, 
ETA: Name: RWFEnum,Domain: RSSL_DMT_DICTIONARY, stream: [00004] Msg Class: RDM_DC_MT_STATUS, ETA Reactor: Connection up: Channel (0x15ea210) fd=9 
ETA Reactor: Connection up: Connected to ads3.4.2.L1.linux.tis.rrg 64-bit device. 
ETA Reactor: Connection ready: Channel (0x15ea210) 
ETA Reactor: Login state ETA Reactor: Dictionary received type 1 stream id [00003] 
ETA Reactor: Dictionary received Response: RWFFld 
ETA Reactor: Dictionary Field dictionary version 4.20.42 load complete, 15595 entries 
ETA Reactor: Dictionary received type 2 stream id [00004] 
ETA Reactor: Dictionary received Response: RWFEnum ETA Reactor: Dictionary Enum dictionary version 4.20.42 load complete, 617 entries

nick.zincone

Hi @andy.sciascia

Thanks for the update.

andy.sciascia

I've ported the cut down app back to my EL6 server and connected it to RTO. It didn't work but then I created a /usr/local/ssl/cert.pem file with the same contents as on the Oracle server and I connected to RTO from my Linux VM.

My main application still doesn't work. The issue is that my application statically links to OpenSSL 1.1.1m. I can hack my app to removed the OpenSSL bits and not link and I can see what's happening

- If I'm linked statically to OpenSSL 1.1.1m then the curl call in the ETA library finds the static 1.1 SSL_CTX_new() in the binary and I get the Cipher issue

- If I'm not linked statically to OpenSSL 1.1.1m then the curl call finds the SSL_CTX_new() in the dynamically loaded 1.0 OpenSSL it depends on and I don't get a cipher issue

Anyone using the ETA libraries in an application statically linked to OpenSSL will probably get this issue.