Why Connection to electron service keeps dropping every 15-20mins

vuk.magdelinic

Cannot download quickstart example as mentioned in earlier comment above. But looked at the code and its very similar to what we have, our code is below:

MarketPrice theMarketPrice;            do {
                    for (String str : values) {
                        theMarketPrice = new MarketPrice.Builder().withOmmConsumer(ommConsumer).withName(str)
                                .withServiceName(SERVICE_NAME).withSynchronousMode(AUTO_DISPATCH).build();
                        theMarketPrice.open();                        //System.out.println(theMarketPrice.toString());
                        jsonResponse = createJsonObject(theMarketPrice.getField("DSPLY_NAME").value().toString(),
                                theMarketPrice.getField("BID").value().toString(),
                                theMarketPrice.getField("ASK").value().toString(),
                                theMarketPrice.getField("CTBTR_1").value().toString(),
                                theMarketPrice.getField("VALUE_TS1").value().toString(),
                                theMarketPrice.getField("MID_PRICE").value().toString());                        //System.out.println(jsonResponse.toString(1));
                        logger.debug(jsonResponse.toString(1));                        readRecordsAndSubmitToKPL(jsonResponse);                        theMarketPrice.close();
                    }            } while (true);

zoya faberov

Hello @vuk.magdelinic,

The code looks reasonable, however, troubleshooting custom code will not be reasonable in the framework of the discussion forum.

Do you have the complete SDK? SDK can be downloaded from Downloads, you will need to be logged in and may need to "request download" the first time.

Otherwise, SDK is open source and so the complete source code can also be downloaded from RTSDK on GitHub and built from source according to Installation Guide found in docs.

The example, in both cases, will be in ... \Ema\Examples, the example of interest, ServiceDiscovery, is com\refinitiv\ema\examples\training\consumer\series400\ex450_MP_QueryServiceDiscovery

Let us know how this works for you

vuk.magdelinic

I have looked into this example but I cant find the below package/class being used: Pls advise since this is the class we are using to fetch data

package com.refinitiv.platformservices.elektron.objects.marketprice;

zoya faberov

Hello @vuk.magdelinic,

Our examples use Refinitiv libraries that are part of RTSDK:

• EMA, ease of use tier
• ETA, lower level access, transport tier

Have you been able to run the example successfully as described in Quickstart guide?

---

There was an influential article published, believe that it was at the introduction of Elektron SDK (currently Relatime SDK) EMA libraries, A simple MarketPrice object for EMA describing how to build value-add objects on top of EMA.

There was companion code included with the article, to illustrate the concepts that were explained by the author in the article, hosted on GiHub, that was last updated by the author 2 years ago, major updates 3 years ago, from what I see.

I read in Readme on GitHub that the article code is tested with EMA 1.1.0. This is a very old version and pre-dates not just the latest major version of RTSDK 2.0, but several versions before it.

I would suggest using the latest EMA from RTSDK 2.0 that you mention you use, directly, since the publication of the article, many issues have been fixed, many new features became available, and this is what you wish to use.

The techniques demonstrated in the article remain valid, and can be used for creating value- added objects, as suggests at the end "It should give you good indications for implementing the same kind of features in your own applications", however would not suggest using the companion code as is, would at least need to verify the parts that you wish to use with the current version of RTSDK prior to doing so, in my view.

vuk.magdelinic

Thanks for your response! If I skip using the value-add objects, then I would need to find the FID values for the fields I'm pulling. can you please help me where I can get those?

For ex -

theMarketPrice.getField("MID_PRICE").value().toString());   

Instead of the above, I would need to do something like this right?

OmmArray array = EmaFactory.createOmmArray();
            array.fixedWidth(2);
            array.add(EmaFactory.createOmmArrayEntry().intValue(???-what should go here?))

chavalit-jintamalit

You can locate the data dictionary files under \Real-Time-SDK-2.0.1.L1.java\RTSDK-2.0.1.L1.java.rrg\Java\etc folder.

From RDMFieldDictionary:

MID_PRICE  "MID-PRICE"            134  NULL        PRICE              17  REAL64           7

I assume that you are trying to use "View" feature when you make a subscription.

Please refer to example 360.

The "View" feature limits the fields in the refresh or update message that your application receives regarding the data subscription.

You can add more fields and you make a subscription:

vuk.magdelinic

@chavalit.jintamalit do you know why when I use EMA directly as explained by you, I dont seem to get data for many of the fields but using ValueAddObject interface gets me data for those fields?

chavalit-jintamalit

Hi @vuk.magdelinic

You provided a snippet code which is using "view" feature.

So I was assuming that you tried to achieve what "view" feature does.

When you refer to the sample code from EMA, it is suggested that you also review the readme file for each sample as well.

Summary
=======

The ex360_MP_View application is provided as an example of OMM Consumer
application written to the EMA library.

This application demonstrates basic usage of the EMA library for opening item
stream with a list of field IDs called "View" and parsing of OMM MarketPrice
data from Refinitiv Data Feed Direct (RDF-D), directly from an OMM Provider
application, or from an Advanced Distribution Server.

ex360_MP_View showcases usage of view request feature supported
by OMM Consumer and server.

Please refer to this article for view (and batch) features.

vuk.magdelinic

@chavalit.jintamalit yes I'm using ex 360. But can you tell me when the refresh/update happens then do we get all the values/fields that we specified in the view or just the updated fields out of those?

For ex - I subscribe to Ask, Bid and Display name

OmmArray array = EmaFactory.createOmmArray();
            array.fixedWidth(0);
            array.add(EmaFactory.createOmmArrayEntry().intValue(3));
            array.add(EmaFactory.createOmmArrayEntry().intValue(22));
            array.add(EmaFactory.createOmmArrayEntry().intValue(25);

But when I receive UpdateMsg, I only get Ask and Bid not Display Name. Is this expected behavior?

chavalit-jintamalit

The update message only contains the fields that are updated.

In this case, Display Name value is not updated(compare to the previous value in the refresh message) so it won't be included in the update message.

vuk.magdelinic

How can I get all the values and not just the updated ones? When using ValueObject MarketPrice I was getting all values regardless of them being updated or not

zoya faberov

Hello @vuk.magdelinic,

You can obtain all fields, only when you need them, by using Snapshot- specifically requesting snapshot when you need it, Snapshot request example can be found with RTSDK EMA Java - Example Market Price 102 - Snapshot

Or you can cache/store the required static values, and retrieve and combine them with every update, analogous to what Simple Market Price Object handles.

vuk.magdelinic

I used interestAfterRefresh(false) to get snapshot but now it seems I dont get any message after the initial refreshMsg.

My Thread.sleep is set to 1 sec.

Why??

zoya faberov

Hello @vuk.magdelinic,

This is correct. The snapshot behavior, as requested with interestAfterRefresh =false is you are requesting a single refresh (Snapshot), and no streaming updates.

There are two modes:

• Snapshot - single request, returns all fields
• Streaming - a stream is open, every update is received, until the stream is closed.

Hope this helps

vuk.magdelinic

@zoya.farberov

But in ex 102, it has a thread.sleep of 60 secs. So how is it getting the snapshots every 60 secs is what I'm trying to understand and replicate at my end.

Should I be calling this every few seconds to get the snapshot? Because I dont see that in ex 102

consumer.registerClient(reqMsg.serviceName("DIRECT_FEED").name("IBM.N").interestAfterRefresh(false), appClient);

vuk.magdelinic

@zoya.farberov

But in ex 102, it has a thread.sleep of 60 secs. So how is it getting the snapshots every 60 secs is what I'm trying to understand and replicate at my end.

Should I be calling this every few seconds to get the snapshot? Because I dont see that in ex 102

consumer.registerClient(reqMsg.serviceName("DIRECT_FEED").name("IBM.N").interestAfterRefresh(false), appClient);

zoya faberov

@vuk.magdelinic,

It is not getting snapshots every 60 sec. This example is timing it's run and exiting after 60 secs

zoya faberov

Hello @vuk.magdelinic,

If you are looking to tweak RTSDK EMA Java example 102 to snapshot every 60 sec, how about something like this, please replace the relevant part of the code in your example:

try
        {
            AppClient appClient = new AppClient();
            
            OmmConsumerConfig config = EmaFactory.createOmmConsumerConfig();
            
            consumer  = EmaFactory.createOmmConsumer(config.host("YOURHOST:14002").username("YOURUSER"));
            
            ReqMsg reqMsg = EmaFactory.createReqMsg();
            
            while(true) {            
                consumer.registerClient(reqMsg.serviceName("YOURSERVICE").name("IBM.N").interestAfterRefresh(false), appClient);            
                Thread.sleep(60000);            // API calls onRefreshMsg(), onUpdateMsg() and onStatusMsg()
            }
        }

this will run continuously and snapshot every 60 secs.

Let us know how this works for you?

vuk.magdelinic

Thanks @zoya.farberov

This is exactly what I did. And it ran fine. I was getting the snapshots of the records with values for all fields not just updated ones.

I kept it running, however, when I checked after 90mins, it failed with the below error: (code is exactly similar to what you have mentioned - I'm calling registerClient on consumer after every 1 sec instead of 60 sec)

Exception in thread "main" java.lang.NullPointerException
    at com.refinitiv.ema.access.SingleItem.rsslSubmit(ItemCallbackClient.java:2994)
    at com.refinitiv.ema.access.SingleItem.open(ItemCallbackClient.java:2856)
    at com.refinitiv.ema.access.BatchItem.open(ItemCallbackClient.java:3823)
    at com.refinitiv.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:2156)
    at com.refinitiv.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:457)
    at com.refinitiv.ema.access.OmmConsumerImpl.registerClient(OmmConsumerImpl.java:143)
    at EMAConsumer.Consumer.main(Consumer.java:548)

zoya faberov

@vuk.magdelinic,

Please confirm that you are still connecting to RTTO?

And the basis of your code is example MP 450 as per Quickstart Guide? (you have just added the loop to it) or you are doing something different?

If this is exactly the case, I can try to reproduce, by running the example with snapshots for 90 min+.

zoya faberov

Hello @vuk.magdelinic,

I have modified RTSDK EMA Java Example 450 (RRTO) to request snapshots every 60 sec and left it running for over two hours.

My modification of the example was:

            ReqMsg reqMsg = EmaFactory.createReqMsg();
            
            while(true) {            
                consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name(itemName).interestAfterRefresh(false), appClient);            
                Thread.sleep(60000);            // API calls onRefreshMsg(), onUpdateMsg() and onStatusMsg()
            }    
  

I did not reproduce the NullPointer exception.

I attach the complete zipped code as well:

Consumer_450_SnapshotLoop.zip

Is this identical to what you have modified and that have run and your run has resulted in the exception?

Does the issue reproduce consistently in your environment by running this code?

If it is not the same, please run the attached, and let us know how this works for you?

vuk.magdelinic

Yes my code is exactly the same as you provided except you are refreshing at 60sec and I'm refreshing at 1sec. Even when I run your code it fails after sometime. The time when it fails is not consistent.

Can you try running running at your end for 1 sec and see if you reproduce the issue?

zoya faberov

Hello @vuk.magdelinic,

Snapshotting every second is not a sound design approach. The snapshot is comprised of registration, receipt of the image or status, complete cleanup of the registration, all handled asynchronously on your behalf, but required.

You have said before "getting snapshots every 60 seconds" is what you were looking to do, and I have tried to help you with design approach and providing the example.

RRTO is already a conflated stream, quotes capped at 3 updates per sec, trade-safe. You will not be facing the situation, for example, with 100+ updates per second.

If you require to have the complete image, all fields, refreshed, every second, I would suggest as your custom consumer design approach:

• Use streaming, rather then snapshot
• Cache/store any fids that you require that are not being updated, and enrich them into the updates that are received
• Discard any updates that you do not require, such as after 1/sec

The article, the companion code of which you have originally used A simple MarketPrice object for EMA discusses a design approach to analogous task (without understanding the complete requirement can not tell if this is a 100% match with your requirements). It does state however:

"The MarketPrice class and the ValueAddObjectsForEMA example library have been written by Refinitiv for the only purpose of illustrating a series of articles published on the Refinitiv Developer. The MarketPrice class and the ValueAddObjectsForEMA example library have not been tested for usage in production environments. Refinitiv cannot be held responsible for any issues that may happen if these objects or the related source code is used in production or any other client environment. "

However, as the article and its code pre-date RTSDK and even RRTO, in my understanding, the article's material can be helpful to learn of the concepts involved and the approach that was used, but you would have to design, for the approach illustrated, rather then use the code as is.

I hope this helps

vuk.magdelinic

@zoya.farberov what is the recommended and the minimum time to get snapshots? 60 secs is too long, can I go with snapshots every 10sec lets say?

zoya faberov

Hello @vuk.magdelinic,

It's a totally different question from what we have started with, and it is better to ask a new question as a new question, while including the link to the related previous discussions, so that the answers are readily visible and easily findable by the next developers with same ot similar question.

However, I hope this discussion is useful to you?

There is no "blanket" answer to this question. The best approach is to do a thorough testing in your local environment and once we identify what works, we always prefer to err on the side of safety. I.e. consider doubling the interval that you find to work safely for your instrument list size in your environment and under your network and internet conditions.

zoya faberov

Hello @vuk.magdelinic,

Attached is a rough port of A simple MarketPrice object for EMA article code, included are:

• Example.EMA.Java.ValueAddObjectsForEMA-master and Article.EMA.Java.MarketPriceSubscriber-master are included
• Built with RTSDK 2.0
• Migrated namespaces to Refinitiv convention
• Added support for RSSL_ENCRYPTED (RRTO)

From my testing, the example runs as expected and does not drop connection with arguments:

-wu -s ELEKTRON_DD -u MACHINEID -p PASSWORD -kf KEYSTOREFILEWITHPATH -kp KEYSTOREPASSWORD -c CLIENTID JPY=

Article.EMA.Java.OlivierDavant.RTSDK20QuickPort.zip

Let us know if you find this useful

vuk.magdelinic

What is the refresh rate you have used in this? We want 1 sec refresh with full data.

zoya faberov

Hello @vuk.magdelinic,

As you have stated ValueAddedObject was not working right for you and you were most interested in getting ValueAddedObject working right, with RRTO, I went back and invested time into rough-porting it to RTSDK 2.0.

I have run MarketpriceSubscriber object, with the parameters noted above.

1 sec refresh:

This is streaming request, every update is received. RRTO connectivity is 3 quotes per sec, conflated stream, trade-safe. If you wish you can process the first update in a sec, and ignore the rest, of course.

Full data:

"Objects, like MarketPrices or Order Books that cache (retain) in memory the latest values sent by the platform so that your application can randomly get this data anytime and from any thread".

You can retrieve the full set of fields at any time, including on update. Where you have in code

print(marketPrice, UPDATE, update);

you can add

print(marketPrice, "ALLFIELDS", marketPrice.getFields());

or you can also getField(name) at any time, and only get the fields you need, not all couple of hundred of fields in the image.

Another option is creating a separate thread and sampling MarketPrice every second from that thread.

The attached should be helpful in giving you a solid start, allowing to next implement your requirements using it as a working base.

vuk.magdelinic

@zoya.farberov does Refinitiv have a streaming API that pushes updates to us instead of we polling it every sec or sub-sec for an update?

zoya faberov

Hello @vuk.magdelinic,

This is an excellent question. I would say separate from the original question asked, and it's always better to ask a separate question as a new and separate question on the forum, rather then as an answer. This way it gets the proper attention from the community as well as moderators.

RTSDK Suite of APIs is expressly designed for streaming. Both EMA at ease of use tier and ETA at transport tier support streaming:

• Please see EMA Quickstart - Connecting to RTTO guide that we have referred you to before, for step-by-step instructions to, it should help you to establish a streaming connection to RTTO quickly and efficiently.
• With that working, next step we suggest is progressing your knowledge of EMA via EMA Tutorials.
• Once you quickly get up to speed on the API, next, you can implement to your custom requirements using our examples code as the basis, the starting point.

You also have an option to use Value Added Objects concepts, and the rough port that was provided above should enable you to do so with the latest version of RTSDK, saving you time and effort if you, as you originally have indicated, prefer to proceed via this approach.

However, it seems that you require more help with your application development. I strongly advise you to contact your Refinitiv account manager to discuss your requirements and concerns, so you can be best helped. Not all questions and issues yield themselves to being fully addressed, or even effectively discussed, on a public forum.

Within Refinitiv developer forums, we are most effective in helping you with general questions centered around Refinitiv APIs. Your organization's specific needs are best addressed individually, by discussing them in detail with your Refinitiv account manager.