Is there a tool that I can record data from TREP ads server and replay the data to TREP infra for te

nikita.ou

I am looking for a tool that I can use to record market data and replay the data in our TREP infra for our testing. We usually use rmdstestclient to record data. Would you please advise if we can use the same command for data replay? And what is the parameter we should use? I am looking for something like a tool that could record data , then replay at faster/slower rates

zoya faberov

Hello @nikita.ou,

First, confirm that your recording contains updateMsgs, and not just refreshMsg

If it does, next try

sink_driven_src -S ACTIV -C 100000 -E rssl -N 14003 -U 1 -K -Q

/reuters/ads/demo/active.out

I think you may be confusing replay with "-l", and instead of replaying the updates from your capture file, the tool is trying to generate an update of length 1, which may not be working

I prefer rmdstestclient with "-v" to rsslSinkApp, but both should work.

zoya faberov

I would say that there are two prevalent options:

1. Infra tools package that includes rmdstestclient also includes sink_driven_src tool. Rmdstestclient can be used to record data into a file, while sink_driven_src can be used to replay it. Document "Demo Tools" that is included with the package goes over the functionality, in brief. Each tool displays help on it's command line switches. One can replay/simulate data at specified rates, i.e. msg/sec.

2. To cover more advanced replay requirements, for example, GUI-driven, drill-in and selection, replay at times the original speed, modifications on the fly, other, one can use ReplayService tool. For more information please contact TR sales.

nikita.ou

Thanks Zoya,

I record the data by using "nohup ./rmdstestclient -S ACTIV -h localhost -p 14002 -f ACTIV.ric -u radmin -ct rssl -m -X -d 3 -micro >Activ.out &" and try to play back the data by using " ./sink_driven_src -S ACTIV -K -N localhost -Q /reuters/ads/demo/Activ.out -c -b -I 10"
But i am getting the following:

Error: Unexpected Exception during XML parsing, possibly file is not found or corrupt

Error rsslBind(): <Impl/ripcsrvr.c:1381> Error: 1004 ripcGetServByName() failed. Port number is incorrect. (0)

Would you please advise how I can resolve this?

zoya faberov

Hello @nikita.ou ,

Inspect the capture file. Is it created? Does it look to be a well-structured and complete xml file?

I would try

-of active.out -l active.out.log

instead of ">"

then you can also review the log looking for errors

nikita.ou

cusersnouonedrive-macquarie-groupmarket-datathomso.txtThe data capture seems to be correct to me. I have attached it for your reference.

Below is the msg I get when I run sink_driven_src this time

Initialized RSSL library.
GLOBAL update order enabled.
Item template & Update template will not be used.
Loaded refresh: 3 updates: 3486

Enter Command (help): .
Loaded pkts: 0 items: 2 images: 2 image_length: 0 unsol images: 0 updates: 3486 update_length: 0 memory (MB): 0.827613
Error rsslBind(): <Impl/ripcsrvr.c:1381> Error: 1004 ripcGetServByName() failed. Port number is incorrect. (0)

zoya faberov

Hi @nikita.ou,

Same here.

Try this replay

sink_driven_src -S ACTIV -C 100000 -E rssl -N 14003 -U 1 -K -Q

/reuters/ads/demo/Activ.out

This is my default replaying command line, with your service and recording name substituted.

Let me know how it works for you.

nikita.ou

Thanks Zoya,

I got below msg. I think it works now. So this command publish the data under feed ACTIV, port 14003. Downsteam user should connect to port 14003 to subscript the data, right?

If I want to tune the update rate, how should I do?
Initialized RSSL library.
Loaded refresh: 3 updates: 55
Loaded pkts: 0 items: 2 images: 2 image_length: 0 unsol images: 0 updates: 55 update_length: 0 memory (MB): 0.0145607
setup_updates: cfg_cfg_ticks_per_second=100 cfg_ticks_per_second=100 usec_per_tick=10000 updates_per_tick=0 extra_updates=1 max_updates_to_pack=10Server fd=3: New client fd=4 newSession

zoya faberov

Hi @nikita.ou,

Yes, in this example, we replay on 14003 and consumer should connect on 14003.

See Installation Guide for update rate tuning example:

"... measure latency at various update rates starting from 10k and going to 200k, in increments of 10k and run the test for 5 minutes at each rate ... you can use the following set of options:

-U 10000 –M 200000 –D 10000 –T 300 –W 120

"

nikita.ou

@zoya.farberov thanks. I started the data replay by using

" ./sink_driven_src -S ACTIV -K -N 14003 -Q /reuters/ads/demo/active.out -c -b -l 1" then

use "./rsslSinkApp -h mdshkdev35 -p 14003 -U radmin" to subscript the data.

However i only get an image update at the subscription, there is no update of the RIC coming after that. Would you please advise?

nikita.ou

@zoya.farberov thanks. I started the data replay by using

" ./sink_driven_src -S ACTIV -K -N 14003 -Q /reuters/ads/demo/active.out -c -b -l 1" then

use "./rsslSinkApp -h mdshkdev35 -p 14003 -U radmin" to subscript the data.

However i only get an image update at the subscription, there is no update of the RIC coming after that. Would you please advise?

nikita.ou

Thanks Zaya, the recording doesn't contains updateMsg. I try your command. it works now. Thank you so much for your help!

nikita.ou

Hi @zoya.farberov . it works when I use the rsslSinkApp, but when our application team try to connect, i get the below error msg.

In readFromSession() calling rsslInitChannel() Session Inactive fd=4 <<Impl/ripcsrvr.c:3335> Error: 1007 Unknown connection type.

In their connection config, they use RSSL too. Would you please advise what might be the root cause? Thanks

zoya faberov

Hi @nikita.ou ,

This is why I like to be explicite and use both "–ct rssl" on record and "-E rssl" on replay.

This ensures you are recording and replaying RSSL.

If the downstream client uses RSSL (?) it should be able to consume.

Confirm if downstream client expects RSSL or SSL. Does it regularly connect on 14002? or on 8101 (SSL port).

nikita.ou

I did use -cr rssl on record and -E rssl on replay. The downstream app uses RSSL too. Below is their config. I replay the data on port 14003 as we have the Prod feed available on port 14002 in the same server.

\Connections\reutnjads\connectionType = "RSSL"

\Connections\reutnjads\rsslPort = "14003"

Downstream app tries to connect to normal live market data. It works. The issue only happened when they connects to the replay data. I can see they connect to that port but the subscriptions doesn't work.

nikita.ou

Below is msg that the downstream app gets in their log
Service: ACTIV
Item: SPY.
StreamState: Open
DataState: Suspect
StatusCode: None

Source unavailable... will recover when source is up

zoya faberov

Hi @nikita.ou

From the error above message- they are connecting.

They are failing to consume item SPY from source ACTIV

I would verify with a standard client. Try testing service ACTIV item SPY with rmdstestclient.

If rmdstestclient is able to consume an item from service via user, then a custom client should be able to do the same.