C# datascope select SDK : When we run more than one download concurrently, e.g., download 2 days of
We have a client : tried to download tick data from datascope for below RICs:
CHK.O,Ric
GDX,Ric
SPY,Ric
GE,Ric
EEM,Ric
BAC,Ric
AMD.O,Ric
XLF,Ric
F,Ric
VXX,Ric
NIO,Ric
T,Ric
TVIXF.PK,Ric
We are using the C# datascope select SDK call for the download. The problem we encounter is: when we run more than one download concurrently, e.g., download 2 days of data at the same time, randomly data of some of the days are missing in the downloaded files. I attach a log file Concurrent_download.log Concurrent_download.txt to explain this in details.
The log shows that we wanted to download 4 days of above data, we run the C# downloader to open 4 connections in parallel each for a day. All of them were downloaded, the notes show correct tick count and everything looks fine. But in fact out of these 4 days, only one day's downloaded file contains full data. The rest 3 contains some data but miss most of it. Our log file shows 3 exceptions were thrown during writing the file to disk: One or more errors occurred. (Error while copying content to a stream.) This sounds like an exception thrown from HttpClient of .NET.
When we download these 4 days of data individually with single connection, all data can be downloaded successfully. It iindicates to us that we have problem with concurrent connection with datascope select. However this only happen with relatively bigger tick data, we use same C# program to download small daily data with 4 concurrent connections, we don't observe similar problem.
Best Answer
-
Hello @beera.rajesh01 ,
On the theoretical level:
For the majority of the request templates, the extraction requests are processed 2 at at time. Up to 50 can be submitted, but they are being queued for processing, see Tick History REST API User Guide->Concurrent Requests. Some templates it's only 1 request processed at a time.
So I personally prefer not to submit requests that are guaranteed to be queued instead of processing, especially large ones that will be queued longer.
However, if submitted, all requests are supposed to be processed completely.
---
To try to determine why this is not happening in this case, we will need to try to reproduce the issue by running the exact same test scenario on our side.
I see two ways:
The client can provide the complete requests, and you can submit 4 parallel requests via HTTP REST tool, for example Postman- verify if the results received will be complete. This is the cleaner test case.
Or the client can share their C# code and then you can try to test to verify using their code.
I hope that this information helps
0
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛