Performance issues when fetching large amounts of data
I'm using the Dex2 COM API for VBA to get fundamental and historical performance data (20 TR. fields) for about 10k instruments. It takes me from 20min up to more than 1h, depending of the time of the day, which I think it's a very long time!
Are there best-practices and concrete peak-times which we should avoid? For example, around 8am seems to be much faster than 5pm (European Time).
Second question about efficiency: i'm sending requests in batches of 500 instruments for 20 TR. fields - is that optimal or should I rather ask for batches of 2000 instruments but only 5 fields? Any experience with the best-performing combination?
Third question about efficiency: how slow is the AVAIL function? Does dividing the assets in categories (Bonds, Equities, Funds) make sense in term of efficiency, so that I can tailor the query with the assetcategory-specific fields instead of using AVAIL?
Many thanks!
Best Answer
-
@nicola.pestalozzi
Eikon is an end user product. The amount of data you retrieve is borderline where you may want to consider using one of Thomson Reuters enterprise products specifically designed and engineered for bulk data downloads.
As for your questions, the answer to them can only be determined empirically. At Thomson Reuters we don't do any benchmarking of Eikon for bulk data download requests. In other words data retrieval times for bulk data download requests is not a key product performance indicator that we routinely monitor for Eikon.0
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛