Encounter performance issues on querying historical price data
We encounter performance issues on querying historical price data when using new Data Platform Python Library.
https://pypi.org/project/refinitiv-data/
The waiting time is long (almost spending more than 30 minute each request) and the situation is not resolved even though we tried methods such as dividing our requests to several smaller batches and adding waiting time between requests.
Below are code samples that how we query the data:
@contextmanager
def rd_session():
"""Context manager for handling a Refinitiv Data Library for Python session."""
try:
rd.open_session()
yield
finally:
rd.close_session()
class Processor(BaseProcessor):
def call_eikon_api(self):
logger.info("Calling Refinitiv Data API...")
with rd_session():
raw_eikon_res = rd.get_data(
universe=self.ticker_list,
fields=self.field_list,
parameters=self.request_settings,
use_field_names_in_headers=True
)
Below is our sample universe, fields and parameters:
{'ticker_list': ['KWEB.K', 'INDY.O', 'ESTC.K', 'VIPS.K', 'EDU', 'OLED.O', 'MDB.O', 'NVDA.O', 'STX.O', 'MBLY.O', 'MSFT.O', 'AMAT.O', 'AMKR.O', 'HCM.O', 'META.O', 'LRCX.O', 'JD.O', 'COIN.O', 'LKNCY.PK', 'EWY', 'UMC', 'INTC.O', 'MU.O', 'PDD.O', 'SNOW.K'], 'field_list': ['TR.CLOSEPRICE', 'TR.CLOSEPRICE.calcdate'], 'request_settings': {'Frq': 'C', 'SDate': '2024-01-17', 'EDate': '2024-01-17', 'FILL': 'PREVIOUS', 'curn': 'USD'}}
May I know if Eikon helpdesk can provide any solution to improve the performance when using Python API? Such situations are happened several times recently and affect our daily operational workflow.
Best Answer
-
Hi @rchan ,
Could you please upgrade your refinitiv-data Python library to version 1.4.0, I cannot reproduce this issue on this version. (the script was run around 18:20, GMT+8)
0
Answers
-
@rchan Thanks for your question and sorry to hear about your issue. I have tried this in the Codebook App in Workspace/Eikon and it returns for me instantly. Please see the following code:
rd.get_data(
universe=['KWEB.K', 'INDY.O', 'ESTC.K', 'VIPS.K', 'EDU', 'OLED.O', 'MDB.O', 'NVDA.O', 'STX.O', 'MBLY.O', 'MSFT.O', 'AMAT.O', 'AMKR.O', 'HCM.O', 'META.O', 'LRCX.O', 'JD.O', 'COIN.O', 'LKNCY.PK', 'EWY', 'UMC', 'INTC.O', 'MU.O', 'PDD.O', 'SNOW.K'],
fields= ['TR.CLOSEPRICE', 'TR.CLOSEPRICE.calcdate'],
parameters = {'Frq': 'C', 'SDate': '2024-01-17', 'EDate': '2024-01-17', 'FILL': 'PREVIOUS', 'curn': 'USD'}
)These 25 RICs returned for me instantly. I'm not sure what version of the library you are using but this was running with Refinitiv Data Library 1.4.0. What version of the library are you using?
thanks
0 -
Thanks for reply.
We are using Refinitiv-data 1.3.0 in Python 10. We suffer performance issue on price querying during 16:00 - 19:00 GMT+8, but the task may not be rescheduled into other timeslot as if do so so other daily works will be affected.0
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛