Python: Eikon is generating unrequested dates for some ID's, using get_data.
I have the following code that requests local returns for a list of companies.
import pandas as pd
import eikon as ek #import Eikon module
import datetime
import time
start_time=time.time()
ek.set_app_id('someId') #setting AppID
#Getting a list of dates
start=datetime.date(2017,11,27)
end=datetime.date.today() - datetime.timedelta(days=1)
row_dates=[x.strftime('%m/%d/%Y') for x in pd.bdate_range(start,end).tolist()]
#getting identifiers to be used on Eikon
csv_data=pd.read_csv('identifiers_test.csv', header=None)
identifiers=csv_data[0].tolist()
This is a list with 2500 identifiers or so (all SEDOLS).
To request the returns, I do this:
df=ek.get_data(identifiers,["TR.TotalReturn.Date","TR.TotalReturn"], {'SDate': row_dates[0], 'EDate': row_dates[len(row_dates)-1], 'Frq':'D'})[0]
To pivot the table, and re-arrange it in a way that I can have dates as column headers and identifiers as indexes without being repeated, I do this:
df=pd.crosstab(df.Instrument, df.Date,values=df['Total Return'], aggfunc='mean')
But the outcome is really strange, it places columns with dates that I didn't even requested, filled with NaN's, and I would like to know how to filter that info. I already tried a couple of approaches with dropna() and other Pandas functions but I can't seem to get them to work , I'll attach 2 photos so you can see what I'm talking about).
And it properly shows dates I've requested, like this (shown 28-Nov, 29-Nov and 30-Nov. Not displaying in the picture but appeared properly in the dataframe, are 27-Nov and 01-Dec).
How can I get rid of those dates? Obviously these dates were produced by Eikon's API, somehow. I printed the original dataframe into a CSV file, I should have only 5 dates, I obviously found more. I can't upload the screencapture for this since this forum doesn't allow more than 2 image uploads.
Why Eikon does this? Is there a way to ellude it? A way to fix it? I'd like to know.
Thanks in advance!
Best Answer
-
Ok, I was able to get the list of the instruments, that are generating incorrect dates, you can use the following code:
instruments = {}
for ix, row in df.iterrows():
instruments.setdefault(row[1], set()).add(row[0])
instrumentA quick manual check shows that the incorrect date is when the asset traded last, for example:
B2Q1MC6 (PGD.P) - 5th of September
B4TDMX4 (SBV.P) - 19th of SeptemberEtc.
So, when you are requesting the latest Total return value, what it can give you is the last possible estimate.
1
Answers
-
Can you share the list of 2500 identifiers? I have used you code with some instruments and it works fine. I am using Eikon 0.1.9.
0 -
I can not reproduce the issue by requesting the same data for S&P 500 constituents, so there might be an issue with one of the instruments in your list
0 -
files.zipjirapongse.phuriphanvichai, here I uploaded two files. The first one has all the identifiers I use for this code. And the other file is where I saw which SEDOL's are giving me issues.
Is there a way to code an exception for these type of errors?
0 -
Yes, I'm guessing that as well. But why? I mean, how does Eikon handle errors with identifiers? Is there a way to detect/handle that kind of error and delete it from the outcome?
0 -
Do you know a way to code a 'filter' for these unrequested dates? Since they are very specific, I didn't find a 'conventional way' that works.
e.g: I cannot use dropna() pandas function because, even though these dates are mostly filled with NA's, a single return value is not a NaN, so I can't use that solution.0 -
you can drop the rows that have the dates that are not equal to the requested range with the standard pandas function
0 -
I tried another approach that worked:
added this after the crosstab line:df=df[row_dates]
Before doing that, I had to convert the Date column to Datetime using:
df['Date']=pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.strftime('%m/%d/%Y')1
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛