DSS REST API - Error 400 malformed request payload
Hello,
I'm trying to extract data (End of day, Time Series,...) using the DSS REST API with Python 3.5.2.
First, I connect to the DSS server as described in REST API Tutorial 1, without any problem
header = {'Prefer': 'respond-async', 'Content-Type': 'application/json; odata.metadata=minimal'}
urlGetToken = 'https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken'
loginData = json.dumps({'Credentials':{'Password':Password,'Username':Login}})
resp = requests.post(urlGetToken, loginData, headers=header)
I can also get the available list of fields without any error
header2 = {'Prefer': 'respond-async, wait=5', 'Content-Type': 'application/json; odata.metadata=minimal', 'Authorization': myToken}
urlGetListFields = "https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/GetValidContentFieldTypes(ReportTemplateType=ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ReportTemplateTypes'TimeSeriesPricing')"
resp2 = requests.get(urlGetListFields,headers=header2)
Finally, when I try to extract some data:
# From users input
listIdentifiers = [{"Identifier":"BE0003592038","IdentifierType":"Isin"},{"Identifier":"BE0003593044","IdentifierType":"Isin"}]
fromdate = "2016-06-01"
todate = "2016-06-30"
header3 = {'Content-Type': 'application/json; odata.metadata=minimal; charset=utf-8', 'Authorization': myToken}
urlGetData = 'https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Extract'
odataType = "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TimeSeriesExtractionRequest"
condition = {"LastPriceOnly": False,"StartDate": fromdate + "T00:00:00.000Z","EndDate": todate + "T00:00:00.000Z"}
# in practice listFields is bigger
listFields = ["Accrued Interest","Close Price","ISIN","RIC"]
requestData = {"ExtractionRequest": {"@odata.type": odataType,"ContentFieldNames": listFields,"IdentifierList": {"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList","InstrumentIdentifiers": listIdentifiers},"Condition":condition}}
resp3 = requests.post(urlGetData, data=json.dumps(requestData), headers=header3)
Sometimes, it works perfectly well as I get resp3.status_code = 200 or resp3.status_code = 202, but most of the time I get resp3.status_code = 400 with one of the following messages for the exact same request:
400{'error': {'message': 'Malformed request payload: For the property name "StartDate" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property.'}}
400{'error': {'message': 'Malformed request payload: Syntax error at Line 1, Char 64: Invalid array element type for property \'ContentFieldNames\'. Expected element type of \'ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.ExtractionRequestBase\', but found \'System.String\'. {"ExtractionRequest": {"ContentFieldNames": ["Accrued Interes'}}
400{'error': {'message': 'Malformed request payload: For the property name "EndDate" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property.'}}
400{'error': {'message': 'Malformed request payload: For the property name "IdentifierList" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property.'}}
400{'error': {'message': 'Malformed request payload: For the property name "Identifier" in the JSON request the value could not be parsed successfully. Please check the casing or spelling of the property.'}}
I tried to solve the problem by including charset=utf-8 in the header, using the json parameter from requests, ... but it seems that the only effective hack is to delete then include again a field in listFields.
Do you have any idea that could help me?
Best Answer
-
Unfortunately, the order of the @odata.type line is crucial (we do intend to fix that in a future release):
You sent in:
{
"ExtractionRequest": {
"ContentFieldNames": ["Accrued Interest",
"ClosePrice",
"ISIN",
"RIC"],
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TimeSeriesExtractionRequest",
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"IdentifierType": "Isin",
"Identifier": "BE0003592038"
},
{
"IdentifierType": "Isin",
"Identifier": "BE0003593044"
}]
},
"Condition": {
"LastPriceOnly": false,
"EndDate": "2016-06-30T00:00:00.000Z",
"StartDate": "2016-06-01T00:00:00.000Z"
}
}
}but when edited to this, it worked (note: I also change "ClosePrice" to "Close Price"):
{
"ExtractionRequest": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TimeSeriesExtractionRequest",
"ContentFieldNames": ["Accrued Interest",
"Close Price",
"ISIN",
"RIC"],
"IdentifierList": {
"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [{
"IdentifierType": "Isin",
"Identifier": "BE0003592038"
},
{
"IdentifierType": "Isin",
"Identifier": "BE0003593044"
}]
},
"Condition": {
"LastPriceOnly": false,
"EndDate": "2016-06-30T00:00:00.000Z",
"StartDate": "2016-06-01T00:00:00.000Z"
}
}
}0
Answers
-
Is there any chance you can capture the full request and response actually sent on the wire?
I should be able to tell you what was wrong with any 1 request that causes a 400 reply as long as I can see the raw data you sent in.
We currently do not record every bad request, but we are considering doing just that so we can be more supportive on these types of requests.
0 -
Hello @Rick Weyrauch, thank you for your quick answer. Here is what I get (I replaced the token by MyToken):
>>> requests.post(urlGetData, data=json.dumps(requestData), headers=header3)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): hosted.datascopeapi.reuters.com
send: b'POST /RestApi/v1/Extractions/Extract HTTP/1.1\r\nHost: hosted.datascopeapi.reuters.com\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: python-requests/2.11.1\r\nConnection: keep-alive\r\nAccept: */*\r\nContent-Type: application/json; odata.metadata=minimal; charset=utf-8\r\nAuthorization: myToken\r\nContent-Length: 577\r\n\r\n'send: b'{"ExtractionRequest": {"ContentFieldNames": ["Accrued Interest", "ClosePrice", "ISIN", "RIC"], "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.TimeSeriesExtractionRequest", "IdentifierList": {"@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [{"IdentifierType": "Isin", "Identifier": "BE0003592038"}, {"IdentifierType": "Isin", "Identifier": "BE0003593044"}]}, "Condition": {"LastPriceOnly": false, "EndDate": "2016-06-30T00:00:00.000Z", "StartDate": "2016-06-01T00:00:00.000Z"}}}'
reply: 'HTTP/1.1 400 Bad Request\r\n'
DEBUG:requests.packages.urllib3.connectionpool:"POST /RestApi/v1/Extractions/Extract HTTP/1.1" 400 343
header: Set-Cookie header: Cache-Control header: Pragma header: Content-Type header: Expires header: Server header: X-Request-Execution-Correlation-Id header: X-App-Id header: X-App-Version header: Date header: Content-Length <Response [400]>0 -
@nvico and @Rick Weyrauch thank you for this post. I was experiencing the same problem using Python and use of the OrderedDict type solved it, i.e. the seemingly random nature of the error when using a Python dictionary to create the payload for an On Demand POST request to DSS: sometimes the very same request worked and sometimes not.
I would like to add something that I found: the 'randomness' occurred when I was using the dictionary Python 3.5 but when I made requests from another machine using Python 3.6.1, with the same dictionary structure and script, the request worked every time. I subsequently upgraded my Python 3.5 to 3.6.1 and now [using the dictionary, not OrderedDict] it works reliably too.
However, for what I am assuming is good practice, and compatibility with earlier versions of Python, I am using OrderedDict.
1
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛