How to Fix, TRTH On Demand Raw Data Extraction Errors

Naveen Tad

Hello ,

I am working on setting up a python downloader for “On Demand Raw Data Extraction” using the documentation provided here (https://developers.refinitiv.com/en/api-catalog/refinitiv-tick-history/refinitiv-tick-history-rth-rest-api/tutorials#rest-api-tutorial-8-on-demand-raw-data-extraction)

I’m running into an issue where I get a 400(Bad Request) response. See below for details

On Demand Extraction Request Url : https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw

Python Function

baseUrl = "https://selectapi.datascope.refinitiv.com/RestApi/v1/"

requestHeaders = { "Prefer":"respond-async", "Content-Type":"application/json" }

# Get Location URL

defgetLocationUrl( token):

ExtractionUrl = baseUrl + "Extractions/ExtractRaw"

requestHeaders["Authorization"] = "token " + token

params = { "ExtractionRequest": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", "IdentifierList": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [ { "Identifier" : "AB.PS" , "IdentifierType" : "Ric" } ] }, "Condition": { "ReportDateRangeType": "Delta", "DaysAgo": 1 } } }

print(ExtractionUrl)

r = requests.post( ExtractionUrl, headers=requestHeaders , params=params)

if (r.status_code==200):

print ("YAY")

else:

print(r.status_code)

print(r.reason )

print(r.text)

Response Code

r.status.code = 400

r.reason = ‘Bad Request’

r.text = {"error":{"code":"6fe04886-854a-45d5-bc57-4d6ab225a700","message":"Query option 'ExtractionRequest' was specified more than once, but it must be specified at most once."}}

Per the Response message , ‘ExtractionRequest’ was specified more than once, but as you can see in the below, It was only specified once

params = { "ExtractionRequest": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", "IdentifierList": { "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList", "InstrumentIdentifiers": [ { "Identifier" : "AB.PS" , "IdentifierType" : "Ric" } ] }, "Condition": { "ReportDateRangeType": "Delta", "DaysAgo": 1 } } }

Can someone please look into this and help us out with resolving this issue.

Thanks,

Naveen.

zoya faberov

Hello @Naveen Tad,

In my understanding, as the tutorial you looking at is par of Postman starter collection, discusses step-by-step the approach for Postman requesting, and Raw request is available and downloadable with our Postman starter collection,

If you would like to run the same/similar Raw extraction request, but in your python code,

The quickest way to get it working in Python would be:

1. Install free tool Postman if you have not had a chance, via www.getpostman.com
2. Download fully working RTH REST Tutorials Postman collection
3. Import collection, run
   1. Authentication request
   2. Raw data extraction request
   3. You have a fully working "naked" request at hand
4. Download working on-demand example code for Python in RTH Python Code Samples
5. Swap the request in on-demand example with your required request, from Postman collection.
6. Any other on-demand DSS HTTP REST request that you will have ready and working in Postman, you will be able to quickly have working from python code the same way, by creating a copy of the working example and just swapping the payload of the request into the example in place of the request.

Does this help, will you be able to proceed?

zoya faberov

Hello @Naveen Tad,

From looking at the code, it seems that you are not passing requestBody into Post request, but only URL, headers and parameters.

I am not fully sure if this is the only issue with the code, but would suggest that by following the steps in the answer and starting with both the working request to replace and the working python code skeleton, you should arrive at a working code per your requirement the quickest way.

Naveen Tad

Thanks. Will test with sample examples and get back on this thread if any issues.

Naveen Tad

I downloaded the sample program RTH_OnDemand_IntradayBars.py and just ran it with my ID and I get ‘Quote Exceeded Error’. Please take a look at the below logs /

$ python ./RTH_OnDemand_IntradayBars.py

Authentication token (valid 24 hours):

**********************************

HTTP status of the response: 202

Extraction is not complete, we shall poll the location URL:

https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x0793c595679ca9d4')

As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)

HTTP status of the response: 202

As we received a 202, we wait 30 seconds, then poll again (until we receive a 200)

HTTP status of the response: 200

jobId: 0x0793c595679ca9d4

Extraction notes:

Extraction Services Version 14.5.42294 (737b0965c07f), Built Apr 8 2021 13:43:46

User ID: 9027207

Extraction ID: 2000000265791722

Schedule: 0x0793c595679ca9d4 (ID = 0x0000000000000000)

Input List (2 items): (ID = 0x0793c595679ca9d4) Created: 06/01/2021 16:13:47 Last Modified: 06/01/2021 16:13:47

Report Template (16 fields): _OnD_0x0793c595679ca9d4 (ID = 0x0793c59567bca9d4) Created: 06/01/2021 16:12:09 Last Modified: 06/01/2021 16:12:09

Schedule dispatched via message queue (0x0793c595679ca9d4), Data source identifier (1795B0303CCE47368DE78572C98769BE)

Schedule Time: 06/01/2021 16:12:10

Processing started at 06/01/2021 16:12:10

Processing completed successfully at 06/01/2021 16:13:48

Extraction finished at 06/01/2021 16:13:48 UTC, with servers: tm02n02, TRTH (89.76 secs)

Instrument <RIC,CARR.PA> expanded to 1 RIC: CARR.PA.

Instrument <RIC,IBM.N> expanded to 1 RIC: IBM.N.

Total instruments after instrument expansion = 2

Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 0; Instruments Approved for Extraction: 0; Tick History Cash Quota Count After Extraction: 0, 100% of Limit; Tick History Cash Quota Limit: 0

Quota Message: ERROR: The RIC 'CARR.PA' in the request would exceed your quota limits. Adjust your input list to continue.

Quota Message: ERROR: The RIC 'IBM.N' in the request would exceed your quota limits. Adjust your input list to continue.

Quota Message: WARNING: Tick History Cash Quota has been reached or exceeded

Content response headers (AWS server): type: text/plain

Saving compressed data to file: .Python_Test.step4.csv.gz ... please be patient

Finished saving data to file:.Python_Test.step4.csv.gz

Decompressed data:

Content response headers (AWS server): type: text/plain

Saving compressed data to file:.Python_Test.step4.csv.gz ... please be patient

Finished saving compressed data to file:.Python_Test.step5.csv.gz

zoya faberov

Hello @Naveen Tad,

As you login into DSS GUI, please click on the menu Your Name->Usage->Quota.

I can see two possibilities: if your user id is not permissioned for Intraday bars request, you should not see any quota, or if you are permissioned, you have exceeded your quota and should see it as well.

---

If you are permissioned for TickHistoryRaw request, once you have have your TickHistoryRaw request tuned and working in Postman, you could replace the request in example code with yours= working request, and run that instead.

Let us know how this works for you?