DSS 429 Response

gareth.teage

Query: "Hi

I'm trying to find an example of a DSS 429 (too many requests) response body & headers in your docs so that we can handle it correctly but I can't seem to find one. Can you provide one please?

Thanks"

Edit/Update with more information:

"

I'm not asking that you provide and example request I can make to see this error, I know that isn't possible.

In your API documentation there should be examples of what headers and body will be received for each kind of error, for example for a 400 error you have an example of the headers and body here https://selectapi.datascope.refinitiv.com/RestApi.Help/Home/KeyMechanisms?ctx=Extractions&opn=Extract&sce=Validation%20Error%20(400%20status%20code)&stp=1&tab=3&uid=Validation

I need a similar example for a 429 error. If you're going to rate limit clients, it's only fair that you provide examples that demonstrate how to handle that situation"

Jirapongse

@gareth.teage

Thank you for reaching out to us.

To get 429 (too many requests), you need to send a lot of requests concurrently that exceeds the

Queued Extraction Request Limit defined in the DataScope Select Best Practices & Fair Usage Policy.

For example the code could like this:

import requests
import concurrent.futures
import time
import json


token = '<token>'


def get_status(url):


    payload = json.dumps({
  "ExtractionRequest": {
    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.HistoricalReferenceExtractionRequest",
    "ContentFieldNames": [
      "RIC",
      "ETH Expiry Date"
    ],
    "IdentifierList": {
      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
      "InstrumentIdentifiers": [
        {
          "Identifier": "ESc1",
          "IdentifierType": "Ric"
        }
      ],
      "ValidationOptions": {
        "AllowHistoricalInstruments": True
      },
      "UseUserPreferencesForValidationOptions": False
    },
    "Condition": {
      "ReportDateRangeType": "Range",
      "QueryStartDate": "2000-01-01",
      "QueryEndDate": "2023-01-01"
    }
  }
})
    headers = {
      'Prefer': 'respond-async',
      'Content-Type': 'application/json',
      'Authorization': 'Token '+token
    }    


    response = requests.request("POST", url, headers=headers, data=payload)


    return response.status_code
    
    


url = "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractWithNotes"

tm1 = time.perf_counter()
with concurrent.futures.ThreadPoolExecutor(max_workers=100) as executor:

    futures = []
    for iter in range(1, 200):
        futures.append(executor.submit(get_status, url=url))

    for future in concurrent.futures.as_completed(futures):
        print(future.result())

tm2 = time.perf_counter()
print(f'Total time elapsed: {tm2-tm1:0.2f} seconds')

gareth.teage

Hi @Jirapongse ,

Thanks for the example to reach the limit but on checking this is not what the user needs.

What they are after is something like the below:

"

In your API documentation there should be examples of what headers and body will be received for each kind of error, for example for a 400 error you have an example of the headers and body here https://selectapi.datascope.refinitiv.com/RestApi.Help/Home/KeyMechanisms?ctx=Extractions&opn=Extract&sce=Validation%20Error%20(400%20status%20code)&stp=1&tab=3&uid=Validation

I need a similar example for a 429 error."

Jirapongse

@gareth.teage

The error 429 can't be demonstrated by using the simple request messages.

Moreover, typically we create examples that demonstrate the valid usages.

If the client needs more examples on that page, please contact the product team directly for further consideration.

gareth.teage

Hi @Jirapongse,

Thanks for sending me the below example response via Teams, I will forward.

HTTP/1.1 429 Too Many Requests
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
BeginRequestTime: 09:16:37.3190382
BeginRequestDate: 2023-11-29
X-Request-Execution-Correlation-Id: CiD/9008895/AAAAAA.08b8e48231894ac0/RA
X-App-Id: Custom.RestApi
X-App-Version: 17.3.161.64
CPUUtilization: 20.83539
RequestsPerSec: 22.10375
W3WP-PrivateBytes: 1597248
Date: Wed, 29 Nov 2023 09:16:36 GMT
Content-Length: 187

{
    "error": {
        "message": "Too many requests for throttling category \"OnDemandExtractions\", user id \"<UserID>\". Approximately 165 requests were made in 60 seconds to 15 active server(s)."
    }
}

Kind regards,

Gareth