full holdings for a fund and ETF DSS Rest API request generates Http error code : 408

rahul.deshmukh

Since 01.02.2021 i am getting below error:

java.io.IOException: Server returned HTTP response code: 408 for URL: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes

I am using below DSS Rest API refinitiv request for full holdings for a fund and ETF in my java code:

"ExtractionRequest": {
            "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.FundAllocationExtractionRequest",
            "ContentFieldNames": [
              "Security Description",
              "Allocation Asset Type",
              "Allocation Date",
              "Allocation Item",     
              "Allocation Percentage",      
              "Allocation ISIN",      
              "Market Value Currency",
              "Market Value Held"     
            ],
            "IdentifierList": {
              "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
              "InstrumentIdentifiers": [%s],
              "ValidationOptions": null,
              "UseUserPreferencesForValidationOptions": false
            },
            "Condition": {
              "FundAllocationTypes": [
                "Asset",
                "Currency",
                "FullHoldings",
                "IndustrySector",
                "InvestmentCountry",
                "TopTenHoldings"
              ]
            }
          }
        }

Below is connection request to url:

//Connection to the URL string https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes
    private static HttpURLConnection connection(String urlSpec) {
        HttpURLConnection connection = new URL(urlSpec).openConnection() as HttpURLConnection

        connection.setRequestProperty('Accept', 'application/json')

        connection.setRequestMethod("POST")
        connection.setRequestProperty("Content-Type", "application/json; utf-8")
        connection.setDoOutput(true)
        connection
    }

We are using the same API request since December 2020 but did not face any issue. We are running this process request everyday at 5am CET in the morning. Now if we try to run the process again at 1pm CET then it runs without any issue.

This is very critical issue for us.

zoya faberov

Hello @rahul.deshmukh,

My understanding from reading the question is that the code is running successfully at some times, but not at other. And this is a repeatable behavior, has happened more then once.

Therefore, we have to assume there is no issue with code.

A much likelier cause is connectivity or service availability issue. Especially, as 408 error is from timeout family of errors.

This forum is targeted at general questions and discussions targeted at Refintiiv API usage, the majority of the forum members are Refinitiv API developers, the moderators of the forums are Refinitiv API experts.

Datascope product support can help you, as a customer, to investigate this type of issue, and especially for a critical issue, in the future, would suggest raising this type of issue directly with Refinitiv Helpdesk Online -> Datascope.

At this time, have raised on your behalf, case # is 09586128 , please expect direct communication by email from DSS support team.

I would suggest investigating in parallel, with your local network admin/group, if any maintenance or network connectivity outages have coincided with your failed submission times- if you find out this is the case - please let us know.

Gurpreet

Hi @rahul.deshmukh,

HTTP 408 means request timeout - and that is what you are experiencing when your request works some times and not other. This could be happening due to the additional load on the DSS server during that time, or due to network contention on your side. Either way, with a long running extraction like yours, it is never a good idea to use synchronous request.

I would recommend that you try asynchronous request and make your application more robust to handle the error conditions. You can see the example of async in the Java and C# examples here. Add a HTTP header:

"Prefer", "respond-async, wait=60"

and keep checking your response for HTTP 202, until you get a HTTP 200.

rahul.deshmukh

please check my previous question in forum https://community.developers.refinitiv.com/questions/67550/full-holdings-for-a-fund-and-etf-dss-rest-api-requ.html?childToView=68146#comment-68146 where it has been mentioned that i should remove this line from http request:

connection.setRequestProperty('Prefer', 'respond-async')

Gurpreet

In that answer, I specifically mention:

Your application might have to wait a very long time for data, and intermediate network components like routers/proxies might drop your connection.

and this is what is happening now. I would strongly urge you to see how the DSS samples have implemented the extraction, and model your application around it.

rahul.deshmukh

Will be really helpful here if you can help here to resolve this issue. What changes i have to do in http request ? I confirmed from our side and there is no proxy or network connection issues.

rahul.deshmukh

I already confirmed from our side and we dont have any proxy or network connection issues. If you run the same request again different time it works then why it failed during sometimes ?

zoya faberov

Hello @rahul.deshmukh,

We can try and suggest, helping is our goal. However, many environment-specific issues do not yield themselves to resolution within the framework of the discussion forums.

I second @Gurpreet on the approach. I would try, in addition:

1. Include additional code, to resubmit the request on status 408 and document the resubmit, and consequently, review "the pattern". Once the status becomes 202 or 200, if it is 202, would keep checking status periodically (as shown in our examples), till it becomes 200, once the status is 200 would retrieve the result.

2. Consider, how large is your instrument list? How long does your request take, on average, at the time when it completes? Fund allocation request limit is 600 at this time, according to DSS Extraction Limits, if you are coming close to the limit, I would, as well as verifying to request async, as suggested by @Gurpreet, test dividing the request into several sub-lists, to see if this allows your request not to timeout at the times of high service load. Just a guess here, are any of the identifiers Chain RICs, and being expanded into many RICs as part of the request - then would count how many RICs in total, when expanded, and approach accordingly.

rahul.deshmukh

Hello @zoya.farberov,

My instrument list is about 3000. Normally it takes around 50 minutes to complete all the request. I am processing 100 instrument at a time which takes around 2 minutes. The failure occur when it start to process request . For example since yesterday it waits for 20 minutes around and throws an error 408.

My concern is how do i test with async and sync method when i said the request does not fail during next run at different time ? Why it started failing for example after 2 months and only during specific time at 5am CET ? Why doesnt it fail till before yesterday ?

zoya faberov

Hello @rahul.deshmukh,

If the error has first manifested on 01.02.2021 5AM CET, does it mean you have seen it twice? Or only once? From Feb 1st or from Jan 2nd?

Are you scheduling the job via cron/task scheduler? Would it make sense to schedule a small, no quota request call, such as search, at the same time, and see if it also fails with a timeout. That would allow to narrow down the possible causes a lot.

You can search the history of service alerts on My Refinitiv looking for your instrument set, on these dates, and to try to verify if there was any outage that was impacting:

I include resolved, otherwise you will not be able to discover what has happened unless it is still happening.

rahul.deshmukh

@zoya.farberov Basically we are using prod and simu server and scheduling control-m job at 5am and 5.15am for both funds and etf. We are using the same API with different instrument list. Its difficult to verify actually with Histrory Service alert for which instrument there is outage and whether this is impacting. Today also we faced the same issue at 5am and 5.15am CET both the job failed with 408. And then we try to run the job at 7.30am again it works fine without any perfromance issue.

Is it possible for you to send me the ISIN list( we are using Identifier type as ISIN) which got affected from 01.02.2021 till 03.02.2021 due to outage ? Then i can verify easily if we are using those ISINs in our API request or not ?

rahul.deshmukh

@zoya.farberov Below are logs from today during our API request for fonds:

2021-02-03 05:00:28,148 INFO b.b.r.d.d.Fworkflow [main] Processing run 1 with 100 items per run with end index 199 

2021-02-03 05:01:59,871 INFO b.b.r.d.d.Fworkflow [main] Successful DSS request to https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes 

2021-02-03 05:02:36,290 INFO b.b.r.d.d.Fworkflow [main] Processing run 2 with 100 items per run with end index 299 

2021-02-03 05:03:45,286 INFO b.b.r.d.d.Fworkflow [main] Successful DSS request to https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes 

2021-02-03 05:04:23,381 INFO b.b.r.d.d.Fworkflow [main] Processing run 3 with 100 items per run with end index 399 

2021-02-03 05:24:24,068 ERROR b.b.runner [main] Error during workflow  java.io.IOException: Server returned HTTP response code: 408 for URL: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractWithNotes

You can see from logs it waited for 20 minutes for processing of instrument from API requests and then throws an error 408. Same happened with ETF also waited for 20 minutes for processing and then throws and error 408.

Can you please suggest any changes in the http request which i can do may be to avoid this error ?

Like Gurpreet said may be i can try ? "Prefer", "respond-async, wait=60"

rahul.deshmukh

@Gurpreet After using:

"Prefer", "respond-async, wait=60"

Now i am getting http error code: 202 and few of my Instruments are not getting processed

Gurpreet

Yes, HTTP 202 is to be expected when requesting async. It means that server has received your query and is working on it. You will need to keep checking the status of your request, by retrying the new URL sent in the 202 response, until you get HTTP 200.

Is this what you tried?

zoya faberov

Hello @rahul.deshmukh,

There is a working approach on this, if you wish. As a customer, you can open a case directly on Refinitiv Helpdesk Online -> DSS -> content, include your list of instruments, dates, and get a confirmation from Refintiiv content experts whether any of these instruments were affected on these dates.

However, as increasing the timeout as suggested by @Gurpreet was effective to avoid the timeout and allow the requests to be accepted for processing (status 202) you seem to have the solution.

Once 202 status is received, your request is accepted for processing successfully, this is different from 208. In this case the request will be completed by the service, resulting in status check = 200 and job being ready. Please refer to our examples (see above from @Gurpreet) on how to check status by ExtractionId and once it becomes 200, retrieve the job by extractionId.

rahul.deshmukh

@zoya.farberov i am not able to open the link for DSS helpline which you have mentioned. I have already filled the API form for further investigation on this issue but did not receive any responce. The most surprising part for me here is we have another project where we using the same url request but with different instruments and we are not facing any issue there. There we are not rechecking the status of the request, by retrying the new URL.

@Gurpreet @zoya.farberov can you please specifically mentioned how to keep keep checking the status of your request, by retrying the new URL sent in the 202 response, until you get HTTP 200 ?

I have checked the java code example here https://developers.refinitiv.com/en/api-catalog/datascope-select/datascope-select-rest-api/download but did not find an example java file which can help in this case. Can you please mentioned java example for my case ?

rahul.deshmukh

Just want to mention few part of my code where i am checking http response:

if (connection.responseCode == HTTP_OK) {        
println("Got http response code: ${connection.responseCode}, message: ${connection.responseMessage}")

log.info("Successful DSS request to ${connection.getURL()}")

LazyMap json = jsonFromDssExtractionConnection(connection)

 //Process the data from the response JSON and put it in the Map object for processing        
tryToProcessMarketDataFromDssExtractionJson(json, ricMap)                   
def validMap = ricMap.findAll { key, val ->        
val.fund_isin != null        
}        
} 
else {             
log.info("Got http error code: ${connection.responseCode}: ${connection.inputStream.text}")         
}

Gurpreet

In the link that you attached, see DSS2OnDemandTimeSeries.java line: 172

zoya faberov

Hello @rahul.deshmukh, when you go to my.refinititv.com to you have a customer login? If you do, please login. If not, please contact your Refinitiv account manager, they will help you register. It is required to login as a customer, prior to being able to submit support cases.

Please note, this is a different login from dev portal, as anyone can register on dev portal, as long as they provide valid info and only customers can login into Refinitiv Helpdesk Online and submit support case requests.

zoya faberov

Hello @rahul.deshmukh,

Please refer in Java Code Examples to this place:

  // Poll the location URL until the extraction is completed:
  while (respStatusCode == 202) {
     wait(30);
     responseGet = httpclient.execute(requestGet);
     respStatusCode = responseGet.getStatusLine().getStatusCode();
     System.out.println("\nHTTP status: " + respStatusCode);
  } ...

This is not a "set in stone rule", but something like this you will need, to asynchronously process long-running requests, does this make sense?

rahul.deshmukh

@zoya.farberov @Gurpreet thanks i will check this code...@zoya.farberov i am able to login customer portal https://my.refinitiv.com/content/mytr/en/helpandsupport.html but dont see the Refinitv Helpdesk online option...If i open your link somehow its not opening. I would like to confirm from my instrument list if there is no outage during this time

zoya faberov

@rahul.deshmukh,

Once you are logged in, press Get Support at top right:

Next select options:

"Refinitiv Products and Content" ->

"I need help understanding content" ->

Select product -> Datascope.

I hope this helps?

rahul.deshmukh

@zoya.farberov @Gurpreet I tried with using async method and tried threading approach to retry the http connection request until get HTTP:200 it work till some request and then somewhat its getting hanged indefinately:

while (true) {
if (connection.responseCode >= HTTP_SERVER_ERROR) 
{
// Server/internal error, we can't do anything
// Maybe throw a custom Exception.
log.info("Got http error code: ${connection.responseCode}: ${connection.inputStream.text}")
break;
}

if (connection.responseCode != HTTP_OK) 
{
try 
{
// Adjust the delay between calls, in ms
Thread.sleep(1000);
} 
catch (final InterruptedException e) {
// Ignore
log.info("Got http error code: ${connection.responseCode}: ${connection.inputStream.text}")
}
// Try again
continue
}

if (connection.responseCode == HTTP_OK) 
{
println("Got http response code: ${connection.responseCode}, message: ${connection.responseMessage}")

log.info("Successful DSS request to ${connection.getURL()}")
//println(connection.getInputStream().getText())

.........
}
break
}
}

zoya faberov

Hello @rahul.deshmukh,

I would suggest trying the working status/asynch processing code from our Java examples.