Automatic Decompression in HTTP request for Tick History Time and Sales report

leonardo.martins

Hi there!

I'm trying to download the Tick History Time and Sales report via R library httr.

Apparently, the GET method decompress it automatcally, what could be a real problem for large reports.

Is there a way to work around this?

I read in this page that if the request doesn't have the "Accept-Encoding" header, the response wouldn't have the "Content-Encoding" header and the client shouldn't decompress it, but that is not true, the "Content-Encoding" is still present.

So I don't know if the problem is with the R or with the API.

Christiaan Meihsl

leonardo.martins,

The TRTH servers deliver compressed (or not) data depending on several things, as described under heading "Compression" in this page. For an On Demand Time and Sales extraction I believe it should always deliver compressed data, whatever you set in the GET header.

What does the response header contain ? I would expect it to contain this:

Content-Encoding: gzip Content-Type: text/plain

If that is the case, the content is compressed.

But some HTTP clients automatically decompress data when they receive compressed data (Postman does that) I guess you have run into that as well. You must disable httr’s content decoding, using "config(http_content_decoding=0)" in the GET call.

Here is a code snippet that does it:

TRTHRawExtractionResults <- function(token,jobid,Path,Overwrite = TRUE) {
  url <- paste0("https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('",jobid,"')/$value")
  r <- httr::GET(url,add_headers(prefer = "respond-async",Authorization = token),config(http_content_decoding=0),write_disk(Path,Overwrite),progress())
  stop_for_status(r)
  return(r)
}

This is an extract from this article that describes an example of R code that does a time and sales extraction.

If that does not help, please post your code so we can have a look at it.

leonardo.martins

Thank you very much! It its working perfectly now.