HttpRequestException. Error while copying content to a stream. The response ended prematurely.

glyn.jagger
glyn.jagger
in Entity Search

Making 95 calls to https://api-eit.refinitiv.com/permid/search?q=ticker:{ticker} from a C# application in a loop we get a HttpRequestException, "Error while copying content to a stream" with inner exception "The response ended prematurely" when reading the response content. The resonse statuscode is 200 so it's not rate-limiting.

Here's some sample code to repeat, with a crude re-try that shows the issue is transient as we actually get data back for all tickers with the retry in place.

using System.Net.Http.Headers;

namespace MyApp
{
  internal class Program
  {
    static async Task Main(string[] args)
    {
      try
      {
        var tickerList = new List<string>()
        {
          "AAF", "AAL", "ABF", "ADM", "AHT", "ANTO", "AUTO", "AV/", "AVV", "AZN", "BA/",
          "BARC", "BATS", "BDEV", "BKG", "BLND", "BME", "BNZL", "BP/", "BRBY", "BT/A", 
          "CCH", "CNA", "CPG", "CRDA", "CRH", "CTEC", "DCC", "DGE", "DPH", "EDV", 
          "ENT", "EXPN", "FCIT", "FLTR", "FRAS", "FRES", "GLEN", "GSK", "HBR", "HL/", 
          "HLMA", "HLN", "HSBA", "HSV", "IAG", "ICP", "IHG", "III", "IMB", "INF", 
          "ITRK", "JD/", "KGF", "LAND", "LGEN", "LLOY", "LSEG", "MNDI", "MNG", "MRO", 
          "NG/", "NWG", "NXT", "OCDO", "PHNX", "PRU", "PSH", "PSN", "PSON", "REL", 
          "RIO", "RKT", "RMV", "RR/", "RS1", "RTO", "SBRY", "SDR", "SGE", "SGRO", 
          "SHEL", "SKG", "SMDS", "SMIN", "SMT", "SN/", "SPX", "SSE", "STAN", "STJ", 
          "SVT", "TSCO", "TW/", "ULVR", "UTG", "UU/", "VOD", "WPP", "WTB",
        };

        using var httpClient = new HttpClient();
        var accessToken = "PersonalAccessTokenDetails
        foreach (var ticker in tickerList)
        {
          var retriesAllowed=1;
          do
          {
            var request = new HttpRequestMessage(HttpMethod.Get, 
                          $"https://api-eit.refinitiv.com/permid/search?q=ticker:{ticker}");
            request.Headers.Add("x-ag-access-token", accessToken);
            request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")
            var response = await httpClient.SendAsync(request, 
                           HttpCompletionOption.ResponseHeadersRead);

            Console.WriteLine($"[{response?.StatusCode}] ticker:{ticker}");
            if ((response is {IsSuccessStatusCode: false} or null))
              break;

            try
            {
              //Deserialize resonseBody removed for brevity
              var responseBody = await response.Content.ReadAsStringAsync(                            
              retriesAllowed = 0;
            }
            catch (HttpRequestException reqEx)
            {
              Console.WriteLine($"{ticker} failed: {reqEx.Message} [{reqEx.InnerException?.Message}] retrying...");
            }
          } while (retriesAllowed > 0);
        }
      }
      catch (Exception ex)
      {
        Console.WriteLine($"{ex.Message}:{ex.InnerException?.Message}");
      }
      finally
      {
        Console.WriteLine("press ENTER to exit");
        Console.ReadLine();
      }
    }
  }
}
Tagged:

Best Answer

  • Jirapongse
    Jirapongse
    Answer ✓

    @glyn.jagger

    Thank you for sharing the code. I can replicate this issue.

    From my test, the server is not quite stable when handling a lot of requests. It could be this error.

    127.0.0.1:58294: GET https://api-eit.refinitiv.com/permid/search?q=ticker:VOD
                  << 200 OK (content missing)
     << HTTP/1 protocol error: peer closed connection without sending complete message body (incomplete chunked read)

    The server cut the connection while the application was reading the data. Therefore, it is a good idea to implement retrying in the code.

    I will contact the product team to see what they can do regarding this issue.

Answers

  • Jirapongse
    Jirapongse

    @glyn.jagger

    The product team mentioned that it could be a network issue.

    Therefore, implementing the retry logic in the code is the best way to handle this kind of issue.

Categories