Problems with downloading a large number of news articles using Eikon API [ Backend error. 503 Servi

j15s
j15s
in Eikon Data APIs

I am using my university's Eikon terminal and I have 2 questions regarding downloading a large number of news articles for the purpose of academic research.

To provide more context, I am attempting to download a large number of news articles connecting to the Eikon's API through Python. I want to download Refinitive News articles written in English that were published between 2000 and 2019 and that contain two key phrases ('climate change' and 'global warming'). Based on the previously published research articles on this topic, I expect there to be around 35 000 articles that match these search criteria.

1) Firstly, I wanted to ask whether this is possible under the academic subscription held by my uninversity. I have successfully connected to the Eikon's API using the App Key and downloaded small samples of news articles, but I am wondering whether this would work for a much bigger sample. I was instructed to post this question here.

2) Secondly, I wanted to ask a specific question regarding an error message (Backend error. 503 Service Unavailable). I wanted to download headlines of the news articles matching the search criteria and received this error message. This function has worked previously on a smaller news sample. I include the code and the error message below. What might be the issue in this case?

I would be very grateful for any help!

Code: 

import eikon as ek
 import numpy as np
 import pandas as pd
 import datetime
 from datetime import time
 import warnings

 ek.set_app_key('xxx')

 df_headlines = ek.get_news_headlines('"global warming" AND "climate change" AND Language:LEN',date_from = "2000-01-01", date_to = "2022-12-04")



Error message:

C:\Users\eikonwin10\PycharmProjects\pythonProject2\venv\Scripts\python.exe C:/Users/eikonwin10/PycharmProjects/pythonProject2/headlines.py

2022-12-06 13:19:35,536 P[12240] [MainThread 13972] Backend error. 503 Service Unavailable

2022-12-06 13:19:35,536 P[12240] [MainThread 13972] HTTP request failed: EikonError-Backend error. 503 Service Unavailable

Traceback (most recent call last):

File "C:\Users\eikonwin10\PycharmProjects\pythonProject2\venv\lib\site-packages\eikon\json_requests.py", line 124, in send_json_request

_check_server_error(result)

File "C:\Users\eikonwin10\PycharmProjects\pythonProject2\venv\lib\site-packages\eikon\json_requests.py", line 215, in _check_server_error

raise EikonError(int(server_response['ErrorCode']), error_message)

eikon.eikonError.EikonError: Error code 503 | Backend error. 503 Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "C:\Users\eikonwin10\PycharmProjects\pythonProject2\headlines.py", line 10, in <module>

df_headlines = ek.get_news_headlines('"global warming" AND "climate change" AND Language:LEN',date_from = "2000-01-01", date_to = "2022-12-04")

File "C:\Users\eikonwin10\PycharmProjects\pythonProject2\venv\lib\site-packages\eikon\news_request.py", line 121, in get_news_headlines

result = eikon.json_requests.send_json_request(News_Headlines_UDF_endpoint, payload, debug=debug)

File "C:\Users\eikonwin10\PycharmProjects\pythonProject2\venv\lib\site-packages\eikon\json_requests.py", line 134, in send_json_request

raise EikonError(eikon_err.code, eikon_err.message)

eikon.eikonError.EikonError: Error code 503 | Backend error. 503 Service Unavailable

Process finished with exit code 1

Tagged:

Best Answer

  • Jirapongse
    Jirapongse
    Answer ✓

    @j15s

    Sorry about the issue that you are facing.

    The get_news_headlines function has limitations, as mentioned on this page.

    get_news_headlines: The current limit value (10-Oct-2019) is 100 headlines per request. The depth of history is 15 month.

    The query should look like this:

     df_headlines = ek.get_news_headlines(
         '\\"global warming\\" AND \\"climate change\\" AND Language:LEN',
         date_from = "2020-01-01", 
         date_to = "2022-12-04",
         count=100)

    The output is:

    1670379278219.png

    To get more news headlines, you may need to contact your Refinitiv account team or sales team to find another solution or product.

    I hope that this information is of help

Answers

  • j15s
    j15s

    @Jirapongse

    Thank you very much for your answer!

    Is Refinitiv News Archive the right product in this case? I have contacted the Refinitiv Account Team previously and was redirected here.




  • Jirapongse
    Jirapongse

    @j15s

    I found this product (Refinitiv News Archive) on MyRefinitiv.

    It delivers deep historical news content with Refinitiv’s metadata enriched data model. Archive files are delivered to customers via SFTP.

    You can contact your Refinitiv Account team directly for more information.

Categories