Backend error. 400 Bad Request

PilOop

Hello guys!

The famous "Backend error 400 Bad Request" error

I wanted to approach the subject with you for that I made a simple code that always returns me a 400 Bad Request.

The code below reads a file that contains 600 rics. The code then queries each ric 1 by 1 to return the set of options. (BFBB161806000.U, etc.)

# Rics (600)
universe = pd.read_csv('rics.csv', delimiter=',', header=None)
universe = universe.values.T.tolist()[0]

# --------------------------
# Get Ric Option
# --------------------------
for u in universe:
    data_grid, err = ek.get_data(u,['BKGD_REF'])
    time.sleep(4) # with or without..

after a few minutes I get the following error:

Traceback (most recent call last):
  File "C:\thomsonreuters\demo.py", line 44, in <module>
    data_grid, err = ek.get_data(u,['BKGD_REF'])
  File "C:\Users\Utilisateur\AppData\Local\Programs\Python\Python36-32\lib\site-packages\eikon\data_grid.py", line 141, in get_data
    result, err = eikon.json_requests.send_json_request(DataGrid_UDF_endpoint, payload, debug=debug, error=error)
  File "C:\Users\Utilisateur\AppData\Local\Programs\Python\Python36-32\lib\site-packages\eikon\json_requests.py", line 88, in send_json_request
    check_server_error(result)
  File "C:\Users\Utilisateur\AppData\Local\Programs\Python\Python36-32\lib\site-packages\eikon\json_requests.py", line 138, in check_server_error
    raise requests.HTTPError(error_message, response=server_response)
requests.exceptions.HTTPError: Backend error. 400 Bad Request

In the file eikon\Profile.py I see a timeout of 30s

self.application_id = application_id
        self.port = get_scripting_proxy_port()
        self.url = "http://localhost:{0}/api/v1/data".format(self.port)
        self.streaming_url = "ws://localhost:{0}/?".format(self.port)
        self.session = Session()
        self.session.trust_env = False
        self.timeout = 30

The 400 appears when the session expires? after 30s?

Are the get_data() sessions properly closed at the end ? is it possible to improve this? without putting a time.sleep of 60s .. the continuation of the code recovers data on each options therefore the treatment is still longer.

Thanks,

Jirapongse

The application can catch the exception and re-request that chain RIC for specific times. The code looks like:

universe = pd.read_csv('chain.csv', delimiter=',', header=None)
universe = universe.values.T.tolist()[0]
for u in universe:
    i = 0    
    while (i < 3):
        try:
            data_grid, err = ek.get_data(u,['BKGD_REF'])
            print ("Success:", u)
            i = 3
        except:
            print ("Exception:", u)
            i = i + 1

Jirapongse

It could be server timeout. Requesting one RIC at a time is not effective.

You need to request by using a list of instruments instead.

def chunks(chunkable, n):
    """ Yield successive n-sized chunks from l.
    """
    for i in range(0, len(chunkable), n):
        yield chunkable[i:i+n]

universe = pd.read_csv('rics.csv', delimiter=',', header=None)
universe = universe.values.T.tolist()[0]
for l in list(chunks(universe, 200)):
    data_grid, err = ek.get_data(l,['BKGD_REF'])
    print (data_grid)

PilOop

Thank for your answer, usually I do batch processing (like your code above). Except that I did not specify that my file rics.csv contained rics options ie:

0#BFB*.U
0#AIV*.U
0#AJG*.U
0#BXP*.U
0#AES*.U
0#AVB*.U
0#APH*.U
0#AIZ*.U
0#CCI*.U
0#BDX*.U
0#AYI*.U
0#BWA*.U
0#ADSK*.U
etc..

So I have to do it 1 by 1. Does the 400 appear if a ric doesn't exist ?

Jirapongse

From my test, it returns NaN if a RIC is not found.

Could you please share the full rics.csv file?

PilOop

Your code returns "NaN" because you are doing batch processing, you have to do it 1 by 1. Please, tested this:

data_grid, err = ek.get_data(['0#AVB*.U','0#APH*.U'],['BKGD_REF'])
print(data_grid)
data_grid, err = ek.get_data(['0#AVB*.U'],['BKGD_REF'])
print(data_grid)
data_grid, err = ek.get_data(['0#APH*.U'],['BKGD_REF'])
print(data_grid) 

The results are different.

Jirapongse

I have tested it with an invalid RIC, such as BFBB161806000x.U and it also returns NaN.

One scenario which cause 400 Bad Request is server timeout. If the request takes more than a specific time slice to process on the server, the request will be timeout and the server will send a response with 400 code and "Backend error. 400 Bad Request" text.

HTTP/1.1 200 OK
...
Content-Length: 65
{"ErrorCode":400,"ErrorMessage":"Backend error. 400 Bad Request"}

Therefore, if this issue happens randomly and is not link to a specific request, it could be server time.

Could you please share the full rics.csv file? Therefore, we can test it and verify if it is server timeout.

PilOop

Your code returns "NaN" because there is no BKGD_REF on options (BFBB161806000x.U)

But before questioning the option:

data_grid, err = ek.get_data(['BFBB161806000x.U'],['BKGD_REF'])

I need to retrieve the list of options available for each RIC.

Example for AVN :

data_grid, err = ek.get_data(['0#AVB*.U'],['BKGD_REF']) 
print(data_grid) 

Result :

===================== RESTART demo.py =====================
          Instrument         BKGD_REF
0                AVB             None
1    AVBB161813500.U  AVBB1618C135000
2    AVBN161813500.U  AVBN1618C135000
3    AVBB161814000.U  AVBB1618C140000
4    AVBN161814000.U  AVBN1618C140000
5    AVBB161814500.U  AVBB1618C145000
6    AVBN161814500.U  AVBN1618C145000
7    AVBB161815000.U  AVBB1618C150000
8    AVBN161815000.U  AVBN1618C150000
9    AVBB161815500.U  AVBB1618C155000
10   AVBN161815500.U  AVBN1618C155000
11   AVBB161816000.U  AVBB1618C160000

But you have to do it 1 by 1. Because if you do it by batch

data_grid, err = ek.get_data(['0#AVB*.U','0#APH*.U'],['BKGD_REF'])
print(data_grid)

Result :

Instrument  BKGD_REF
0   0#AVB*.U       NaN
1   0#APH*.U       NaN

Sorry, I can not provide the list of 600 rics however here are 25 rics that generates a 400:

0#AJG*.U
0#BXP*.U
0#AES*.U
0#TRMB*.U
0#UMPQ*.U
0#WAB*.U
0#ADSK*.U
0#APD*.U
0#ADP*.U
0#BBBY*.U
0#ALK*.U
0#AZO*.U
0#ADS*.U
0#AMAT*.U
0#AIG*.U
0#AMGN*.U
0#ABT*.U
0#ADM*.U
0#APA*.U
0#BSX*.U
0#CBS*.U
0#CBSH*.U
0#APC*.U
0#AXP*.U
0#AMG*.U

then with the following code, (I replaced BKGD_REF with CF_NAME)

# Rics (0#AMG*.U, 0#AXP*.U, etc..)
universe = pd.read_csv('rics.csv', delimiter=',', header=None)
universe = universe.values.T.tolist()[0]

# --------------------------
# Get Ric Option
# --------------------------
for u in universe:
    data_grid, err = ek.get_data(u,['CF_NAME'])
    print(data_grid)

The issue with error 400 is that it appears randomly and break the code

rics.txt. (please, change the extension to csv)

Jirapongse

I am unable to replicate the issue with those 25 chain RICs. However, I am quite sure that it is a server timeout issue.

Next, I will add more chain RICs into the rics.csv to replicate the issue.