'utf-8' codec can't decode byte 0x80 in position 22: invalid start byte

nitin.jagwani

When testing websocketapi example using python for market_price, I have encountered an error 'utf-8' codec can't decode byte 0x80 in position 22: invalid start byte.

I have tested the same using the wsapi application tool and I was able to retrieve the market price using my hostname and port 15000 successfully. However, I end up getting an error "'utf-8' codec can't decode byte 0x80 in position 22: invalid start byte" when using python.

python market_price.py --hostname "myhostname" --port "15000"

'utf-8' codec can't decode byte 0x80 in position 22: invalid start byte

WebSocket Closed

Could you please assist?

pf

Hi @nitin.jagwani ,

We detected same issue when we use websocket-client lib.
(see https://websocket-client.readthedocs.io/en/latest/faq.html#i-get-the-error-utf8-codec-can-t-decode-byte-0x81-in-position-0)
It can be fixed by overriding websocket.read_headers function to replace 'utf-8' decoding with 'ISO-8859-1' decoding.

import websocket
def my_read_headers(sock):
    status = None
    status_message = None
    headers = {}
    trace("--- response header ---")

    while True:
        line = recv_line(sock)
        trace(line)
        line = line.decode('ISO-8859-1').strip()
        # line = line.decode('utf-8')
        line = line.strip()
        if not line:
            break
        trace(line)
        if not status:

            status_info = line.split(" ", 2)
            status = int(status_info[1])
            if len(status_info) > 2:
                status_message = status_info[2]
        else:
            kv = line.split(":", 1)
            if len(kv) == 2:
                key, value = kv
                if key.lower() == "set-cookie" and headers.get("set-cookie"):
                    headers["set-cookie"] = headers.get("set-cookie") + "; " + value.strip()
                else:
                    headers[key.lower()] = value.strip()
            else:
                raise WebSocketException("Invalid header")

    trace("-----------------------")

    return status, headers, status_message

websocket.read_headers = my_read_headers

umer.nalla

Hi @nitin.jagwani

Please advise which RIC code(s) you are subscribing to when you see the above error.

Philip Voldsgaard

Hi

This issue still persists, and I just encountered it when streaming through a platform-session, where the utf-8 decoding caused trouble with my particular application-id.

I realize that managing the WebSocket library is out of scope on this forum, however, I would suggest that the RD library incorporate a temporary handling of this error, so that end-users don't have to bother with it.

Details on my specific case.
-----------------------------------------------------------------------------------------------------------------------------------