Python programs hangs after aycnio exception due to Pyppeteer unexpectedly closing

Question

My pyppeteer connection unexpectedly closed, and it left my Python program hanging instead of shutting down and properly logging the error. Does anyone know how to properly catch this exception and properly exit from Python program?

Here is part of my script:

 async def _proxy_browser(self, url, headless = False):

    args = []

    for i in range(5):

        try:
            browser = await launch(headless = headless,
                                   args = args,
                                   defaultViewport = None)
            

            page = await browser.newPage()
            await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0')
            await page.goto(url, {'waitUntil' : 'domcontentloaded'})
            time.sleep(5)
           
            content = await page.content()

            return content

        except (pyppeteer.errors.PageError, pyppeteer.errors.TimeoutError) as e:
          print('error', e)
          time.sleep(2)
          continue

        finally:
            time.sleep(1)
            await browser.close()

    return 

def proxy_browser_request(self, url, headless = False):

    loop = asyncio.get_event_loop()
    response = loop.run_until_complete(self._proxy_browser(url, headless))

    return response

Here is the traceback I received:

[E:pyppeteer.connection] connection unexpectedly closed
    Task exception was never retrieved
    future: <Task finished name='Task-1379' coro=<Connection._async_send() done, defined at C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\connection.py:69> exception=InvalidStateError('invalid state')>
    Traceback (most recent call last):
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\connection.py", line 73, in _async_send
        await self.connection.send(msg)
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\websockets\protocol.py", line 567, in send
        await self.write_frame(True, opcode, data)
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\websockets\protocol.py", line 1083, in write_frame
        await self.ensure_open()
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\websockets\protocol.py", line 803, in ensure_open
        raise self.connection_closed_exc()
    websockets.exceptions.ConnectionClosedError: code = 1006 (connection closed abnormally [internal]), no reason
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\connection.py", line 79, in _async_send
        await self.dispose()
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\connection.py", line 170, in dispose
        await self._on_close()
      File "C:\Users\name\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\connection.py", line 151, in _on_close
        cb.set_exception(_rewriteError(
    asyncio.exceptions.InvalidStateError: invalid state

Answer 1

Assume 1006 status tells you that site can understand - your session is automated process and drops your ws connection, you have to play with chrome args or try pyppeteer stealth extension (it is ported on python - use google)