No error type for "Connection closed unexpectedly". How to catch?

Question

In attempting to stream a very large number of reports from an API to Bigquery my App Engine app occasionally hits an error that I don't know how to catch.

I am already handling urlfetch_errors.InternalTransientError and urlfetch_errors.DeadlineExceededError.

while True:
    try:
        result = service.tabledata().insertAll(
            projectId=svcdata['project_id'],
            datasetId=cfg.DATASET_ID,
            tableId=convert_table_id(site),
            body=insert_all_data).execute(num_retries=cfg.STREAM_RETRIES)
        break
    except (urlfetch_errors.InternalTransientError, urlfetch_errors.DeadlineExceededError):
        log.info("An transient error occured while streaming to BQ. Retrying in 30 secs.")
        time.sleep(30)

Typically I use the type of the error such as DeadlineExceededError to catch an error. However this error is just 'error'.

Traceback (most recent call last):
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/f~stat-data-logging/20190801t112501.420000215685215810/controllers/cron.py", line 19, in get
    runStatDownload()
  File "/base/data/home/apps/f~stat-data-logging/20190801t112501.420000215685215810/utils/utils_stat.py", line 138, in runStatDownload
    streamResult = stream_rows_to_bigquery(data, report.title)
  File "/base/data/home/apps/f~stat-data-logging/20190801t112501.420000215685215810/utils/utils_bigq.py", line 165, in stream_rows_to_bigquery
    body=insert_all_data).execute(num_retries=cfg.STREAM_RETRIES)
  File "./lib/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "./lib/googleapiclient/http.py", line 846, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "./lib/googleapiclient/http.py", line 164, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "./lib/oauth2client/transport.py", line 159, in new_request
    credentials._refresh(orig_request_method)
  File "./lib/oauth2client/client.py", line 749, in _refresh
    self._do_refresh_request(http)
  File "./lib/oauth2client/client.py", line 780, in _do_refresh_request
    body=body, headers=headers)
  File "./lib/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "./lib/httplib2/__init__.py", line 2135, in request
    cachekey,
  File "./lib/httplib2/__init__.py", line 1796, in _request
    conn, request_uri, method, body, headers
  File "./lib/httplib2/__init__.py", line 1737, in _conn_request
    response = conn.getresponse()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/941d77da994078b1/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 536, in getresponse
    'An error occured while connecting to the server: %s' % e)
error: An error occured while connecting to the server: Connection closed unexpectedly by server at URL: https://oauth2.googleapis.com/token

I would like to add the error shown in my stack trace to my error handling. What is the best way to do this?

Answer 1

I think you can do the follwoing:

while True:
    try:
        result = service.tabledata().insertAll(
            projectId=svcdata['project_id'],
            datasetId=cfg.DATASET_ID,
            tableId=convert_table_id(site),
            body=insert_all_data).execute(num_retries=cfg.STREAM_RETRIES)
        break
    except (urlfetch_errors.InternalTransientError, urlfetch_errors.DeadlineExceededError):
        log.info("An transient error occured while streaming to BQ. Retrying in 30 secs.")
        time.sleep(30)
    except:
        print "Unexpected error:", sys.exc_info()[0]
        raise