I am trying to execute clean, introspect operations on baremetal nodes via openstack undercloud but it fails and after researching and getting out the logs, it looks like ironic-inspector is throwing the error full log is given below.
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines [-] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last):
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines connection.scalar(select([1]))
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 912, in scalar
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines return self.execute(object_, *multiparams, **params).scalar()
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines return meth(self, multiparams, params)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines return connection._execute_clauseelement(self, multiparams, params)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines distilled_params,
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines e, statement, parameters, cursor, context
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines util.raise_from_cause(newraise, exc_info)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines reraise(type(exception), exception, tb=exc_tb, cause=cause)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines cursor, statement, parameters, context
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines result = self._query(query)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines conn.query(q)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 516, in query
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 727, in _read_query_result
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines result.read()
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1066, in read
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet()
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 656, in _read_packet
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines packet_header = self._read_bytes(4)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 702, in _read_bytes
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query")
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)
2021-07-14 09:36:13.750 7 ERROR oslo_db.sqlalchemy.engines
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines [-] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last):
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines connection.scalar(select([1]))
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 912, in scalar
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines return self.execute(object_, *multiparams, **params).scalar()
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines return meth(self, multiparams, params)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines return connection._execute_clauseelement(self, multiparams, params)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines distilled_params,
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines e, statement, parameters, cursor, context
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines util.raise_from_cause(newraise, exc_info)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines reraise(type(exception), exception, tb=exc_tb, cause=cause)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines cursor, statement, parameters, context
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines result = self._query(query)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines conn.query(q)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 516, in query
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 727, in _read_query_result
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines result.read()
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1066, in read
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet()
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 667, in _read_packet
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines "Lost connection to MySQL server during query")
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)
2021-07-14 09:43:58.979 7 ERROR oslo_db.sqlalchemy.engines
2021-07-14 10:06:05.165 8 DEBUG ironic_inspector.common.service_utils [-] logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2571
2021-07-14 10:06:05.310 8 DEBUG oslo_service.service [-] logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2571
2021-07-14 10:06:05.526 8 DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:307
Why I am posting this here because did not get any response on the open source community, I am stuck, This is also not purely related to MySQL; because Mysql is also part of the installation process of Undercloud. Maybe I am missing something here.
Additional Info: My Undercloud Deployment is kolla Containerized