i am running a matrix-synapse server behind an nginx using docker
i had in my dockerfile a line :
image: matrixdotorg/synapse:latest (which was, in hindsight, dumb...)
and i think was around 1.80-88 ....
the server stopped speaking to my element client on a tablet, so i thought to update the image....
stopped the server, pulled, restarted.... server didn't restart....
i tried to go back to 1.88, no luck, went forward explicitely to 1.95, same error for all versions:
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,979 - synapse.federation.federation_server - 1387 - INFO - main - Registering federation query handler for 'profile'
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,980 - synapse.federation.federation_server - 1367 - INFO - main - Registering federation EDU handler for 'm.presence'
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,983 - synapse.federation.federation_server - 1367 - INFO - main - Registering federation EDU handler for 'm.typing'
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,984 - synapse.federation.federation_server - 1387 - INFO - main - Registering federation query handler for 'directory'
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,984 - twisted - 275 - INFO - main - Redirected stdout/stderr to logs
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,984 - synapse.app.homeserver - 173 - INFO - sentinel - Running
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,987 - synapse.app.homeserver - 29 - INFO - sentinel - Set file limit to: 1048576
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: 2023-11-08 23:00:31,989 - twisted - 275 - CRITICAL - sentinel - Unhandled Error
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: Traceback (most recent call last):
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/internet/base.py", line 502, in fireEvent
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: DeferredList(beforeResults).addCallback(self._continueFiring)
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 645, in addCallback
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: return self.addCallbacks(callback, callbackArgs=args, callbackKeywords=kwargs)
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 558, in addCallbacks
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: self._runCallbacks()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 1101, in _runCallbacks
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: current.result = callback( # type: ignore[misc]
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: --- <exception caught here> ---
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/internet/base.py", line 514, in _continueFiring
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: callable(*args, **kwargs)
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/enterprise/adbapi.py", line 237, in _start
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: return self.start()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/enterprise/adbapi.py", line 247, in start
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: self.threadpool.start()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 158, in start
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: self.adjustPoolsize()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 304, in adjustPoolsize
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: self._team.grow(self.min - self.workers)
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_team.py", line 117, in grow
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: @self._coordinator.do
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_threadworker.py", line 107, in do
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: working.pop(0)()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_team.py", line 120, in createOneWorker
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: worker = self._createWorker()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_pool.py", line 55, in limitedWorkerCreator
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: return ThreadWorker(startThread, Queue())
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_threadworker.py", line 49, in __init__
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: startThread(work)
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/site-packages/twisted/_threads/_pool.py", line 49, in startThread
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: return threadFactory(target=target).start()
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: File "/usr/local/lib/python3.11/threading.py", line 964, in start
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: _start_new_thread(self._bootstrap, ())
Nov 9 00:00:31 localhost matrix.nohkumado.eu[1339]: builtins.RuntimeError: can't start new thread
followed by some more errors of the same type
so something fundamental seems to be broken, since the load on that machine is otherwise light?
how can i get this working again?
thanks in advance
I had the same issue. Upgrading docker-ce resolved the issue:
References here: