AttributeError running Django site on Mac 11.0.1

Question

I'm getting an error running a django site locally that was working fine before I updated my Mac OS to 11.0.1. I'm thinking this update is the cause of the problem since nothing else was really changed between when it was working and now.

10:15:05 worker.1 | Traceback (most recent call last):
10:15:05 worker.1 |   File "/usr/local/bin/celery", line 5, in <module>
10:15:05 worker.1 |     from celery.__main__ import main
10:15:05 worker.1 |   File "/usr/local/lib/python2.7/site-packages/celery/__init__.py", line 133, in <module>
10:15:05 worker.1 |     from celery import five  # noqa
10:15:05 worker.1 |   File "/usr/local/lib/python2.7/site-packages/celery/five.py", line 20, in <module>
10:15:05 worker.1 |     from kombu.five import monotonic
10:15:05 worker.1 |   File "/usr/local/lib/python2.7/site-packages/kombu/five.py", line 56, in <module>
10:15:05 worker.1 |     absolute_to_nanoseconds = CoreServices.AbsoluteToNanoseconds
10:15:05 worker.1 |   File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 379, in __getattr__
10:15:05 worker.1 |     func = self.__getitem__(name)
10:15:05 worker.1 |   File "/usr/local/Cellar/python@2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 384, in __getitem__
10:15:05 worker.1 |     func = self._FuncPtr((name_or_ordinal, self))
10:15:05 worker.1 | AttributeError: dlsym(RTLD_DEFAULT, AbsoluteToNanoseconds): symbol not found

Here is my brew config

HOMEBREW_VERSION: 2.6.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 1d5e354cc2ff048bd7161d95b3fa7f91dc9dd081
Last commit: 2 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: fdb83fcfb482e5ed1f1c3c442a85b99223fcabeb
Core tap last commit: 27 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.rZ1F30XomO/org.macosforge.xquartz:0
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit icelake
Clang: 12.0 build 1200
Git: 2.24.3 => /Applications/Xcode-beta.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
Java: 14.0.2, 1.8.0_265
macOS: 11.0.1-x86_64
CLT: 12.3.0.0.1.1605054730
Xcode: 12.3 => /Applications/Xcode-beta.app/Contents/Developer
XQuartz: 2.7.11 => /opt/X11

Typically I'll run the site with a virtualenv running python 2.7.15, I was getting the same error with that. I reinstalled python with pyenv and remade the virtualenv but the same error appeared. I'm running Django 1.10.8 with Kombu 3.0.37

Answer 1

Ok this is a dirty workaround for Big Sur compatibility:

https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

so in order to find these libraries I just put the static paths in the find_library function at <path to your Python 2 installation>/lib/python2.7/ctypes/util.py just below os.name == "posix" and sys.platform == "darwin":

if name == 'CoreServices':
    return '/System/Library/Frameworks/CoreServices.framework/CoreServices'
elif name == 'libSystem.dylib':
    return '/usr/lib/libSystem.dylib'

at the end it would look like this:

if os.name == "posix" and sys.platform == "darwin":
    from ctypes.macholib.dyld import dyld_find as _dyld_find
    def find_library(name):
        if name == 'CoreServices':
            return '/System/Library/Frameworks/CoreServices.framework/CoreServices'
        elif name == 'libSystem.dylib':
            return '/usr/lib/libSystem.dylib'

        possible = ['@executable_path/../lib/lib%s.dylib' % name,
                    'lib%s.dylib' % name,
                    '%s.dylib' % name,
                    '%s.framework/%s' % (name, name)]
        for name in possible:
            try:
                return _dyld_find(name)
            except ValueError:
                continue
        return None

Answer 2

A less invasive way of doing this would be to monkey patch ctypes dynamically, before invoking whatever code has the problem. That way you can avoid editing core python files.

I ran into a very similar issue where an old version of celery would not start on MacOS, because of this same error from import vine.five.

Since celery is directly invoked as a script, the following patch to bin/celery in my virtualenv resolved the issue:

import os, sys
if os.name == "posix" and sys.platform == "darwin":
    import ctypes.util
    _old_ctypes_find_library = ctypes.util.find_library
    def _new_ctypes_find_library(name):
        if name == 'CoreServices':
            return '/System/Library/Frameworks/CoreServices.framework/CoreServices'
        elif name == 'libSystem.dylib':
            return '/usr/lib/libSystem.dylib'
        else:
            return _old_ctypes_find_library(name)
    ctypes.util.find_library = _new_ctypes_find_library

Answer 3

I had the same issue after Bigsur 11.5.1 update (maybe, just coincidently). The answer provided above still works, but I was confused on what to do as I'm new to this. Here are the steps to follow:

1. Look at the error you get to find the path to the file util.py where your python 2.7 is installed. Open it using a text editor.
2. With the file opened, search for "posix", make sure it look like this "posix" and sys.platform == "darwin".
3. Find the line def find_library(name):, and add the script in the answer above under it.

I have included screenshots for reference.