Python 3.4 crashes when producing some – but not all – Cartopy maps with segmentation fault 11

Question

I have installed the Python mapping tool Cartopy on a Mac running El Capitan 10.11.6 with Python 3.4. I can use Cartopy to successfully plot some maps but, in some cases, the Python kernel dies with Segmentation Fault 11.

I wanted a setup that I could remove easily from my computer should the need arise. Therefore, I installed Python 3.4 and the necessary dependencies using fink:

$ fink install python34
$ fink install gdal2
$ fink install gdal2-dev
$ fink install proj
$ fink install libproj9

I then created a virtual environment using pyvenv (but also tried virtualenv and venv) and activated it.

In the activated virtual environment, I used pip to install:

$ pip install cython        # Successfully installed cython-0.25.2
$ pip install numpy         # Successfully installed numpy-1.12.1
$ pip install shapely       # Successfully installed shapely-1.5.17.post1
$ pip install pyshp         # Successfully installed pyshp-1.2.10
$ pip install pandas        # Successfully installed pandas-0.19.2 python-dateutil-2.6.0 pytz-2017.2 six-1.10.0
$ pip install matplotlib    # Successfully installed cycler-0.10.0 matplotlib-2.0.0 pyparsing-2.2.0
$ pip install pillow        # Successfully installed olefile-0.44 pillow-4.1.0
$ pip install pyepsg        # Successfully installed pyepsg-0.3.1
$ pip install scipy         # Successfully installed scipy-0.19.0
$ pip install OWSLib        # Successfully installed OWSLib-0.14.0 pyproj-1.9.5.1 requests-2.13.0
$ pip install mock          # Successfully installed mock-2.0.0 pbr-3.0.0
$ pip install nose          # Successfully installed nose-1.3.7
$ pip install pep8          # Successfully installed pep8-1.7.0
$ pip install jupyter       # Successfully installed MarkupSafe-1.0 appnope-0.1.0 backports-abc-0.5 bleach-2.0.0 decorator-4.0.11 entrypoints-0.2.2 html5lib-0.999999999 ipykernel-4.6.1 ipython-6.0.0 ipython-genutils-0.2.0 ipywidgets-6.0.0 jedi-0.10.2 jinja2-2.9.6 jsonschema-2.6.0 jupyter-1.0.0 jupyter-client-5.0.1 jupyter-console-5.1.0 jupyter-core-4.3.0 mistune-0.7.4 nbconvert-5.1.1 nbformat-4.3.0 notebook-5.0.0 pandocfilters-1.4.1 pexpect-4.2.1 pickleshare-0.7.4 prompt-toolkit-1.0.14 ptyprocess-0.5.1 pygments-2.2.0 pyzmq-16.0.2 qtconsole-4.3.0 simplegeneric-0.8.1 terminado-0.6 testpath-0.3 tornado-4.5.1 traitlets-4.3.2 typing-3.6.1 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-2.0.0

The above seemed to meet all the Cartopy dependancy requirements listed at: http://scitools.org.uk/cartopy/docs/v0.15/installing.html

I then installed Cartopy, ensuring to build against (if that's the correct term) the geos library installed with fink:

pip install --global-option=build_ext --global-option="-I/sw/opt/libgeos3.5.0/include" --global-option="-L/sw/opt/libgeos3.5.0/lib"  cartopy
                             # Successfully installed cartopy-0.14.2

I could run Python either in a Jupyter notebook or in the Terminal and it allowed me to import Cartopy with no errors. I downloaded some example code from the Cartopy website to test the installation.

The following example worked perfectly:

import matplotlib
matplotlib.use("TkAgg")
cartopy.crs as ccrs
import matplotlib.pyplot as plt
ax = plt.axes(projection=ccrs.Mollweide())
ax.stock_img()
plt.show()

As did this code:

import os
import matplotlib
matplotlib.use("TkAgg")
import matplotlib.pyplot as plt

from cartopy import config
import cartopy.crs as ccrs

fig = plt.figure(figsize=(8, 12))

# get the path of the file. It can be found in the repo data directory.
fname = os.path.join(config["repo_data_dir"],
                     'raster', 'sample', 'Miriam.A2012270.2050.2km.jpg'
                     )
img_extent = (-120.67660000000001, -106.32104523100001, 13.2301484511245, 30.766899999999502)
img = plt.imread(fname)

ax = plt.axes(projection=ccrs.PlateCarree())
plt.title('Hurricane Miriam from the Aqua/MODIS satellite\n'
          '2012 09/26/2012 20:50 UTC')

# set a margin around the data
ax.set_xmargin(0.05)
ax.set_ymargin(0.10)

# add the image. Because this image was a tif, the "origin" of the image is in the
# upper left corner
ax.imshow(img, origin='upper', extent=img_extent, transform=ccrs.PlateCarree())
ax.coastlines(resolution='50m', color='black', linewidth=1)

# mark a known place to help us geo-locate ourselves
ax.plot(-117.1625, 32.715, 'bo', markersize=7, transform=ccrs.Geodetic())
ax.text(-117, 33, 'San Diego', transform=ccrs.Geodetic())

plt.show()

But this code caused the kernel to crash:

import matplotlib
matplotlib.use("TkAgg")
import matplotlib.pyplot as plt
import cartopy

ax = plt.axes(projection=cartopy.crs.PlateCarree())

ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.add_feature(cartopy.feature.LAKES, alpha=0.5)
ax.add_feature(cartopy.feature.RIVERS)

ax.set_extent([-20, 60, -40, 40])

plt.show()

When the code was entered line-by-line in the Terminal, all the lines were fine until either of the last two were entered.

The only error message produced at the command line was:

Segmentation fault: 11

Has anyone comes across a reason and a solution for this problem?

Answer 1

Finally managed to make some progress so I'll summarise my solution. It may not resolve all issues but it did fix the problem I was having initially.

I posted an issue on the Cartopy GitHub page (https://github.com/SciTools/cartopy/issues/879) where QuLogic suggested a solution to stop the segmentation fault by reinstalling shapely using:

    pip uninstall shapely; pip install --no-binary :all: shapely

This indeed stopped the segmentation fault 11 but running the 'problem' code then produced an error suggesting that the geos_c couldn't be found, even though it was present. The exact error was:

OSError: Could not find lib geos_c or load any of its variants ['/Library/Frameworks/GEOS.framework/Versions/Current/GEOS', '/opt/local/lib/libgeos_c.dylib'].

It seems that the code insisted in looked for this library in a predefined location and refused to look at the locations where fink had installed the library even though I had added to location to my .bash_profile file. The solution was to create a symbolic link in that predefined location that pointed to the fink installed library. Hope that makes sense. (See Jace Browning at OSError geos_c could not be found when Installing Shapely).

So here's a summary of my entire solution, in case it helps someone else. Any suggestions for improvements would be welcomed.

1. For the record, my setup is a standard (not admin) account on Mac OS 10.11.6 (El Capitan) running on an iMac. I do, however, also have access to an admin account if necessary.

2. Installed version Python 3.6 using the installer provided at python.org

3. As admin, used fink to install gdal2, gdal2-dev, libproj9, libgeos3.6.1. (Also used fink to install a version of python3.6, gdal-py36, freetype, freetype219, cairo, gsl, sqlite3 and libspatialite7 but not sure if these packages are absolutely necessary.)

4. Python 3.6 was installed in /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6. Created a virtual enviroment (called venv36) using -m venv as follows:

At command-line:

    $ mkdir <name_of_directory_for_virtual_env>
    $ cd <name_of_directory_for_virtual_env>
    $ /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -m venv venv36

5. In user account, used nano to edit the .bash_profile file to include paths to locations where fink installed libgeos3.6.1:

At command-line:

    $ cd
    $ nano .bash_profile

Added the following lines to the .bash_profile file and saved (ctrl-O):

    GEOS_CONFIG="/sw/opt/libgeos3.6.1/bin/geos-config"; export GEOS_CONFIG
    GEOS_DIR="/sw/opt/libgeos3.6.1"; export GEOS_DIR

6. Activated the virtual environment and pip installed required packages. The pandas and jupyter packages are optional but why wouldn't you want to install them?

At the command-line:

    $ cd <path_to_virtual_environment>
    $ source venv36/bin/activate
    
    (venv36) $ pip install cython
    (venv36) $ pip install numpy
    (venv36) $ pip install --no-binary :all: shapely
    (venv36) $ pip install pyshp
    (venv36) $ pip install pyproj
    (venv36) $ pip install six
    (venv36) $ pip install matplotlib
    
    (venv36) $ export CPLUS_INCLUDE_PATH=/sw/include/gdal2/
    (venv36) $ export C_INCLUDE_PATH=/sw/include/gdal2/
    (venv36) $ pip install gdal
    (venv36) $ pip install pillow
    (venv36) $ pip install pyepsg
    (venv36) $ pip install scipy
    (venv36) $ pip install OWSLib
    (venv36) $ pip install mock nose pep8
    (venv36) $ pip install pandas
    (venv36) $ pip install jupyter
    
    (venv36) $ pip install --global-option=build_ext --global-option="-I/sw/opt/libgeos3.6.1/include" --global-option="-L/sw/opt/libgeos3.6.1/lib"  cartopy

7. Finally, added a symbolic link in /opt/local/lib/ (which is where cartopy – or some other package – insists on looking for libgeos) that points to the libgeos library installed by fink (called libgeos_c.1.dylib). It may be necessary to create the /opt/local/lib pathway (or bits of it) if it doesn't already exist.

Then, at the command-line:

    $ cd /opt/local/lib
    $ sudo ln -s /sw/opt/libgeos3.6.1/lib/libgeos_c.1.dylib libgeos_c.dylib

That's it. In the activated virtual environment, open jupyter-notebook. Make sure the first line contains the following if you want the maps to be drawn in the notebook:

%matplotlib inline

Then add the following to the next cell:

import cartopy
import matplotlib.pyplot as plt

ax = plt.axes(projection=cartopy.crs.PlateCarree())

ax.add_feature(cartopy.feature.LAND)
ax.add_feature(cartopy.feature.OCEAN)
ax.add_feature(cartopy.feature.COASTLINE)
ax.add_feature(cartopy.feature.BORDERS, linestyle=':')
ax.add_feature(cartopy.feature.LAKES, alpha=0.5)
ax.add_feature(cartopy.feature.RIVERS)

ax.set_extent([-20, 60, -40, 40])

plt.show()

When the code is run, it will likely produce a warning (Failed CDLL(/Library/Frameworks/GEOS.framework/Versions/Current/GEOS)) but it should, hopefully, still produce the following image:

So, that's it. Hope it helps. Any comments or suggestions for improvements would be appreciated.

Answer 2

I will write down my experience with this issue here, hoping it helps somebody.

How I managed to install Cartopy on MacOS:

I have MacOS 10.12.6, using a virtualenv with Python 3.7.0 (so I am not using conda).

1. Following the requirements, install required Python libraries if you don't have them yet:

   pip install numpy  
   pip install Cython  
   pip install --no-binary :all: shapely  
   pip install pyshp  
   pip install six  
   

   Obtaining the following versions in my case:

   numpy==1.15.3
   Cython==0.29
   Shapely==1.6.4.post2
   pyshp==2.0.1
   six==1.11.0
   

2. Also, I brew installed:

   brew install proj  
   brew install geos
   

   Getting versions:

   proj --> Rel. 5.2.0, September 15th, 2018
   geos-config --version --> 3.7.0
   

   Also add export GEOS_DIR=/usr/local/Cellar/geos/3.7.0/ to my ~/.bash_profile.

3. Finally, pip install Cartopy:

   pip install Cartopy
   Cartopy==0.17.0
   

So in principle, brew installing geos should be enough (I read in several places about the Kyng Chaos website, which should also be fine). Apparently, the stopper for me was

pip install --no-binary :all: shapely

What is the explanation for this?

Checking Shapely's installation instructions:

If you want to build Shapely from source for compatibility with other modules that depend on GEOS (such as cartopy or osgeo.ogr) or want to use a different version of GEOS than the one included in the project wheels you should first install the GEOS library, Cython, and Numpy on your system (using apt, yum, brew, or other means) and then direct pip to ignore the binary wheels.

Conclusion: I believe this should be included somewhere in Cartopy's installation instructions for Mac users.

---

As user1718097 mentions above, there is an issue in Cartopy's GitHub repo with this information.