how to force buildout to use already installed package

Question

On a server, for disk optimization, we do not install a C compiler, and here is the problem :

I want to install 'spur' python package with buildout 2.3.1, spur is using pycrypto which requires a C compiler.

To avoid compilation, I installed on the server (Suse 11SP2) the rpm python-pycrypto (python-pycrypto-2.6-31.7.x86_64.rpm). I can check with 'pip freeze' that pycrypto is installed :

pip freeze 

lxml==2.1.2
mod-python==3.3.1
Numeric==24.2
numpy==1.3.0
pycrypto==2.6
pywbem==0.7.0
PyXML==0.8.4

Nevertheless, when I start buildout, it still tries to build pycrypto with a C compiler that does not exist.

So I added include-site-packages = true and allowed-eggs-from-site-packages = pycrypto in buildout.cfg, but they are not taken in account. I also tried to do a :

python bootstrap.py --allow-site-packages
bin/buildout

but that did not help too.

What can I do to avoid buildout compile a package that is already compiled and installed from rpm ?

Here is the buildout.cfg

[buildout]
newest = false
extensions = gp.vcsdevelop
develop-dir = src
parts = eggs tests wsgi
include-site-packages = true
allowed-eggs-from-site-packages = pycrypto

eggs =
    Django==1.4.8
    ordereddict
    ipython==0.13.2
    django-webtest
    django-grappelli<2.5.0
    django-bootstrap3-datetimepicker
    z3c.recipe.usercrontab
    rrdgraphs
    south
    achilterm
    pynag
    pyaml<14
    DjangoDevKit==1.0.3
    Sphinx
    xlwt
    xlrd
    spur

...

and the logs :

# bin/buildout
Unused options for buildout: 'allowed-eggs-from-site-packages' 'include-site-packages'.
Installing eggs.
Getting distribution for 'pycrypto!=2.4,>=2.1'.
configure: error: in `/tmp/easy_install-QFXca_/pycrypto-2.6.1':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 2254, in main

  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 380, in run

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 610, in easy_install

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 659, in install_item

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 842, in install_eggs

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1070, in build_and_install

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1056, in run_setup

  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 240, in run_setup
  File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
    self.gen.throw(type, value, traceback)
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 193, in setup_context
  File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
    self.gen.throw(type, value, traceback)
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 164, in save_modules
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 139, in resume
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 152, in save_modules
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 193, in setup_context
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 237, in run_setup
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 267, in run
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 236, in runner
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 46, in _execfile
  File "/tmp/easy_install-QFXca_/pycrypto-2.6.1/setup.py", line 456, in <module>
  File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 160, in run
  File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 146, in call_command
  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/install_lib.py", line 10, in run
  File "/usr/lib64/python2.6/distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/tmp/easy_install-QFXca_/pycrypto-2.6.1/setup.py", line 251, in run
  File "/usr/lib64/python2.6/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/tmp/easy_install-QFXca_/pycrypto-2.6.1/setup.py", line 278, in run
RuntimeError: autoconf error
An error occurred when trying to install pycrypto 2.6.1. Look above this message for any errors that were output by easy_install.
While:
  Installing eggs.
  Getting distribution for 'pycrypto!=2.4,>=2.1'.
Error: Couldn't install: pycrypto 2.6.1

Answer 1

I found a workaround :

I just created a egg-link file into develop-eggs dir :

$ cat develop-eggs/pycrypto.egg-link 
/usr/lib64/python2.6/site-packages

Answer 2

I got no evidence that zc.buildout > 1.7.1 supports include-site-packages, nor allowed-eggs-from-site-packages.

On my local machine I searched for the mentioned features in the following versions of zc.buildout:

• zc.buildout-1.4.4-py2.7.egg
• zc.buildout-1.6.3-py2.7.egg
• zc.buildout-1.7.1-py2.7.egg
• zc.buildout-2.0.0-py2.7.egg
• zc.buildout-2.0.1-py2.7.egg
• zc.buildout-2.1.0-py2.7.egg
• zc.buildout-2.2.0-py2.7.egg
• zc.buildout-2.2.1-py2.7.egg
• zc.buildout-2.2.3-py2.7.egg
• zc.buildout-2.2.4-py2.7.egg
• zc.buildout-2.2.5-py2.7.egg
• zc.buildout-2.3.0-py2.7.egg
• zc.buildout-2.3.1-py2.7.egg

Check Readme on https://pypi.python.org/pypi/zc.buildout/1.7.1. The features were introduced somewhere around zc.buildout 1.5.0. The last release, which supports those features is 1.7.1.

They made some kind of relaunch with zc.buildout 2.0.0...

Quote from the changlog, version 2.0.0:

This is a backward incompatible release of buildout that attempts to correct mistakes made in buildout 1.

Buildout no-longer tries to provide full or partial isolation from system Python installations. If you want isolation, use buildout with virtualenv, or use a clean build of Python to begin with.

In your case this means, if you want to use those features you need to use zc.buildout 1.7.1.

It's also important, that it's not possible to use both options at the same time. Use allowed-eggs-from-site-packages (only specific packages), or include-site-packages.

I assume you're using the boostrap.py version 1: http://downloads.buildout.org/1/bootstrap.py

This version does not support the ´--allow-site-packages´ options, since it was intended to use with zc.buildout 1.x, which handles this case (or at least 1.5.x).

---

Well, after this short trip in buildout history, I guess the following two points will solve your problem:

• Be sure you're using boostrap.py 2: https://bootstrap.pypa.io/bootstrap-buildout.py. Only this version supports the --allow-site-packages option.
• Make sure you are using a suitable zc.buildout version for yout boostrap.py verson.

Then python bootstrap.py --allow-site-packages works as expected.