pip with virtualenv requires sudo

Question

I'm on OSX and have a python virtualenv set up through virtualfish called ds-mapreduce-env. With the venv activated in my project directory it looks like the correct pip and python are chosen:

~/d/ds-mapreduce (venv) master ➜ which pip
/Users/cody/.virtualenvs/ds-mapreduce-env/bin/pip
~/d/ds-mapreduce (venv) master ➜ which python
/Users/cody/.virtualenvs/ds-mapreduce-env/bin/python

But when I run pip install it doesn't install to the virtual environment as expected. However, when I use sudo it does.

~/d/ds-mapreduce (venv) master ➜ pip install geonamescache
Requirement already satisfied (use --upgrade to upgrade): geonamescache in /usr/local/lib/python2.7/site-packages

~/d/ds-mapreduce (venv) master ➜ sudo pip install geonamescache
The directory '/Users/cody/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/cody/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting geonamescache
Installing collected packages: geonamescache
Successfully installed geonamescache-0.18

I'm guessing this is some kind of permissions issue but I can't figure it out. I installed python with brew install python and virtualfish with pip install virtualfish. I tried removing and re-installing to no effect.

My Solution

My issue was with my PYTHONPATH. I had /usr/local/lib/python2.7/site-packages prepended to my PYTHONPATH. Removing it fixed the issue!

Answer 1

The first step is to uninstall the globally-installed package:

pip uninstall -y geonamescache

Confirm that it has been fully removed by inspecting /usr/local/lib/python2.7/site-packages/. If not, manually remove any related detritus that you find there.

The next step is to better isolate your virtual environments' site-packages from your global site-packages. Add the following to ~/.config/fish/config.fish:

set -x PIP_REQUIRE_VIRTUALENV "true"

If you open a new terminal session, you should now get an error if you run pip install geonamescache without an activated virtual environment — which is what we want. Create and activate a new virtual environment via vf new ds-mapreduce-env and try the same pip install geonamescache invocation again, and the package should be installed into the virtual environment.

Since this will prevent you from installing anything into your global site-packages, you may want to set up an alias that allows you to override this isolation and perform global pip operations. Add the following function as ~/.config/fish/functions/gpip.fish:

function gpip -d "Manage globally-installed Python packages"
    env PIP_REQUIRE_VIRTUALENV="" pip $argv
end

With that in place, you can (for example) upgrade your global packages via:

gpip install --upgrade setuptools pip wheel virtualenv

Most of the above is already addressed via Tacklebox + Tackle (look for the pip plugin), which are projects designed to easily add enhancements to your fish environment.