Why is an outdated version of a Ruby script used when running Ruby from a Subprocess in Python?

Question

I'm trying to run a Ruby project from a Python interface, but in its most current form it only works when run from a terminal, not from a Subprocess in Python. The Ruby project consists of an executable file (let's call it exeFile) which runs a command-line interface tool (let's call it cli.rb), which requires and calls various structures from various other Ruby files in the project. The CLI file takes command-line arguments via Thor. This works in a terminal, but fails in a subprocess call when the cli.rb file has been modified.

I've made sure I'm passing all the right arguments to subprocess.Popen. For example:

popen = subprocess.Popen(['/home/daveshere/.rbenv/shims/ruby', '/media/daveshere/Data/exeFile', '--b', '1.1.1', '-p'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, universal_newlines=True, shell=False)

The p argument hasn't changed in the cli.rb file, and works in both the terminal and the Subprocess. But days ago, I changed the type of b from numeric to string, which allows the program to run properly. Again, this works in the terminal, but not when running from the Subprocess. Instead, printing the Subprocess output shows me the error:

Expected numeric value for '--b'; got "1.1.1"

This is despite the cli.rb file requiring a string type for b now, and the fact that the same call works in the terminal. It seems as if there is some locally-cached outdated version of the cli.rb file being called by ruby in the Subprocess, but not the terminal. I even tried adding this to the cli.rb file to see if it registers:

puts "TESTING MODIFICATION"

That string prints when running from the terminal, but not from the Subprocess (although other, older output does).

I also made sure ruby -v returns the same version in both cases, and it does. I'm really not sure what's causing the disconnect here. I'm also running the Python Subprocess via Pycharm on Ubuntu, if that has any relevance. Any ideas?

Answer 1

Fixed it thanks to engineersmnky's help. For whatever reason, the Subprocess, regardless of whether shell=True or shell=False was used, whether bash was or wasn't used, whether env=os.environ was used, etc., could not read the ruby environment, including the local changes to the gem. It could only read the git-published changes. I'm a ruby/gem newbie, so the reason wasn't obvious to me.

I didn't have ownership of the repo for the gem I was editing, so I created a private fork of the repo, moved my local changes to that repo, edited the *.gemspec file to reflect the changes, and built and installed a new gem locally with the same name. Now everything works. All I have to do now is remember to add my local changes to the private git and reinstall as needed when new changes are made. Technically, maybe I could have just added the local changes to the public repo without committing anything, but I'd prefer playing it safe with the private repo.