I am trying to define a program to receive an HTML request to trigger a process in my server and leave it running there, as a daemon, until another HTML call tells the server to kill it. My problem is leaving the daemon running after a proper HTML response.
I am using a Flask-based python web server. I have tried python's multiprocessing and subprocess modules to no success, typically the main process would finish and kill the subprocess before it could do any good.
In my latest iteration I've tried to combine both, using a multripocessing thread as daemon to start a subprocess but in this case the server never returns a response - although the thread is daemonised, it is waiting for the subprocess to finish (which it never will) and holding the main program from returning...
I am out of ideas... help please?
This is the code:
from flask import Flask, request, abort
app = Flask(__name__)
import sys, time
def f():
import subprocess as sub
p = sub.Popen(['/path/to/file.py'])
print "process created " + str(p.pid) # prints to log
@app.route("/", methods = ['POST', 'GET'])
def home():
if request.method == "GET":
# return an HTML form with a 'Login' button
return """
<!DOCTYPE html>
<html>
<head>
</head>
<body>
Testing Login
<form action = "/" method = "post">
<input type="submit" value="Login">
</form>
</body>
</html>
"""
elif request.method == "POST":
import sys
import multiprocessing as mp
try:
m = mp.Process(name = 'sub', target = f)
m.daemon = True
m.start()
time.sleep(1) # artifically wait 1 sec for m to trigger subprocess
return "Logged in!"
except:
return "error! <br> " + str(sys.exc_info())
else:
abort(401)
I am answering my own question to close it in case anyone stumbles upon this. I am not sure how applicable this is to anyone else, as all my issues may be due to my server setup (nginx -> uwsgi -> Flask -> python 2.7)
I tried what @J.F.Sebastian mentioned in the comments;
concurrent.futures(the backported version for python 2.7) did not do the job (the process waited for the loop to finish and never returned a response to the HTTP request).celeryI did not test, as it seemed too much for the purpose.I also tried the
daemonsmodule I found, but besides needing a lot of rework to my code, it (for some reason I could not figure out) kills the parent uwsgi process.Lastly, I tried the
close_fds=Trueargument in thesubprocess.Popen, like @J.F.Sebastian suggested, and it works but when I killed the process, it becomes a zombie. I also tried the&argument to tell Linux to work on the background, it didnt change much, only the killing of the process is a bit cleaner.Hope this helps, if anyone is interested.