Beeline not working with subprocess.run in python, getting stuck forever

955 views Asked by At

I am trying to connect thru beeline from Python script and run hql script. I am unable to run with subprocess.run or subprocess.popen. i am able to run it directly on command prompt

beeline_connect=str('"jdbc:hive2://192.168.0.100:10000/serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;transportMode=http;httpPath=cliservice;principal=hive/[email protected]"')
        
#passing params as a list
 
         param_list= ['beeline -u' + beeline_connection,'-hivevar', 'VAR1=val1', '-hivevar', 'VAR2=val2', '-hivevar', 'VAR3=val3', '-hivevar', 'VAR4=val4', '-hivevar', 'VAR5=val5', '-f', '/my/path/filename.hql']

command = subprocess.run(param_list, check=True, stdout=subprocess.PIPE)
                output = command.stdout
                status = str(output.decode('utf-8'))
                print(
                    '****Its failed****, return code {1} with return count {2} :'.format(command.returncode, status))

                if command.returncode > 0:
                    print('Job failed. Raising exception')
                    raise Exception('Job ' + job_name + ' failed')

            else:
               ## do something else###

#####i have also tried with shell=True and False both, and removing stdout

Its getting stuck forever, is there anything wrong with the way i am contacting my beeline command ? i have tried many combinations of below concatenation but no luck. any help would be much appreciable. thanks

param_list =['beeline', '-u' + beeline_connection...]
param_list =['beeline', '-u', beeline_connection....]
 
1

There are 1 answers

0
user3495160 On BEST ANSWER

Changing the line of code as below has worked for me

param_list= ['beeline', '-u',beeline_connection,'-hivevar', 'VAR1=val1', '-hivevar', 'VAR2=val2', '-hivevar', 'VAR3=val3', '-hivevar', 'VAR4=val4', '-hivevar', 'VAR5=val5', '-f', '/my/path/filename.hql']