Matillion Calling a Python Script file on Network

Question

Is it possible for an "orchestrated job" to call a python file code on a network where it is all ready working?

I have this line that is not working in Matillion:

call(["python", "r'//caid-fs02/Python Work/data.py"])

I get this error:

Traceback (most recent call last):
  File "/tmp/interpreter-input-8418eaf0-9ef7-4d99-a0fd-7cb519c67759.tmp", line 19, in <module>
    call(["python", "r'//caid-fs02/Python Work/data.py"])
NameError: name 'call' is not defined
Script failed with status: 1

Answer 1

Assuming that the network path is on the Matillion VM, in the python script component in your orchestration job, you can use:

import subprocess
subprocess.run(["python", "r'//caid-fs02/Python Work/data.py"])

Otherwise, you would need to use the python paramiko library to upen an SSL connection to the server where the script resides:

import paramiko
client = paramiko.SSHClient()
client.connect(hostname=hostname, username=username, password=password)
client.exec_command("python", "r'//caid-fs02/Python Work/data.py")

Answer 2

I'm not clear on the question. Based on the assumptions you wanted to call a python script from Matillion Orchestration which is available in backend Matillion VM. To call a python script from Bash component. You will have to install python on the VM where Matillion is hosted if you're using dedicated VM for Matillion. Then deploy your python script into any common path for scripts and call the script from Bash component as below

python <path_to_script>/data.py "arg if any"

More details can be found here - https://metlcommunity.matillion.com/s/question/0D54G00008ZwIUgSAN/how-to-execute-a-python-script