Netflix Conductor Python client generates an error when creating a task

Question

I tried to use the Python client code from Netflix Conductor to crate a task but I get errors no matter how I call it:

from conductor import conductor
import json

server_url = "http://localhost:8080/api"

task_def = '''[{
      "name": "verify_if_idents_are_added",
      "retryCount": 3,
      "retryLogic": "FIXED",
      "retryDelaySeconds": 10,
      "timeoutSeconds": 300,
      "timeoutPolicy": "TIME_OUT_WF",
      "responseTimeoutSeconds": 180,
      "ownerEmail": "[email protected]"
    }]'''

metadataClient = conductor.MetadataClient(server_url)
metadataClient.registerTaskDef(task_def)

ERROR: {"status":400,"message":"Validation failed, check below errors for detail.","retryable":false,"validationErrors":[{"path":"updateTaskDef.arg0.ownerEmail","message":"ownerEmail cannot be empty"}]}
...

HTTPError: 400 Client Error: Bad Request for url: http://localhost:8080/api/metadata/taskdefs

Also:

task_def_json = json.loads(task_def)
metadataClient.registerTaskDef(task_def_json)

Gets:

ERROR: {"code":"INTERNAL_ERROR","message":"INTERNAL_ERROR - Cannot deserialize instance of `com.netflix.conductor.common.metadata.tasks.TaskDef` out of START_ARRAY token\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 1, column: 1]","retryable":false,"instance":"d4f36b3da428"}
...
HTTPError: 500 Server Error: Internal Server Error for url: http://localhost:8080/api/metadata/taskdefs

How can I create a task with this Python module?

Answer 1

Because it's so simple of a HTTP requests I would recommend using requests to post directly rather than that outdated python client that probably nobody uses. Here's a DIY task definition post:

import requests

req_header={'Content-Type': 'application/json', 'Accept': 'application/json'}

task_def_endpoint='http://localhost:8080/api/metadata/taskdefs'
task_def = '''[{"name": "verify_if_idents_are_added", "retryCount": 3, "retryLogic": "FIXED", "retryDelaySeconds": 10, "timeoutSeconds": 300, "timeoutPolicy": "TIME_OUT_WF", "responseTimeoutSeconds": 180, "ownerEmail": "[email protected]"}]'''

response= requests.post(task_def_endpoint, data=task_def, headers=req_header)
print(response.status_code)

Now you can skip the extra info below and be done with it, but carry on reading if you are curious.

If you curious what is going on with the conductor package that came with Netflix python client, look at https://github.com/Netflix/conductor/blob/master/client/python/conductor/conductor.py

You'll see that both MetadataClient's registerTaskDef() and updateTaskDef() are actually calling the update or PUT endpoint, not the create (POST endpoint). Assuming you have verify_if_idents_are_added already defined on conductor, eg via

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '[{"name": "verify_if_idents_are_added", "retryCount": 3, "retryLogic": "FIXED", "retryDelaySeconds": 10, "timeoutSeconds": 300, "timeoutPolicy": "TIME_OUT_WF", "responseTimeoutSeconds": 180, "ownerEmail": "a%40a.com"}]' 'http://localhost:8080/api/metadata/taskdefs'

you'll find the following code works (notice that the task is not in a List element anymore cause that's what is what expected by the PUT endpoint):

task_def = '''{"name": "verify_if_idents_are_added", "retryCount": 3, "retryLogic": "FIXED", "retryDelaySeconds": 10, "timeoutSeconds": 300, "timeoutPolicy": "TIME_OUT_WF", "responseTimeoutSeconds": 180, "ownerEmail": "[email protected]"}'''

metadataClient.registerTaskDef(task_def)

So the registerTaskDef of this client is totally smurfed in my opinion. You can create PR to fix it or just use requests to do what you need.