Ansible: How to use multi-value data list as inventory and pass to target host

Question

The source of my host inventory is from an internal tool that outputs pairs of values, example, here are six observations, I currently have 160 observations:

   servername1  processname1
   servername1  processname2
   servername1  processname3
   servername2  processname1
   servername3  processname1
   servername4  processname1

So column 1 is my target host list (my inventory). Column2 are unique processname values, assigned specifically to the value of the servername. Often the same server will occur. Some servers have only one processname, others may have 2 to N. Meaning my target host may repeat for a unique list of processnames. I want to use both dynamic inventory from this output list of pairs, and I need both values on each observation to be associated and assigned to variables. I'm not absolutely required to use dynamic inventory, I just need a solution. I also need to pass to the target host and the value in {{ processname# }}, via the command: or shell: modules. (This is unique, there are no modules related to this need)

If required, I have a way to filter this data and output it in JSON format or YAML, making a separate YML file for each host. While I'd prefer to process these dynamically; pre-processing the list is acceptable.

Because ansible-playbook, requires some known host inventory list, I'm getting stuck understanding how I can create this list, from my dynamic output, at the time I start the play.

What I've done so far: I've tried reading up and trying to set these pairs as in the /etc/ansible/hosts/host_vars/servername#.yml files. This is extremely ugly, as I have to pre-process the output of the data, into YML format. But it does not give me a host list to reference in my playbook. So while it seems that hostvar is the logical choice, I cannot get my head around it.

What I need:

- The suggested format of the data?  JSON? YAML? Other? (if I cannot read it in dynamically.
- Is putting this in host_vars correct?
- Last night I saw another answer using set_fact, would that help?

Thank you for any insight. I've now been using Ansible for 3.5 weeks! I've done pretty good using static and dynamic inventories, but this stumps me as the inventory list is not obvious, give the format of the matched pairs.

Note: MANY have suggested using host_vars, but that seems to me, to be reserved to hostnames, and related port and proxy values. I could be wrong.

===================================================================

UPDATE: Thanks for the help in the right direction. I have updated our inventory script to output host list in JSON. The first new option is to output the hosts in JSON. Example: {"my_host":["servername1","servername2",]}

Calling this as a dynamic inventory script, works great!

ansible all -m ping servername1 | SUCCESS => { "changed": false, "ping": "pong" } servername2 | SUCCESS => { "changed": false, "ping": "pong" }

Next: The second new option to the inventory script was to add a new switch to input a hostname. This part is still confusing me. Here is the output: showInv --host=servername1

{"servername1":["processname1","processname2","processname3",]}

The final part that I am missing is how I call the inventory script with a specific "--host={{ my_host }} , from within my playbook.

It seems that I need to find the variable for the existing hostname and pass that back to the inventory script as the switch option "--host= "

Answer 1

You say that you are OK with dynamic inventories. Make your own.

Here is the docs.

You need to make a script that will do two things:

• when executed with --list, processes your file and prints this JSON to stdout:

  { "myhosts": ["servername1", "servername2", "servername3"] }
  

• when executed with --host servername1, prints this JSON to stdout:

  { "myprocesses": ["processname1", "processname2"] }
  

So with --list you should provide uniq list of hosts. In my example they belong to myhosts group.

And with --host <hostname> you should provide list dict of host vars for that host (<hostname>). In my example there is a list variable myprocesses that contains all processes for that host.

Then just call ansible-playbook -i my_inv_script myplaybook.yml.

Example playbook:

---
- hosts: myhosts
  tasks:
    - debug:
        msg: "Process name is {{ item }}"
      with_items: "{{ myprocesses }}"

This playbook will go trough all hosts in your dynamic inventory and print all processes for each host.

Answer 2

You will need to develop a dynamic inventory script, that takes the first column as the hostname, and the second column as variables for that host.

Please, find below the link to my dynamic inventory written in php https://github.com/walden-it/ansible-ij/blob/master/inventory.php

take a look at the functions get_vars() and get_hosts() to see how is the array being populated.

And in case you need it, here is the dump for the database this script is looking at: https://github.com/walden-it/ansible-ij/blob/master/ansible.sql

Then you just specify it with -i inventory in the ansible run, or add it as inventory_file to the ansible.cfg

Answer 3

Closing this out. With the help of Konstantin's suggestions, I now have a working play. What is not immediately apparent, is that Ansible is doing some "magic" behind the scenes. I had to modify my inventory script, that generates my dynamic inventory to accept the "--list" switch option, and the "--host hostname" option.

Once this was done, I could run the playbook with the -i listInv and Ansible internally calls this script as listInv --list, which produces my dynamic inventory list. Then it loops, through to the with_items, and internally calls the script as, listInv --host {{ items }} and outputs the matching processnames.

Additionally, the JSON output generated by my script, had to make the "group" (first) field, "myprocess". Initially, I had it as "my_process", and this failed. Removing the underscore, fixed, that error.

All working now. This is a great example for learning, but it's still magic.

The playbook looks like this:

- hosts: all
  gather_facts: no
  connection: local

  tasks:
    - debug:
        msg: "Process name is {{ item }}"
      with_items:  "{{ myprocess }}"