Ansible couldn't resolve module/action 'community.general.xml'

Question

I wish to use ansible's community.general.xml module to parse an xml.

The ansible playbook host (runner) is a docker container offered by OLAM EE Environment (Oracle Linux Automation Manager) that is not with my team.

I tried to install community general inside the playbook before utilizing the xml module like below.

    - name: Install general
      shell: ansible-galaxy collection install community.general
      become_user: root
      delegate_to: localhost
      register: installgeneral
 

{
  "changed": true,
  "stdout": "Starting galaxy collection install process\nProcess install dependency map\nto see the full traceback, use -vvv",
  "stderr": "\u001b[1;35m[WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an\u001b[0m\n\u001b[1;35munexpected error when getting available versions of collection\u001b[0m\n\u001b[1;35mcommunity.general: '/api/v3/plugin/ansible/content/published/collections/index/\u001b[0m\n\u001b[1;35mcommunity/general/versions/'\u001b[0m\n\u001b[0;31mERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'\u001b[0m",
  "rc": 250,
  "cmd": "ansible-galaxy collection install community.general",
  "start": "2024-01-28 08:57:19.687034",
  "end": "2024-01-28 08:57:20.569548",
  "delta": "0:00:00.882514",
  "msg": "non-zero return code",
  "invocation": {
    "module_args": {
      "_raw_params": "ansible-galaxy collection install community.general",
      "_uses_shell": true,
      "warn": false,
      "stdin_add_newline": true,
      "strip_empty_ends": true,
      "argv": null,
      "chdir": null,
      "executable": null,
      "creates": null,
      "removes": null,
      "stdin": null
    }
  },
  "stdout_lines": [
    "Starting galaxy collection install process",
    "Process install dependency map",
    "to see the full traceback, use -vvv"
  ],
  "stderr_lines": [
    "\u001b[1;35m[WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an\u001b[0m",
    "\u001b[1;35munexpected error when getting available versions of collection\u001b[0m",
    "\u001b[1;35mcommunity.general: '/api/v3/plugin/ansible/content/published/collections/index/\u001b[0m",
    "\u001b[1;35mcommunity/general/versions/'\u001b[0m",
    "\u001b[0;31mERROR! Unexpected Exception, this is probably a bug: '/api/v3/plugin/ansible/content/published/collections/index/community/general/versions/'\u001b[0m"
  ],
  "_ansible_no_log": false,
  "_ansible_delegated_vars": {
    "ansible_host": "localhost",
    "ansible_port": null,
    "ansible_user": "olamuser",
    "ansible_connection": "local"
  }
}

As a solution i tried installing with --no-cache and it worked successfully like below but the xml module still errors:

---
- name: Get DB details from tomcat xml
  hosts: your_target_host
  environment:
    ANSIBLE_COLLECTIONS_PATHS: "/runner/requirements_collections"
  tasks:
    - name: Install general
      shell: ansible-galaxy collection install community.general --no-cache
      become_user: root
      delegate_to: localhost
      register: installgeneral
      
    - name: Get DB details from tomcat xml
      include_tasks: "{{ playbook_dir }}/extractfromxml.yml"
      vars: 
        xmlpath_rec: "{{ xmlpath_rec }}"
        DBname: "{{ DBname }}"
    
    - debug:
        msg: "INSTALLGENERAL: {{ installgeneral }}"
    

cat extractfromxml.yml

---

- name: Extract DB details from XML
  community.general.xml:
    path: "{{ xmlpath_rec }}"
    xpath: "/Server/Service/GlobalNamingResources/Resource[@name='{{ DBname }}']"
    content: text
  register: db_details      

Output:

{
  "msg": "INSTALLGENERAL: {'changed': True, 'stdout': \"Starting galaxy collection install process\\nProcess install dependency map\\nStarting collection install process\\nDownloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-general-8.2.0.tar.gz to /root/.ansible/tmp/ansible-local-125pzia2sj6/tmpgs0_jwew/community-general-8.2.0-xv57fqkm\\nInstalling 'community.general:8.2.0' to '/runner/requirements_collections/ansible_collections/community/general'\\ncommunity.general:8.2.0 was installed successfully\", 'stderr': '', 'rc': 0, 'cmd': 'ansible-galaxy collection install community.general --no-cache', 'start': '2024-01-28 09:19:41.413594', 'end': '2024-01-28 09:19:45.208997', 'delta': '0:00:03.795403', 'msg': '', 'stdout_lines': ['Starting galaxy collection install process', 'Process install dependency map', 'Starting collection install process', 'Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-general-8.2.0.tar.gz to /root/.ansible/tmp/ansible-local-125pzia2sj6/tmpgs0_jwew/community-general-8.2.0-xv57fqkm', \"Installing 'community.general:8.2.0' to '/runner/requirements_collections/ansible_collections/community/general'\", 'community.general:8.2.0 was installed successfully'], 'stderr_lines': [], 'failed': False}",
  "_ansible_verbose_always": true,
  "_ansible_no_log": false,
  "changed": false
}

TASK [Get DB details from tomcat xml] ******************************************
fatal: [remotehost]: FAILED! => {"reason": "couldn't resolve module/action 'community.general.xml'. This often indicates a misspelling, missing collection, or incorrect module path.\n\nThe error appears to be in '/runner/project/tomcat_templates/extractfromxml.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Extract DB details from XML\n  ^ here\n"}

I tried to list and check if the community.general was installed properly as below:

- name: LIST general
  shell: ansible-galaxy collection list -p /runner/requirements_collections
  become_user: root
  delegate_to: localhost
  register: listgeneral

- debug:
    msg: "LISTGENERAL: {{ listgeneral }}"

Output:

{
  "msg": "LISTGENERAL: {'changed': True, 'stdout': '\\n# /runner/requirements_collections/ansible_collections\\nCollection        Version\\n----------------- -------\\nansible.posix     1.5.4  \\ncommunity.general 8.2.0  ', 'stderr': '', 'rc': 0, 'cmd': 'ansible-galaxy collection list -p /runner/requirements_collections', 'start': '2024-01-28 09:56:18.101264', 'end': '2024-01-28 09:56:18.484268', 'delta': '0:00:00.383004', 'msg': '', 'stdout_lines': ['', '# /runner/requirements_collections/ansible_collections', 'Collection        Version', '----------------- -------', 'ansible.posix     1.5.4  ', 'community.general 8.2.0  '], 'stderr_lines': [], 'failed': False}",
  "_ansible_verbose_always": true,
  "_ansible_no_log": false,
  "changed": false
}

Can you please suggest why am i not able to use xml module despite installation?

Sample tomcat server.xml:

cat /web/bea_apps/uat/Tomcat_Home_v9.0.56/TomcatNode01/conf/server.xml

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8048" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
          <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />

    <Resource auth="Container"
        driverClassName="oracle.jdbc.OracleDriver"
        global="dsCanweb"
        name="dsCanweb"
        password="P5jYL3yhwMF_Ep8P"
        username="CAN_WEB_USER"
        url="jdbc:oracle:thin:@MYHOST36db06v.mybc.com:1521:DB01"
        type="javax.sql.DataSource"
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
        initialSize="15" maxTotal="150" maxActive="100" maxWaitMillis="10000" minIdle="15"
        accessToUnderlyingConnectionAllowed="true"
        minEvictableIdleTimeMillis="90000"
        logAbandoned="true" removeAbandoned="true" removeAbandonedTimeout="7200"
        removeAbandonedOnBorrow="true" removeAbandonedOnMaintenance="true"
        jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer;SlowQueryReport(threshold=600000,logFailed=true)"
        testWhileIdle="true"
        validationQuery="select 1 from dual"
        validationInterval="30000"
        testOnBorrow="true" />

    <Resource auth="Container"
        driverClassName="oracle.jdbc.OracleDriver"
        global="canwebds"
        name="canwebds"
        password="RMbbgwfxt2Os_2024"
        username="CANADA_WEB_USER"
        url="jdbc:oracle:thin:@MYHOST36DB06V.mybc.com:1521/oltt206"
        type="javax.sql.DataSource"
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
        initialSize="15" maxTotal="150" maxActive="100" maxWaitMillis="10000" minIdle="15"
        accessToUnderlyingConnectionAllowed="true"
        minEvictableIdleTimeMillis="90000"
        logAbandoned="true" removeAbandoned="true" removeAbandonedTimeout="7200"
        removeAbandonedOnBorrow="true" removeAbandonedOnMaintenance="true"
        jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer;SlowQueryReport(threshold=600000,logFailed=true)"
        testOnBorrow="true" />

</GlobalNamingResources>

  <Service name="Catalina">

    <Connector port="12408" address="uswl1212myb01.MYHOST35as28v.mybc.com" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="12409" />

<Connector port="12404" address="uswl1212myb01.MYHOST35as28v.mybc.com" protocol="AJP/1.3" redirectPort="12409" requiredSecret="QmtmM3hWcEVzUW5VTDNLdQo=" />

    <Engine name="Catalina" defaultHost="localhost" jvmRoute="worker8">

      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">
        <Valve className='org.apache.catalina.valves.ErrorReportValve'
                showServerInfo='false' />

        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

Answer 1

The ansible playbook host (runner) is a docker container offered by OLAM EE Environment (Oracle Linux Automation Manager) that is not with my team.

From community.general.xml docs:

You might already have this collection installed if you are using the ansible package. It is not included in ansible-core.

So basically the main issue is that your execution environment seems to be incorrect because it likely has only ansible-core installed instead of ansible.

As a solution I tried installing with --no-cache and it worked successfully like below but the xml module still errors

If your EE is a Docker container then most likely you run a fresh instance every time. And Ansible is not able to dynamically load the collections (e.g. when the playbook is already running). See Ansible source code for details.

The only correct solution I see is to properly prepare the execution environment - either install the whole ansible package or install all the required collections separately.

EDIT: I've just created a test playbook that uses a collection that I don't have installed:

---
- name: Get DB details from tomcat xml
  hosts: localhost
  environment:
    ANSIBLE_COLLECTIONS_PATHS: "runner/requirements_collections"
  tasks:
    - name: Install general
      shell: ansible-galaxy collection install servicenow.servicenow --no-cache

    - name: Try to invoke a ServiceNow task
      snow_record_find:
        username: ansible_test
        password: my_password
        instance: dev99999
        table: incident
        query:
          assignment_group: d625dccec0a8016700a222a0f7900d06
        return_fields:
          - number
          - opened_at

When I run this playbook Ansible validates the tasks and returns an error even before running any of them:

$ ansible-playbook playbook.yaml 

ERROR! couldn't resolve module/action 'snow_record_find'. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in '/Users/alexander/IdeaProjects/stackoverflow/77894304/playbook.yaml': line 10, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:


    - name: Try to invoke a ServiceNow task
      ^ here

Having these tasks in two separate plays makes no difference. Ansible (at least as of core version 2.16.2) checks for available collections before running the playbook.