TripleO Openstack train introspection timed out

Question

i have a question why when i run "openstack overcloud node introspect --all-manageable" is always failed,

here is my instackenv.json :

(undercloud) [stack@tripleo ~]$ cat instackenv.json
{
        "nodes":[
        {
            "capabilities": "profile:compute,boot_option:local",
            "name": "tripleo.compute1",
            "pm_type": "pxe_ilo",
            "pm_user": "admin",
            "pm_password": "password",
            "pm_addr": "10.14.32.9",
            "mac":[
                 "98:F2:B3:21:DA:46"
                 ]
        },
        {
            "capabilities": "profile:control,boot_option:local",
            "name": "tripleo.controller1",
            "pm_type": "pxe_ilo",
            "pm_user": "admin",
            "pm_password": "password",
            "pm_addr": "10.14.32.8",
            "mac":[
                 "98:F2:B3:21:FA:10"
                 ]

        }
    ]
}

here is my ironic-inspector.log :

2020-10-16 10:37:33.775 7 ERROR ironic_inspector.node_cache [-] Introspection for nodes [u'14b4b5f0-71ec-4107-9f30-236abb705808', u'fa623d6d-0241-4b3a-a04b-079f1e7ed847'] has timed out
2020-10-16 10:37:33.777 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Attempting to acquire lock acquire_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:105
2020-10-16 10:37:33.778 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Successfully acquired lock acquire_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:107
2020-10-16 10:37:33.778 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Successfully released lock release_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:119
2020-10-16 10:37:33.778 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Executing fsm(waiting).process_event(timeout) fsm_event /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:207
2020-10-16 10:37:33.778 7 INFO ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Updating node state: waiting --> error
2020-10-16 10:37:33.778 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state waiting] Committing fields: {'state': 'error'} _commit /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:150
2020-10-16 10:37:33.789 7 DEBUG ironic_inspector.node_cache [-] [node: 14b4b5f0-71ec-4107-9f30-236abb705808 state error] Committing fields: {'finished_at': datetime.datetime(2020, 10, 16, 3, 37, 33, 778442), 'error': 'Introspection timeout'} _commit /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:150
2020-10-16 10:37:33.800 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Attempting to acquire lock acquire_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:105
2020-10-16 10:37:33.800 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Successfully acquired lock acquire_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:107
2020-10-16 10:37:33.801 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Successfully released lock release_lock /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:119
2020-10-16 10:37:33.801 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Executing fsm(waiting).process_event(timeout) fsm_event /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:207
2020-10-16 10:37:33.801 7 INFO ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Updating node state: waiting --> error
2020-10-16 10:37:33.801 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state waiting] Committing fields: {'state': 'error'} _commit /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:150
2020-10-16 10:37:33.807 7 DEBUG ironic_inspector.node_cache [-] [node: fa623d6d-0241-4b3a-a04b-079f1e7ed847 state error] Committing fields: {'finished_at': datetime.datetime(2020, 10, 16, 3, 37, 33, 801202), 'error': 'Introspection timeout'} _commit /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:150

here is my journalctl :

(undercloud) [stack@tripleo ~]$ sudo journalctl -u openstack-ironic-inspector-dnsmasq
-- No entries --
(undercloud) [stack@tripleo ~]$ sudo journalctl -u openstack-ironic-inspector
-- No entries --
(undercloud) [stack@tripleo ~]$

here is my controller not getting dhcp and my result in undercloud.

thanks

Answer 1

and there is some dhcp request :

    [stack@tripleo ~]$ sudo tcpdump -i any port 67 or port 68 or port 69
        tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
        listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
        listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 
        09:50:57.118995 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:50:57.118995 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:50:57.118995 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:05.193145 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:05.193153 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:05.193153 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:05.602879 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:05.602879 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:05.603323 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:09.667411 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:09.667412 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:09.667411 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:17.741535 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:17.741536 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:17.741535 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:42:a5 (oui Unknown), length 548
    09:51:21.286506 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:21.286506 ethertype IPv4, IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    09:51:21.286871 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 98:f2:b3:2c:02:f5 (oui Unknown), length 548
    10:15:53.427748 IP tripleo.localdomain.52678 > tripleo.localdomain.tftp: 47 RRQ "map-file" octet tsize 0 blksize 512 timeout 3

and here is my undercloud.conf :

[stack@tripleo ~]$ sudo cat undercloud.conf | grep -v "^#" | grep -v "^$"       [DEFAULT]
certificate_generation_ca = local
generate_service_certificate = true
local_interface = eno2
local_ip = 192.168.35.1/24
undercloud_admin_host = 192.168.35.2
undercloud_enable_selinux = true
undercloud_hostname = tripleo.undercloud
undercloud_public_host = 192.168.35.3
[ctlplane-subnet]
cidr = 192.168.35.0/24
dhcp_end = 192.168.35.20
dhcp_start = 192.168.35.5
gateway = 192.168.35.1
inspection_iprange = 192.168.35.25,192.168.45.80
[stack@tripleo ~]$

just info if I deploy a tripleo baremetal with 1 undercloud, 1 controller, and 1 compute.and for the mac address controller is 98: f2: b3: 2c: 02: f5 and compute is 98: f2: b3: 2c: 42: a5

I've tried to turn off firewall and selinux but it had no effect and error was still same