What is the correct way to programmatically de-register and remove offline instances in Spring Boot Admin

Question

We use kubernetes and despite having spring.boot.admin.client.auto-deregistration=true instances often are unable/unsuccessful at deregistering before their pod is killed.

This leaves us with many phantom instances on SBA that don't seem to ever be automatically cleaned up.

I've attempted to solve this by writing a scheduled process to automate deregistering offline apps, but it also does not appear to be actually deregistering them, and they still show up as offline in the UI.

It looks like this:

@Component
class InstanceReaper(val instanceRegistry: InstanceRegistry) {
    private val log = KotlinLogging.logger {}

    @Scheduled(fixedDelay = 5, timeUnit = TimeUnit.MINUTES)
    fun clearDeadInstances() {
        // Any instance reporting down for more than 60 seconds we can remove.
        // It should simply re-register if it comes back online
        val cutoffInstant = Instant.now().minusSeconds(60)

        log.debug { "Launching reaper to kill offline instances older than $cutoffInstant" }

        var total = 0
        var removed = 0

        this.instanceRegistry.instances.subscribe {
            total++
            val statusTimestamp = it.statusTimestamp

            // Only remove offline instances, as applications can be online but be DOWN due to failing health indicators.
            if (it.isRegistered && it.statusInfo.isOffline && statusTimestamp.isBefore(cutoffInstant)) {
                log.info {
                    "De-registering ${it.registration.name} instance ${it.id} having management url " +
                        "${it.registration.managementUrl} for reporting as offline for more than 60 seconds." +
                        " Last status Timestamp: $statusTimestamp"
                }
                removed++
                it.deregister()
            }
        }

        log.info { "Reaper execution removed $removed instances out of an initial total of $total" }
    }
}

I'm sure i'm just misunderstanding something, but instances that are no longer online will continue to show in the UI and the SBA application log will have the same log message printed for the application every 5 minutes:

De-registering foo-bar-app instance c3df5fcf6dbb having management url http://10.x.x.x:8081/actuator for reporting as offline for more than 60 seconds. Last status Timestamp: 2024-02-07T16:15:19.896560955Z