Slow Rendering During VisionOS Image Tracking

Question

I am following this documentation on tracking images in 3D space on Vision Pro. https://developer.apple.com/documentation/visionos/tracking-images-in-3d-space

The expected result is that a 3D sphere will smoothly track a moving reference image. However, when I run the app, the 3D sphere will only update positions about twice a second or so. There's not a lot of documentation for troubleshooting. Is there some sort of render function I should be calling to have the RealityView render more quickly?

import SwiftUI
import ARKit
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    var arkitSession = ARKitSession()

    @State var entityMap : [UUID: ModelEntity] = [:]
    let imageInfo = ImageTrackingProvider(
        referenceImages: ReferenceImage.loadReferenceImages(inGroupNamed: "ref")
    )
    var material = SimpleMaterial()

    var rootEntity = Entity()

    func updateImage(_ anchor: ImageAnchor) {
        if entityMap[anchor.id] == nil {
            // Add a new entity to represent this image.
            let entity = ModelEntity(mesh: .generateSphere(radius: 0.05))
            entityMap[anchor.id] = entity
            rootEntity.addChild(entity)
        }

        if anchor.isTracked {
            entityMap[anchor.id]?.transform = Transform(matrix: anchor.originFromAnchorTransform)
        }

    }

    var body: some View {

        RealityView { content in
            // Add the initial RealityKit content
            content.add(rootEntity)

            Task {
                try await arkitSession.run([imageInfo]);
                for await update in imageInfo.anchorUpdates {
                    updateImage(update.anchor)
                }
            }
        }

    }
}

Answer 1

Per the Apple Developer Forums, ImageTrackingProvider on VisionOS currently seems to fire anchorUpdates quite slowly, leading to inability to smoothly track moving images. I filed an enhancement request via the Feedback Assistant and encourage others to do the same.

https://forums.developer.apple.com/forums/thread/747118

Answer 2

I worked with Unity on the Vision Pro and I also saw this behaviour. I could not find any official documentation from Apple about this issue, but it seems like updating the position of a tracking image on visionOS is not very smooth at the moment.

Source: I asked a Unity dev if this was a problem with Unity or VisionOS. It seem to be the latter. Hopefully they will improve this in the future.

Unity Forum Post