RealityKit Systems Not Ticking Every Frame on visionOS

Question

Please see also the video demo of the problem I'm encountering: https://youtu.be/V0ZkF-tVgKE

I've noticed that the custom Systems I've been creating for my RealityKit/visionOS app do not get updated every frame as the documentation (and common sense) would suggest. Instead, they appear to tick for a time after each UI interaction and then "stall". The systems will be ticked again after some interaction with the UI or sometimes with a large enough movement of the user. My understanding was that these Systems should not be tied to UI by default so I'm a bit lost as to why this is happening.

I've reproduced this by starting from a template project and adding a very simple couple of systems.

Here is the main System, which simply rotates the pair of spheres:

import RealityKit
import RealityKitContent
import SwiftUI

public struct RotationSystem: System {

    static let query = EntityQuery(where: .has(RealityKitContent.WobblyThingComponent.self))

    public init(scene: RealityKit.Scene) {
    }

    public func update(context: SceneUpdateContext) {
        
        print("system update, deltaTime: \(context.deltaTime)")
        
        let entities = context.scene.performQuery(Self.query).map({ $0 })

        for entity in entities {
            
            let newRotation = simd_quatf(angle: Float(context.deltaTime * 0.5), axis: [0, 1, 0]) * entity.transform.rotation
                    
            entity.transform.rotation = newRotation
        }
    }
}

The component (WobblyThingComponent) is attached to a parent of the two spheres in Reality Composer Pro, and both system and component are registered on app start in the usual way.

This system runs smoothly in the simulator, but not in the preview in XCode and not on the Vision Pro itself, which is kinda the whole point.

Here is a video of the actual behaviour on the Vision Pro: https://youtu.be/V0ZkF-tVgKE

The log during this test confirms that the system is not being ticked often. You can see the very large deltaTime values, representing those long stalled moments:

system update, deltaTime: 0.2055550068616867
system update, deltaTime: 0.4999987483024597

I have not seen this problem when running the Diaroma sample project, yet when comparing side-by-side with my test projects I cannot for the life of me identify a difference which could account for this.

If anyone could tell me where I'm going wrong it would be greatly appreciated as I've been banging my head against this one for days.

Xcode: Version 15.3 (15E204a)

visionOS: 1.1 and 1.1.1

Answer 1

Interesting. I found several other SO/Apple Forum questions hitting the same issue. Apparently update(context:) is not guaranteed to be called on every frame render. I remember something similar from my SceneKit days, where optimizations could omit certain calls.

I did find this suggestion in https://forums.developer.apple.com/forums/thread/740956, but I don't understand why it would work:

    func update(context: SceneUpdateContext) {
    count += 1
    
    let entities = context.entities(matching: MySystem.query, updatingSystemWhen: .rendering)
    
    for entity in entities {
        print(entity.name)
    }
    
    print(count)
}

I'd be interested to hear what happens in your app if you reduce the update(context:() definition to just the print() call.

Note the Hello World sample code has orbits updating smoothly, but the way the updates are handled seems baroque to me.

By my reading of the doc at Implementing systems for entities in a scene, your code should work.

One possible miss: did you call RotationSystem.registerSystem() at some point?

Answer 2

I don't have Vision Pro on hand at the moment, so I can't say this with 100% certainty, but I suspect that the drop frame happens due to the use of for-in loop inside the system's update(context:) method. Try using the built-in higher order function map(_:) instead - it is at least 1.5 times faster in performance than the for-in loop (especially when you're printing deltaTime in debug mode).