SwiftUI technique to update UINavigationBarAppearance.backgroundImage dynamically?

Question

I am implementing a gradient background for the top navigation bar in SwiftUI using a standard NavigationView. For the gradient I am using a UIGraphicsImageRenderer to create a gradient image which I then assign to the backgroundImage of UINavigationBarAppearance. So far everything is behaving correctly in terms of drawing the gradient and having it render on the top bar.

However, I have not yet been able to make the backgroundImage change dynamically per view. Full code below that should work if you paste it into a playground.

import SwiftUI

struct NavBarGradient: ViewModifier {
    
    init(from: UIColor, to: UIColor) {
        let appearance = UINavigationBarAppearance()
        appearance.backgroundColor = .clear
        
        let imageRenderer = UIGraphicsImageRenderer(size: .init(width: 1, height: 1))
        let gradientImage : UIImage = imageRenderer.image { ctx in
            let gradientLayer = CAGradientLayer()
            gradientLayer.frame = imageRenderer.format.bounds
            gradientLayer.colors = [from.cgColor, to.cgColor]
            gradientLayer.locations = [0, 1]
            gradientLayer.startPoint = .init(x: 0.0, y: 0.0)
            gradientLayer.endPoint = .init(x: 0.5, y: 1.0)
            
            gradientLayer.render(in: ctx.cgContext)
        }
        appearance.backgroundImage = gradientImage
        
        UINavigationBar.appearance().standardAppearance = appearance
        UINavigationBar.appearance().compactAppearance = appearance
        UINavigationBar.appearance().scrollEdgeAppearance = appearance
    }
    
    func body(content: Content) -> some View {
        content
    }
    
}

extension View {
    func navBarGradient(from: UIColor = .systemRed, to: UIColor = .systemYellow) -> some View {
        modifier(NavBarGradient(from: from, to: to))
    }
}

struct SimpleView: View {
    var body: some View {
        Text("Gradient View").navigationTitle("Gradient Colors")
    }
}

struct ContentView: View {
    var body: some View {
        NavigationView {
            List {
                NavigationLink(
                    "Blue to Cyan",
                    destination: SimpleView()
                        .navBarGradient(from: .systemBlue, to: .systemCyan)
                )
                NavigationLink(
                    "Green to Mint",
                    destination: SimpleView()
                        .navBarGradient(from: .systemGreen, to: .systemMint)
                )
                NavigationLink(
                    "Red to Yellow",
                    destination: SimpleView()
                        .navBarGradient() // comment me out and previous modifier wins
                )
            }
                .navigationTitle("Main Menu")
                .navigationBarTitleDisplayMode(.inline)
        }
    }
}

Right now the .navBarGradient() view modifier seems to be "last one wins" which makes sense given how the view system works in SwiftUI. So what I am trying to understand is - what is the "SwiftUI way" of updating the appearance dynamically?

I have been able to achieve a gradient background in the top bar by adapting solutions like this one; it works exactly as desired. But it seems that it would be a useful thing to be able to update the top bar background image dynamically for things like server-side images.

Thanks in advance!