removeObserver doesn't remove an observer attached to an AVPlayer.currentItem object

Question

Essentially I have a scrolling list of videos (think TikTok). I want each video to play while it's on the screen. The functionality is all there, but when you swipe to the next video, I want to remove the observer attached to the last video that caused it to loop. For some reason, calling .removeObserver with the previous AVPlayer's currentItem does not remove the AVPlayer from the NotificationCenter (I know this because I still see the old observer in the debugDescription).

@State var players: [AVPlayer] = [myAVPlayers]
@State var selectedIndex: Int = 0

VStack {
    // ForEach players makes a PlayerView that takes in an AVPlayer for the specific video
}.gesture(
    DragGesture()
        .onEnded { _ in

            // here, I know I need to somehow remove the observer for the previous video
            // I've tried NotificationCenter.default.removeObserver(self, name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: players[i].currentItem) 
            // Also tried storing the players[selectedIndex].currentItem object in a @State var right before the .addObserver() is called and then passing that into the object parameter for both the add and remove functions.

            // selectedIndex is updated to new value

            NotificationCenter.default.addObserver(forName: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: players[selectedIndex].currentItem, queue: .main) { (_) in
                print(NotificationCenter.default.debugDescription)
                let playerItem = players[selectedIndex]
                playerItem.seek(to: .zero)
                playerItem.play()
            }
        }
)

Any help is much appreciated!