attach(mesh) issue using @react-three/xr interactive

Question

I'm trying to recreate this three.js attach-to-ray demo in @react-three/xr and have come up against a very strange hiccup that I can't figure out.

Using onSelectEnd to attach the mesh works just fine, but once it's attached onSelectEnd will no longer fire. I've tried using doing it "manually" with add(), tried saving the object data in a different spot in userData, but ... nada.

I'm still very new with @react-three/xr so I'm sure it's something obvious, but I'd love a hand if anyone has a moment. Here's the big giving me trouble, here's a link to the full demo.

<Interactive
  ref={groupRef}
  onSelectStart={(e) => {
    console.log("Attach! Works!");
    controlRef.current = e.target.controller;
    const intersections = getIntersections(controlRef.current);

    if (intersections.length > 0) {
      const intersection = intersections[0];
      const object = intersection.object;
      controlRef.current.attach(object); // this causes onSelectEnd to stop firing
      controlRef.current.userData.selected = object;
    }

    onSelectStart?.(e);
  }}
  onSelectEnd={(e) => {
    console.log("Detach! Please!");
    const object = controlRef.current.userData.selected;
    if (object) {
      controlRef.current.detach();
    }
    onSelectEnd?.(e);
  }}
  {...rest}
>
  {children}
</Interactive>;