Updating OrbitControls rotation when camera is rotated

Question

I'm using @react-three/fiber and I'm implementing first person controls (WASD & cursor keys) with addition of OrbitControls to navigate a scene. I'm pretty much cloning PointerLockControls and I've got it working for the WASD controls as such (updating the target vector of the OrbitControls, passed as ref):

const moveForward = (distance) => {
    vec.setFromMatrixColumn(camera.matrix, 0)
    vec.crossVectors(camera.up, vec)
    camera.position.addScaledVector(vec, distance)
    orbitControls.current.target.addScaledVector(vec, distance)
}
const moveRight = (distance) => {
    vec.setFromMatrixColumn(camera.matrix, 0)
    camera.position.addScaledVector(vec, distance)
    orbitControls.current.target.addScaledVector(vec, distance)
}

However, I'm not quite sure how to go about updating the target when the camera is rotated. Here's how I'm rotating the camera and its working just fine without OrbitControls:

const euler = new THREE.Euler(0, 0, 0, 'YXZ' );
euler.setFromQuaternion(camera.quaternion)
euler.y -= 0.25 * radians;
camera.quaternion.setFromEuler(euler)

Preview here: https://codesandbox.io/s/wasd-with-orbit-9edup7

Answer 1

OK, so you can see the working version here: https://yvod70.csb.app/

The idea is quite simple, attach the camera to your player/box and then move said object instead of the camera. The camera being a child of the player will be translated and rotated relative to the player.

To do this the first step is to get a reference to the mesh:

const scene = () => {
    const mesh = useRef();

    return (
        <Canvas>
            <mesh
              ref={mesh}
            >
                <boxBufferGeometry args={[1, 1, 1]} />
                <meshBasicMaterial wireframe color={"green"} />
            </mesh>
            <Controls mesh={mesh} />
        </Canvas>
    );
}

After setting this up, we just pass the mesh ref to whatever React component we want and use it however. In our case it's to replace the camera for movement and attach the camera to the box. Which can be done like so in your Controls component:

const Controls = ({ orbitControls, mesh }) => {
    /** @type {THREE.Mesh} */
    const box = mesh.current;

    const { camera } = useThree();
    const code = useCodes();

    // ...your movement code here, remember to replace camera with box.

    // FIXME: Move the Controls component lower down the component tree so that we don't end up with these race conditions.
    if (!!box) {
      box.add(camera);
    }

    // ...animation code
}

These were the steps I took to attach the orbit controls to the player