Unable to access mediaStreamTracks of local and remote participants

Question

I am unable to obtain the mediaStreamTracks for local and remote participants, even after the roomState changes to connected. However, I noticed that they become available after reconnecting. Please advice how I could access the mediaStreamTracks for both local and remote participants after connecting to the room.

To reproduce the bug try the following code in Room component of this repo.

// Below function is from room component of this repo
import useRoomState from '../../hooks/useRoomState/useRoomState';
import useVideoContext from '../useVideoContext/useVideoContext';

export default function Room() {
const { room } = useVideoContext();
  const {
    room: { localParticipant },
  } = useVideoContext();
  const [participants, setParticipants] = useState(Array.from(room.participants.values()));
  const roomState = useRoomState();

  useEffect(() => {
    if(roomState === 'connected'){
          localParticipant.tracks.forEach(track => {
          if (track.kind === 'data' ) {
                      return;
            }
          console.log(track.mediaStreamTrack)  // Getting undefined!!
          }

        participants.forEach(participant => {
            participant.tracks.forEach(track => {
                  if (track.kind === 'data') {
                      return;
                   }
                   console.log(track.mediaStreamTrack)  // Getting undefined!!
            }
       }
  }, [roomState]);
}

Answer 1

Twilio developer evangelist here.

When you call tracks on a Participant object you don't actually get the Track objects. Instead you get an array of TrackPublication objects. A TrackPublication represents a track that has been published to a room, however it is not the Track object (which has the reference to the MediaStreamTrack that you're looking for) and, for remote participants, a RemoteTrack object may not yet exist until the localParticipant has subscribed to the track.

To handle this, you need to check whether the TrackPublication has the track available and if it does, use it, if it doesn't, listen for the "subscribed" event.

For example, with the remote participants:

participants.forEach(participant => {
  participant.tracks.forEach(trackPublication => {
    if (track.kind === 'data') {
      return;
    }
    if (trackPublication.track) {
      console.log(trackPublication.track.mediaStreamTrack);
    } else {
      trackPublication.on("subscribed", track => {
        console.log(track.mediaStreamTrack)
      });
    }
  }
}

Let me know if that helps.