Authentication with passport.js shows different behavior depending on browser

Question

I have this very strange bug where passport.js and passport-local-mongoose only work with Firefox. I have tried with Chrome, Internet Explorer and Edge, but I can't seem to login.

Here's the main code for login.

Routes

const passport = require('passport')
const express = require('express');
const router = express.Router({ mergeParams: true });

router.post('/login', passport.authenticate('local', {
  failureRedirect: '/login/fail',
  successRedirect: '/login/success'
});

router.get('/login/success', (req, res) => {
  req.session.userID = req.user._id;
  res.status(200).json(req.user);
});

UserProfile Schema

const mongoose = require('mongoose');
const passportLocalMongoose = require('passport-local-mongoose');

const userProfileSchema = new mongoose.Schema({
  email: { type: String, required: true, unique: true },
  // other fields
});

userProfileSchema.plugin(passportLocalMongoose, {
  usernameField: 'email',
  usernameLowerCase: true
});

const UserProfile = mongoose.model('UserProfile', userProfileSchema);
module.exports = UserProfile;

app.js

const express = require('express');
const cookieParser = require('cookie-parser');

// Setup authentication
const passport = require('passport');
const UserProfile = require('@userProfile/model/userProfile');
passport.use(UserProfile.createStrategy());
passport.serializeUser(UserProfile.serializeUser());
passport.deserializeUser(UserProfile.deserializeUser());

// Setup session handling
const oneWeek = 1000 * 60 * 60 * 24 * 7;
const expressSession = require('express-session')({
  secret: keys.session.secret,
  resave: false,
  saveUninitialized: false,
  cookie: { maxAge: oneWeek }
});

var app = express();
app.use(expressSession)
app.use(cookieParser());
app.use(passport.initialize());
app.use(passport.session());

The passport docs say:

By default, when authentication succeeds, the req.user property is set to the authenticated user, a login session is established, and the next function in the stack is called.

When logging in with Firefox, this works. The req.user is correctly set and I can use it for further processing.

When logging in with any other Browser, it doesn't work. The authentication is successful, but req.user is not set and an error occurs in subsequent handling. Interestingly, I have just found out that this only happens on localhost. When using the IP address to access the frontend, everything works as expected.

Frontend Axios setup

import axios from 'axios';

const api = axios.create({
  withCredentials: true,
  baseURL: 'http://123.456.789.10:3000',
  timeout: 5000,
  validateStatus: () => true // to suppress axios errors
});

Backend CORS settings

app.use(cors({
  origin: [
    'http://localhost:9000',
    'http://123.456.789.10:9000'
  ],
  methods: ['GET', 'PATCH', 'POST', 'DELETE'],
  credentials: true
}));

When I change baseUrl to http://localhost:3000, the behavior is exactly reversed. I can still successfully login with Firefox on both localhost and the IP, while I can only login with the other browsers via localhost now.

I don't understand it. How can using the frontend with localhost or IP both work with either baseURL on Firefox, but only one works on all the other browsers, even though nothing else has changed?