Changes in base url in vite is causing trouble rendering Remixjs APP. Getting 404 for multiple files while loading

Question

I am using Vite as the build system and remix js for building an application. I added a base URL to my vite config and the app started breaking.

Vite Cionfig

import { unstable_vitePlugin as remix } from '@remix-run/dev';
import { installGlobals } from '@remix-run/node';
import envOnly from 'vite-env-only';

installGlobals();

// https://vitejs.dev/config/
export default defineConfig({
  base: '/app',
  server: {
    port: 3001,
    host: '0.0.0.0',
    strictPort: true,
  },
  ssr: {
    external: ['xxxxxxxxx'],
  },
  plugins: [
    remix({
      appDirectory: './src',
      serverModuleFormat: 'esm',
    }),
    envOnly(),
  ],
});

The errors that I see on Console are as follows:-

I have no clue whats happening. I feel changes in the base url is causing the app to malfunction.