Depth Pre-Pass using rgba-packed texture in three.js. I'm stuck

Question

My goal is to use (externally rendered) depth textures (32-bit, rgba packed) in a depth prepass, so that all subsequently rendered geomtries will be depth-tested against this custom depth buffer.

I'm using a custom shader on a full screen triangle to write the unpacked depth value into gl_fragDepth. However, geometries that render after the prepass are not occluded as expected. The depth test seems to be kind of inverted, but not really (???).

Maybe I'm missing something fundamentally here...?

I added a picture to better illustrate what is happening:

I also created a codepen. The only difference is, that instead of loading an external rgba-packed depth texture, the texture is created on the fly (in the "Pack" mode). In the "Prepass" mode, this texture is then used as described above. You can switch between the modes with the two buttons.

Here is the shader code as used on codepen in case the issue is shader related.

// shader code for rendering the rgba-packed depth texture
const VERT_PACK = `
  void main(){
    gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
  }
`

const FRAG_PACK = `
  #include <packing>
    
  void main(){
    vec4 packed = packDepthToRGBA( gl_FragCoord.z );
    gl_FragColor = packed;
  }
`

const VERT_PREPASS = `
    varying vec2 vUv;
  
  void main(){
    vUv = uv;
        gl_Position = vec4(position.xyz, 1.0);
  }
`

// shader code for rendering the depth pre-pass
const FRAG_PREPASS = `
  #include <packing>
  
  uniform sampler2D depthTex;
  uniform float cameraNear;
    uniform float cameraFar;
  varying vec2 vUv;
  
  float linear01Depth(const in float depth, const in float near, const in float far){
    float a = 1. - far/near;
        float b = far/near;
        return 1.0 / (a * depth + b);
    }
  
  void main(){
    vec4 sampleDepth = texture2D(depthTex, vUv);
    float depth = unpackRGBAToDepth(sampleDepth);
    
    float depthLinear = linear01Depth(depth, cameraNear, cameraFar);
    
    // note: enable colorWrite for material before writing to gl_FragColor
    // gl_FragColor = vec4(1., 0., 0., 1.);
    // gl_FragColor = vec4(vUv, 0., 1.);
    // gl_FragColor = sampleDepth;
    // gl_FragColor = vec4(depthLinear, depthLinear, depthLinear, 1.);
    // gl_FragColor = vec4(depth, depth, depth, 1.);
    
    gl_FragDepth = depth;
  }
`