THREE.js Correctly UV Map texture onto custom buffer geometry while updating THREE.js

Question

I'm trying to update threejs versions and essentially I want to use an example I found on the internet for a very early version of threejs (r44) in a more recent version of threejs, (r143) and I'm having a problem with uv unwrapping for the 3d model to texture a bust. In order to do that, I realized that I would run into many problems, but mainly the thing that I'm stuck on at the moment is correctly placing the UVs so my 3D model will look like this:

But instead it looks like this :

Of course I've solved a bunch of other problems relating to importing the geometry and converting the face data myself (I'm using a not-so-kosher way of importing the geometry data to a buffer geometry) but I am stuck on this particular issue.

If you dare to help me--the disaster that lives within glitch awaits.

The properly mapped texture model lives at https://pricey-healthy-paranthodon.glitch.me/FACESHADER/index-testUV.html

the wildly improperly mapped texture model lives at https://pricey-healthy-paranthodon.glitch.me/FACESHADER/index-r44-copy2.html

and you can freely remix-to-edit the glitch project from https://glitch.com/edit/#!/pricey-healthy-paranthodon?path=FACESHADER%2Findex-r44-copy2.html%3A239%3A20

glitch autosaves, so once you remix you can edit and should be able to see changes live.

This is my "almost last ditch" effort to solve this problem, and I know that it's a tough problem to solve, especially with a custom geometry.

Some context:

Most of the action (and problems that I'm dealing with) live in index-r44-copy2.html starting at function createScene() on line 659 in that html file.

The geometry data is being imported using JSONLoader from an early version of threejs, and is sent to that function. I'm using bufferGeometry.fromPoints() (line 552) (window.fromPoints) to convert it to a new buffer geometry in the current version of threejs—I know this isn't a best practice, but the data for the geometry is being properly imported. From there I'm setting the faces of the geometry with .setIndex() and this appears to be working properly as well since the geometry is solid.

The real action with the UVs happens at convertGeometryToFiveBufferGeometry() (line 455) with "let uvs = []" (line 476) which is where I'm sure something wrong is happening.

If anyone has any advice or solutions on how to fix the UV issue please let me know! I'm very stuck D:

Answer 1

I made some progress in the mean time, however, I still need to detail things out. I successfully generate/mapped both vertex color and UVs.

Currently I'm suspect the setAttribute method messes with the array order. The hack for now is to fetch back the position array with geometry.attributes.position.array and iterate to generate e.g. a cylindrical UV projection in that array order. The funny/confusing part is that I still use the geometry.setAttribute to assign the UV array (just as I assign positions) or another attribute, but it works! This may not be exactly what you need, but if you can live with a cylindrical UV projection for a while then I can give you some code snippets.

In the meanwhile, this should/may be addressed to/by the developers. One other cause could be the data "bit depth" mismatch e.g. maybe need float16Array instead.

Cheers

Answer 2

As promised, some code snippets for cylindrical uv projection.

Map vertices if not available yet:

mesh.faces.forEach((face: number[]) => {
    face.forEach(vertex_index => {
        const vertex = mesh.vertices[vertex_index];
        positions.push(vertex[0]); // x
        positions.push(vertex[1]); // y
        positions.push(vertex[2]); // z
    });
});

geometry.setAttribute('position', new THREE.BufferAttribute(new Float32Array(positions), 3));
geometry = mergeVertices(geometry);
geometry.computeVertexNormals();

Calculate some boundary information. I just used the mesh:

const templateMeshes = new THREE.Mesh(geometry, meshMaterial);
let bbox = new THREE.Box3().setFromObject(templateMeshes);
let size = new THREE.Vector3(); bbox.getSize(size)
let bMin = bbox.min
let center = new THREE.Vector3(); bbox.getCenter(center)

Generate cylindrical UV projection:

let uvs = [];
const arr = geometry.attributes.position.array
for (let t = 0; t < arr.length; t+=3)
{
    const x = arr[t]
    const y = arr[t+1]
    const z = arr[t+2]
    let U = (Math.atan2(y-center.y, x-center.x) / (Math.PI)) *0.5 + 0.5
    let V = ((z - bMin.z) / size.z)
    uvs.push(U,V)
}
geometry.setAttribute('uv', new THREE.BufferAttribute(new Float32Array(uvs), 2))

But using the original array (order) doesn't work!!!???:

let uvs = [];
mesh.faces.forEach((face: number[]) => {
    face.forEach(vertex_index => {
        const vertex = mesh.vertices[vertex_index];
        const x = vertex[0]
        const y = vertex[1]
        const z = vertex[2]
        let U = (Math.atan2(y - center.y, x - center.x) / (Math.PI)) * 0.5 + 0.5
        let V = ((z - bMin.z) / size.z)
        uvs.push(U, V)
    });
});
geometry.setAttribute('uv', new THREE.BufferAttribute(new Float32Array(uvs), 2))

I still have another problem with BufferGeometry where Us are still messed up at the borders (2nd image is U vertex "color" mapping)

this is due to the fact that vertices on either side of the model "seam" will have a "flipped" UV mapping for the particular triangle. Not sure how that can be solved.

Answer 3

I found out that mergeVertices (form BufferGeometryUtils) is reason for why the BufferGeometry position array is messed up.

A dedicated normal smooth method (not found one yet) is a better solution than merging vertices. Having separate triangles already has solved my triangle crossing UV bounds.

Answer 4

Working but not an elegant solution to avoid the UV flips is to "wrap back" the U values properly. Below a snippet which should be done for every mesh face i.e. every vertex triplet.

let U = ((Math.atan2(y - center.y, x - center.x) / Math.PI) * 0.5 + 1.0) % 1;
let U2 = ((Math.atan2(y2 - center.y, x2 - center.x) / Math.PI) * 0.5 + 1.0) % 1;
let U3 = ((Math.atan2(y3 - center.y, x3 - center.x) / Math.PI) * 0.5 + 1.0) % 1;

Since the atan yields values between [-1..1] I scaled them with 0.5 and offset them with 1 so that the projected image edges start at the expected locations.

Answer 5

This is how I solved it. Please note, I haven't checked the newest library and its behaviour, though. For the code below, I used "three": "^0.138.3". For completeness, I create the BufferGeometry manually since we have a custom mesh type, your input type might differ:

    let geometry = new THREE.BufferGeometry();
    geometry.name = 'Project Geometry'; // for debugging
    let vertices: number[] = [];
    let indices: number[] = [];
    meshes.forEach((mesh: IMesh) => {
      // Safer to concat than push
      vertices = vertices.concat(mesh.vertices.flat());
      indices = indices.concat(mesh.faces.flat());
    });
    geometry.setAttribute('position', new THREE.BufferAttribute(new Float32Array(vertices), 3));
    geometry.setIndex(new THREE.BufferAttribute(new Uint16Array(indices), 1));
    geometry.computeVertexNormals();

Then use the non-optimized cylindrical texture projection method below (it was fast enough for our application). It may also contain some redundancy that I didn't test extensively so I kept it in there (will discuss below).

The basic idea was first, convert to non-indexed vertices for individual vertice control, and second, to check whether a U value within a single triangle "flips/wraps" with respect to the others, since atan2 will naturally flip around the -PI and PI limits. Due to atan2 the resulting U values within a single triangle could therefore differ/jump with about 1.0, so I used a threshold of 0.5 for "detection" and betting that there will never be a "big" triangle that would span more than half of a texture width, which is true for our assets. "Unwrapping" is done by adding 1.0 to smaller U values.

The potential "redundancy" may exist in the U value calculations i.e.

    let U = ((Math.atan2(y - center.y, x - center.x) / Math.PI) * 0.5 + 1.0) % 1;

The shift (+ 1.0) and the "wrapping" with the modulo (% 1) was necessary for us to obtain the sametexture offset (shifting the texture horizontally around the model) results in another app for consistency. So, this should works as well:

    let U = (Math.atan2(y - center.y, x - center.x) / Math.PI) * 0.5;

but I haven't put too much thought into it after the issue was solved and haven't tested the latter extensively. Recommend to test it extensively with your assets or reason thoroughly about the complete behaviour of this before actually use it in production.

NOTE: this method only had cylindrical texture projection in mind and adjustments are likely required for other projections.

  public addUv(geometry: BufferGeometry) {
    // Calculate model dimensions
    if (!geometry.boundingBox) {
      geometry.computeBoundingBox();
    }
    const center = new THREE.Vector3();
    geometry.boundingBox.getCenter(center);
    const size = new THREE.Vector3();
    geometry.boundingBox.getSize(size);
    const bMin = geometry.boundingBox.min;

    // Generate separate vertices per face to avoid UV "boader crossing" problems
    const newGeometry = geometry.toNonIndexed();
    geometry.dispose();

    // Actual UV calculation. projection method: cylindrical
    const vertexArray = newGeometry.attributes.position.array;
    const nComponents = 3;
    const nVectorsPerTriangle = 3;
    const arrayIncrement = nComponents * nVectorsPerTriangle;
    const uvs = [];
    for (let t = 0; t < vertexArray.length; t = t + arrayIncrement) {
      // NOTE: needs separate vertex "extraction" method
      const x = vertexArray[t];
      const y = vertexArray[t + 1];
      const z = vertexArray[t + 2];
      const x2 = vertexArray[t + 3];
      const y2 = vertexArray[t + 4];
      const z2 = vertexArray[t + 5];
      const x3 = vertexArray[t + 6];
      const y3 = vertexArray[t + 7];
      const z3 = vertexArray[t + 8];

      // NOTE: needs separate vertex (cylindrical) projection method including boarder check below
      // NOTE: the extra 0.5 "phase shift" is reqiured to align with the actual texturizer algo in another app of ours
      // NOTE: shift and modulo maybe redundant now. if so ignore above notes
      let U = ((Math.atan2(y - center.y, x - center.x) / Math.PI) * 0.5 + 1.0) % 1;
      let U2 = ((Math.atan2(y2 - center.y, x2 - center.x) / Math.PI) * 0.5 + 1.0) % 1;
      let U3 = ((Math.atan2(y3 - center.y, x3 - center.x) / Math.PI) * 0.5 + 1.0) % 1;

      const V2 = (z2 - bMin.z) / size.z;
      const V = (z - bMin.z) / size.z;
      const V3 = (z3 - bMin.z) / size.z;

      // Check triangle passing UV map "boarder". For understanding: draw triagle with U, U2, and U3 corners. Draw (border)line through triangle. There are only 6 possibilities when rotating and/or flipping the triangle.
      const uTh = 0.5; // U difference threshold
      if (Math.abs(U - U2) > uTh && Math.abs(U - U3) > uTh) {
        if (U > U2) {
          U2 += 1;
          U3 += 1;
        } else {
          U += 1;
        }
      }
      if (Math.abs(U2 - U) > uTh && Math.abs(U2 - U3) > uTh) {
        if (U2 > U) {
          U += 1;
          U3 += 1;
        } else {
          U2 += 1;
        }
      }
      if (Math.abs(U3 - U) > uTh && Math.abs(U3 - U2) > uTh) {
        if (U3 > U) {
          U += 1;
          U2 += 1;
        } else {
          U3 += 1;
        }
      }

      uvs.push(U, V, U2, V2, U3, V3);
    }
    const nUvComponents = 2;
    newGeometry.setAttribute('uv', new THREE.BufferAttribute(new Float32Array(uvs), nUvComponents));
    return newGeometry;
  }