Diffuse normal inversed when 3D model too large (cilinder) using ASSIMP and Phong shading

Question

Currently I'm setting up some lighting in a 3D scene I created in Blender and loaded via assimp with the following options set:

aiProcess_GenSmoothNormals | aiProcess_Triangulate | aiProcess_CalcTangentSpace | aiProcess_FlipUVs

Currently I'm stuck on a really weird glitch in my program. I'm implementing Phong shading on the fragment shader for lighting with the following properties:

1. Each of the models have textures set up.
2. Each of the models have normal vectors loaded from the model (with some pre-calculations on them, probably because of the aiProcess_GenSmoothNormals flag)
3. The specular light is working on all objects.
4. Diffuse colors work as they should.

However, the pipe objects are different: the diffuse colors are always at the opposite side of the pipe that should be lit, while the specular light is on the correct side. This makes things weird, since the specular light is working as it should, while the diffuse component is always on the wrong side. I noticed this effect when scaling my cilinder objects beyond a certain point in blender (smaller scaled cilinders still work as they should) so scaling cilinder objects beyond a certain treshold probably has something to do with it.

My scene, where the pipe-like objects have working specular components but the diffuse colors are on the opposite side of the light source.

Normals as seen in blender

My first guess was that it had something to do with normal scaling, but I already used a normal matrix for that purpose in the vertex shader and the other objects in my scene work just fine.

Vertex Shader:

#version 330
layout (location = 0) in vec3 vertex;
layout(location = 1) in vec3 normal;
layout(location = 2) in vec3 tangent;
layout(location = 3) in vec3 color;
layout(location = 4) in vec2 texCoord;

uniform mat4 projection;
uniform mat4 view;
uniform mat4 model;
uniform vec3 lightPos;

out vec3 Position;
out vec3 Normal;
out vec3 LightPos;
out vec2 TexCoord;

void main()
{
    gl_Position = projection * view * model * vec4(vertex, 1.0);
    // Position
    Position = vec3(view * model * vec4(vertex, 1.0));
    // Normal
    mat3 normalMat = transpose(inverse(mat3(view * model)));
    Normal = normalMat * normal;
    // Lighting
    LightPos = vec3(view * vec4(lightPos, 1.0));
    // Texture
    TexCoord = texCoord;
}

Fragment Shader:

#version 330
in vec3 Position;
in vec3 Normal;
in vec3 LightPos;
in vec2 TexCoord;

uniform sampler2D texture0;

out vec4 outColor;

void main()
{
    // defaults
    vec4 ambient = vec4(0.2);
    vec4 diffuse = vec4(0.4);
    vec4 specular = vec4(0.5);
    vec4 texColor = texture(texture0, TexCoord);
    // Phong shading
    vec3 LightDir = normalize(LightPos - Position);
    vec3 Norm = normalize(Normal);

    vec3 ViewDir = normalize(-Position);
    vec3 ReflectDir = reflect(-LightDir,Norm);
    float specularContribution = pow(max(dot(ViewDir, ReflectDir), 0.0), 32);
    // Calculate diffuse component
    vec4 I = diffuse * max(dot(Norm, LightDir), 0.0);
    diffuse = clamp(I, 0.0, 1.0);
    // Calculate specular component
    specular = specular * specularContribution;

    outColor = texColor * (diffuse + ambient + specular);
}

---

Edit

I added a geometry shader that displays all the vertex normals in a second drawing pass for debugging purposes. However, when displaying the normals they are slightly moving with camera movement which they should not do. I am guessing this is probably the cause for my beforementioned issue.

I made a small video illustrating the normal movement: Youtube video that displays the normal movement issues.

The video shows the pink normals changing direction as the camera moves. This should not be the case and I don't know why. Is it an incorrect normal matrix or maybe assimp loads the normals incorrectly?