Skip to content

Normals use mat3(model) instead of the inverse-transpose, breaking lighting under non-uniform scale #140

@bdero

Description

@bdero

The unskinned and skinned vertex shaders transform normals by the model matrix's upper-left 3x3 directly. The correct transform for a normal is that matrix's inverse-transpose. Any node whose world transform has non-uniform (anisotropic) scale or shear shades incorrectly: the normal points the wrong way, corrupting both analytic lighting and IBL.

Where:

  • flutter_scene_unskinned.vert:26: v_normal = (mat3(frame_info.model_transform) * normal).xyz;
  • flutter_scene_skinned.vert:69: v_normal = (mat3(frame_info.model_transform) * mat3(skin_matrix) * normal).xyz;
  • Consumed at flutter_scene_standard.frag:234 (normalize(v_normal)), also fed to IBL via env_normal at :268.

Why: positions and tangents transform by M, but a normal must transform by (M^-1)^T to stay perpendicular to the surface. mat3(M) equals (M^-1)^T only for rotation, uniform scale (up to a positive scalar that normalize removes), and reflections. It diverges for non-uniform scale and shear, a direction error that survives normalization.

Not a regression, and unrelated to the winding fix in #137: rotation, uniform scale, translation, and axis mirrors (negative unit scale) all render correctly, since reflections are self-inverse-transpose.

Repro: a CuboidGeometry + PhysicallyBasedMaterial (low roughness) under a Node with scale(3, 1, 1). Its specular highlight is displaced versus the same shape modeled at 3x1x1 with an identity transform.

Proposed fix: add a normal matrix to the FrameInfo UBO, computed CPU-side as transpose(inverse(mat3(model_transform))), and use v_normal = normalize(normalMatrix * normal) in both vertex shaders. Skinned path: same treatment relative to model * skin (rigid joints are already fine). Mind std140 layout (pass a mat4) and the web shim repacking, then rebuild the shader bundle.

Priority: correctness, lower than the winding fix. Many models bake non-uniform scale into geometry or avoid it, so real-world impact is limited.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions