Apply the lightmap to the object using MaterialPropertyBlock instead of cloning materials. The lightmap texture and its per-object UV transform are set as overrides via PropertyBlock. Three.js reads material.lightMap to determine shader defines and upload uniforms, and uses texture.offset/repeat to compute lightMapTransform in the vertex shader.
Remove the lightmap from the object
This component is automatically added by the Renderer component if the object has lightmap uvs AND we have a lightmap.