In the SnapMoGen/tree/main/model/transformer/transformer.py
126 self.lvl_embed = nn.Embedding(len(self.patch_sizes), self.latent_dim)
127 nn.init.trunc_normal_(self.lvl_embed.weight.data, mean=0, std=init_std)
143 self.apply(self.__init_weights)
I found that self.lvl_embed get reset by self.apply(self._init_weights) again.
I am not why the nn.init.trunc_normal should be implemented before _iniy_weights.
In my understanding, the weight initialization would be overwritten in line 143.
I doubt it might a small setup mistake.
In the SnapMoGen/tree/main/model/transformer/transformer.py
126 self.lvl_embed = nn.Embedding(len(self.patch_sizes), self.latent_dim)
127 nn.init.trunc_normal_(self.lvl_embed.weight.data, mean=0, std=init_std)
143 self.apply(self.__init_weights)
I found that self.lvl_embed get reset by self.apply(self._init_weights) again.
I am not why the nn.init.trunc_normal should be implemented before _iniy_weights.
In my understanding, the weight initialization would be overwritten in line 143.
I doubt it might a small setup mistake.