Self and mat2 must have the same type

[Teli Ma]

Hi,

When I ran the forward function in

LLaMA-Adapter/llama_adapter_v2_multimodal7b/llama/llama_adapter.py

Line 151 in 95b6389

 def forward(self, tokens, labels, imgs): 

I met the following error

I printed out the dtype of the input and self.weight, self.bias

, it showed the same dtype.

[Teli Ma]

I solved the problem with the help of @csuhan .
with torch.cuda.amp.autocast():
is also needed in the inference process if forward function is called, like in

LLaMA-Adapter/llama_adapter_v2_multimodal7b/engine_pretrain.py

Line 37 in 5e8c8b6

 with torch.cuda.amp.autocast(): 

I will close the issue.