RuntimeError: expand(torch.HalfTensor{[1025, 4096]}, size=[1]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)

[一枚嘉应子]

RuntimeError: expand(torch.HalfTensor{[1025, 4096]}, size=[1]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)

train_utils 45
.....["transformer.position_embeddings.weight"] = extend_embedding_weights()...
train_utils 38
new_weights[:original_length] = state_weights......

Has anyone encountered this problem?

[一枚嘉应子]

I print the model.state_dict()["transformer.position_embeddings.weight"].data AND model.state_dict()["transformer.block_position_embeddings.weight"].data

tensor([1.], devices='cuda:0', dtype=torch.float16)

Is it correct?

[一枚嘉应子]

problem fixed