Evaluate a quantize model

[Javier]

I am trying to run an evaluation in a model that is quantized. I have to instantiate it using 'accelerate' to use the GPU, because otherwise it cannot fit in memory. The problem is that when using compute() in the metric I want, I get this error:

ValueError: The model has been loaded with accelerateand therefore cannot be moved to a specific device. Please discard thedevice argument when creating your pipeline object.

Is there any way to allow using the compute() method in a model that is in GPU?

[Pedro Vitor Quinta de Castro]

I'm also having the same problem... @Psancs05 were you able to find a workaround for this?

[shuvam mandal]

yeah me too what's the solution for it