Can I save the compressed model for direct inference only?

[Fangkai Jiao]

Excellent work

May I know if I could save the compressed model locally for further inference, e.g., combined with lora adapters?

I see the NotImplementedError raised so I'm not sure if there is something should be noted.

Thanks!

[Egiazarian Vage]

Hey!

Thanks for the interest in our work!

We've released only the evaluation code for now (you can use it to evaluate the compressed model's quality).

We'll add code for efficient inference soon (in ~2 weeks).

[Fangkai Jiao]

Thanks for your clarification!

[tower]

Hey!

Thanks for the interest in our work!

We've released only the evaluation code for now (you can use it to evaluate the compressed model's quality).

We'll add code for efficient inference soon (in ~2 weeks).

has the inferance code released by now? failed to find it!