EricLBuehler / candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

LongRope support for Phi 3

EricLBuehler opened this issue 4 months ago · comments

Eric Buehler commented 4 months ago

Reference implementation: https://github.com/EricLBuehler/mistral.rs/blob/master/mistralrs-core/src/layers.rs#L69

Guoqing Bao commented 4 months ago

Reference implementation: https://github.com/EricLBuehler/mistral.rs/blob/master/mistralrs-core/src/layers.rs#L69

Supported, please refer to latest updates.

Eric Buehler commented 4 months ago

Thank you, that looks great. Implemented in #46.