How to use canle_lora modle with rust auxm web server

[Ancient]

Auxm code is below

let model = Llama::load(
        vb,
        &cache,
        &config,
        false,
        loraconfig,
        linearconfig,
        embedconfig,
    )
    .unwrap();

Router::new().layer(Extension(model))

the error is

(dyn EmbeddingLayerLike + 'static)` cannot be sent between threads safely
the trait `Send` is not implemented for `(dyn EmbeddingLayerLike + 'static)`
the trait `tower_layer::Layer<S>` is implemented for `Extension<T>`
required for `Arc<(dyn EmbeddingLayerLike + 'static)>` to implement `Send`
required for `Extension<LoraLLM>` to implement `tower_layer::Layer<Route>

[Ancient]

I found the problem is "trc" dependence should use "SharedTrc"

[Eric Buehler]

Yes, that is true. I will update the code shortly.

[Eric Buehler]

I have added some code which adds Send + Sync bounds to the macro generation. Could you please try it again?

[Ancient]

thx It worked