vin136 / llm-infer

Benchmark and identify the best ways to speedup LLM inference.

llm-infer

Benchmark and identify the best ways to speedup LLM inference.

Resources

llm-hacker

coursera-llm

fine-tune-mistral

prompt-engineering-guide : This has mistral specific details.

Coursera prompt-engineering course

another prompt engineering course

mistral finetune blog

For structuring experimets(mlops): https://github.com/vin136/MLOPS

Fine tuning llms

https://www.youtube.com/playlist?list=PL23FjyM69j92o_j5JFH9sNlbhCx4n0ZYh

https://wandb.ai/johnowhitaker/llmemo/reports/Can-LLMs-learn-from-a-single-example---Vmlldzo1MjQ2MDYy

https://johnowhitaker.dev/

Hugging face blog/references

generation strategies chat-template prompting

End-point detection

alexa paper

basically, use an ML model to detect end-of-command.

Practical solutions:

vad - detect end point faster whisper, whisper-live

Learning

TODO:

• Implement Transformer from scratch
• Implement Llama2 from scratch,code
• Quantization
• Lora
• Dist Training
• Mistral