Add Public Available LLM Support

[Pascal Sun]

Current we only use OpenAI models to handle it.

Even the cost is relatively low compared to do it manually(for 20,000 documents, roughly 3,000 AUD for GPT3.5, 20,000 for GPT-4)

Still there is privacy concern with the data.

The task we require LLM to do here is quite simple, so llama3 and variants can handle it quite good.

So we should add support for this

Design principle should be Treat Public available LLM as seperate service to make sure the package is as light as possible

[Aries Li]

Could you expand the GPT support to Azure OpenAI too?

[Pascal Sun]

Could you expand the GPT support to Azure OpenAI too?

I will try to get this solved in one go, as most of the Public Available LLM can run in OpenAI server way.