[Feature]: support Ollama backend

[yankay]

Checklist

• I've searched for similar issues and couldn't find anything matching
• I've discussed this feature request in the K8sGPT Slack and got positive feedback, https://k8sgpt.slack.com/archives/C050SKW7SHF/p1712890993773069

Is this feature request related to a problem?

None

Problem Description

No response

Solution Description

support Ollama like localai.

Benefits

The Ollama can make it easier for users to interact with K8SGPT.

Potential Drawbacks

No response

Additional Information

No response

[liyimeng]

@yankay I think it is already supported, see #981

basically, u do

root@testing:/# k8sgpt auth add --backend localai --model tinyllama --baseurl http://my-ollama:8080/v1
localai added to the AI backend provider list

[wilmardo]

On my Mac I did this to make it work. Installed ollama as application, then:

k8sgpt auth add --backend localai --model llama2 --baseurl http://localhost:11434/v1
k8sgpt auth default -p localai. # sets it as default, not really needed

[arbreezy]

yes we can use localai, cause both support the OpenAI's API spec, but I guess we can surface it as its own backend since it's a different project that localai

[JuHyung-Son]

if they have same inference interface, how about using opensource bankend which can ollama, localai can use.

[ronaldpetty]

Hello, any comment on this code to support Ollama directly?

#1131

Seems to be working locally for me.

# run ollama, assuming locally, here with llama3
make build
./bin/k8sgpt auth add --backend ollama --model llama3 --baseurl http://localhost:11434/v1
kubectl run brokenpod --image=breakit:v1.2.3 
./bin/k8sgpt analyze --explain --backend ollama # output looks good

[yankay]

Hello, any comment on this code to support Ollama directly?

#1131

Seems to be working locally for me.

# run ollama, assuming locally, here with llama3
make build
./bin/k8sgpt auth add --backend ollama --model llama3 --baseurl http://localhost:11434/v1
kubectl run brokenpod --image=breakit:v1.2.3 
./bin/k8sgpt analyze --explain --backend ollama # output looks good

HI @ronaldpetty , follow the #1065 (comment).
Using the official go client makes more sense than the OpenAI client. :-)