Google Vertex AI Integration Idea

[arjunattam]

Discussed in https://github.com/empirical-run/empirical/discussions/235

^{Originally posted by TalkTomeG00se May 29, 2024}
Hello!
First I want to say thank you for this awesome tool, really enjoy it's flexibility and ease of use.

This is part idea and question. I think integration somehow with the Vertex AI platform would be great. I do see that Empirical supports Gemini models via a Google API key. Those API keys are generated through access to Google AI Studio, which in turn is accessed through Google Workspaces. In my instance however, my company doesn't leverage Workspaces, but instead has access to Vertex AI within Google Cloud, and access to that API is not done through API keys, but through IAM policies/application default credentials.

The nice thing about using the Gemini API for Vertex AI, is that we have access to dozens of models including all the Google of course, Anthropic, Mistral etc.

Not sure if that can be done, but wanted to throw that idea out there. Thanks again for a amazing tool, keep up the amazing work!

cc @TalkTomeG00se

[arjunattam]

Thanks for the request. How do you authenticate your requests to Vertex AI endpoints? Based on this guide, I see that there are few different approaches. It'll help to know which one to prioritize.

[TalkTomeG00se]

We leverage Application Default Credentials to access Vertex AI right now. Though if one implementation is easier to implement within Empirical than another, we can do that too, we just use ADC right now.