Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batched inputs to genai python api #555

Open
DavidePaglieri opened this issue Sep 12, 2024 · 0 comments
Open

Batched inputs to genai python api #555

DavidePaglieri opened this issue Sep 12, 2024 · 0 comments
Assignees
Labels
component:support How to do xyz? status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement

Comments

@DavidePaglieri
Copy link

DavidePaglieri commented Sep 12, 2024

Description of the feature request:

Is it possible to have batched inputs with the genai python api? Currently from the documentation I see that only Vertex AI supports batched prediction for Gemini

It seems like a crucial feature to have for widespread gemini adoption to allow batched inputs directly from this genai python api. Openai python api supports it, and it is quite convenient. Would it be possible to add this feature?

What problem are you trying to solve with this feature?

This would be very useful in cases we want to have large batches of predictions with gemini directly from python, without having to go through vertex ai. Would be overall a convenient feature to have.

@gmKeshari gmKeshari added type:feature request New feature request/enhancement status:triaged Issue/PR triaged to the corresponding sub-team component:support How to do xyz? labels Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:support How to do xyz? status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement
Projects
None yet
Development

No branches or pull requests

3 participants