Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(GoogleGemini): added api usage with costs #316

Merged
merged 1 commit into from
May 22, 2024
Merged

feat(GoogleGemini): added api usage with costs #316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<PackageVersion Include="DotNet.ReproducibleBuilds" Version="1.1.1" />
<PackageVersion Include="FluentAssertions" Version="6.12.0" />
<PackageVersion Include="GitHubActionsTestLogger" Version="2.3.3" />
<PackageVersion Include="Google_GenerativeAI" Version="0.1.20" />
<PackageVersion Include="Google_GenerativeAI" Version="1.0.0" />
<PackageVersion Include="GroqSharp" Version="1.1.2" />
<PackageVersion Include="H.Generators.Extensions" Version="1.22.0" />
<PackageVersion Include="H.NSwag.Generator" Version="14.0.7.76">
Expand Down
2 changes: 1 addition & 1 deletion src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
if (arguments == null)
return string.Empty;

return JsonSerializer.Serialize(arguments, SourceGenerationContext.Default.DictionaryStringObject);
return JsonSerializer.Serialize(arguments);

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.

Check warning on line 30 in src/Providers/Google/src/Extensions/GoogleGeminiExtensions.cs

View workflow job for this annotation

GitHub Actions / Build and test / Build, test and publish 

Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.
}
}

Expand Down
60 changes: 48 additions & 12 deletions src/Providers/Google/src/GoogleChatModel.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,19 @@ namespace LangChain.Providers.Google;
/// </summary>
public partial class GoogleChatModel(
GoogleProvider provider,
string id)
string id,
int contextLength = 0,
double inputTokenPriceUsd = 0,
double outputTokenPriceUsd = 0,
double secondaryInputTokenPrice = 0,
double secondaryOutputTokenPrice = 0)
Comment on lines +12 to +17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Constructor calls in src/Providers/Google/test/Tests.cs do not include the new parameters for token pricing and context length. Please update the test cases to reflect the changes in the GoogleChatModel constructor.

  • src/Providers/Google/test/Tests.cs: Lines where new GoogleChatModel is called.
Analysis chain

Constructor now correctly includes parameters for token pricing and context length, aligning with new features.

Also applies to: 23-23

Scripts Executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify integration of `GoogleChatModel` constructor with other components.

# Test: Search for the usage of `GoogleChatModel` constructor. Expect: Correct usage of new parameters.
rg --type csharp $'new GoogleChatModel'

Length of output: 270

Script:

#!/bin/bash
# Extract and review the relevant lines from `src/Providers/Google/test/Tests.cs` to verify the usage of new parameters in `GoogleChatModel` constructor.

# Display lines around the constructor usage to check parameter usage.
rg --type csharp --context 5 'new GoogleChatModel' src/Providers/Google/test/Tests.cs

Length of output: 1163

: ChatModel(id)
{
#region Properties

/// <inheritdoc />
public override int ContextLength => 0;
public override int ContextLength => contextLength;


private GenerativeModel Api { get; } = new(
provider.ApiKey,
Expand Down Expand Up @@ -138,11 +144,13 @@ public override async Task<ChatResponse> GenerateAsync(
OnPartialResponseGenerated(Environment.NewLine);
OnCompletedResponseGenerated(response.Text() ?? string.Empty);

// Unsupported
var usage2 = Usage.Empty with

var usage2 = GetUsage(response) with
{
Time = watch.Elapsed
};

//Add Usage
AddUsage(usage2);
provider.AddUsage(usage2);

Expand Down Expand Up @@ -176,7 +184,8 @@ public override async Task<ChatResponse> GenerateAsync(

messages.Add(message);

usage2 = Usage.Empty with
//Add Usage
usage2 = GetUsage(response) with
{
Time = watch.Elapsed
};
Expand All @@ -185,12 +194,8 @@ public override async Task<ChatResponse> GenerateAsync(
}
}
}


//Function Call


// Unsupported

//Add Usage
var usage = Usage.Empty with
{
Time = watch.Elapsed
Expand All @@ -201,11 +206,42 @@ public override async Task<ChatResponse> GenerateAsync(
return new ChatResponse
{
Messages = messages,
Usage = usage,
Usage = Usage,
UsedSettings = ChatSettings.Default
};
}
private Usage GetUsage(EnhancedGenerateContentResponse response)
{
var outputTokens = response.UsageMetadata?.CandidatesTokenCount ?? 0;
var inputTokens = response.UsageMetadata?.PromptTokenCount ?? 0;
var priceInUsd = CalculatePriceInUsd(
outputTokens: outputTokens,
inputTokens: inputTokens);

return Usage.Empty with
{
InputTokens = inputTokens,
OutputTokens = outputTokens,
Messages = 1,
PriceInUsd = priceInUsd,
};
}
/// <inheritdoc/>
public double CalculatePriceInUsd(int inputTokens, int outputTokens)
{
if (inputTokens < 128 * 1024)
{
var inputCost = inputTokenPriceUsd * inputTokens;
var outputCost = outputTokenPriceUsd * outputTokens;
return inputCost + outputCost;
}
else
{
var inputCost = secondaryInputTokenPrice * inputTokens;
var outputCost = secondaryOutputTokenPrice * outputTokens;
return inputCost + outputCost;
}
}
private static Message ToFunctionCallMessage(string jsonResult, string functionName)
{
//var result = JsonSerializer.Deserialize<JsonNode>(jsonResult, SerializerOptions);
Expand Down
16 changes: 14 additions & 2 deletions src/Providers/Google/src/Predefined/GeminiModels.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,22 @@ namespace LangChain.Providers.Google.Predefined;
public class GeminiProModel(GoogleProvider provider)
: GoogleChatModel(
provider,
GoogleAIModels.GeminiPro);
GoogleAIModels.GeminiPro, 32 * 1024, 0.5 * 0.000001, 1.5 * 0.000001);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

The GeminiProModel constructor calls in GenerativeModelTests.cs and Tests.cs do not include the new parameters for token pricing and context length. Please update these test files to reflect the changes in the constructor.

  • src/Providers/Google/test/GenerativeModelTests.cs
  • src/Providers/Google/test/Tests.cs
Analysis chain

Constructor now correctly includes parameters for token pricing and context length, aligning with new features.

Scripts Executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify integration of `GeminiProModel` with other components.

# Test: Search for the usage of `GeminiProModel` constructor. Expect: Correct usage of new parameters.
rg --type csharp $'new GeminiProModel'

Length of output: 252

Script:

#!/bin/bash
# Extract lines around the usage of `GeminiProModel` constructor in `GenerativeModelTests.cs`
rg --context 5 'new GeminiProModel' src/Providers/Google/test/GenerativeModelTests.cs

# Extract lines around the usage of `GeminiProModel` constructor in `Tests.cs`
rg --context 5 'new GeminiProModel' src/Providers/Google/test/Tests.cs

Length of output: 1181


/// <inheritdoc cref="GoogleAIModels.GeminiProVision" />
public class GeminiProVisionModel(GoogleProvider provider)
: GoogleChatModel(
provider,
GoogleAIModels.GeminiProVision);
GoogleAIModels.GeminiProVision, 32 * 1024, 0.5 * 0.000001, 1.5 * 0.000001);

/// <inheritdoc cref="GoogleAIModels.GeminiProVision" />
public class Gemini15FlashModel(GoogleProvider provider)
: GoogleChatModel(
provider,
GoogleAIModels.Gemini15Flash, 1024 * 1024, 0.35 * 0.000001, 1.05 * 0.000001, 0.70 * 0.000001, 2.1 * 0.000001);

/// <inheritdoc cref="GoogleAIModels.GeminiProVision" />
public class Gemini15ProModel(GoogleProvider provider)
: GoogleChatModel(
provider,
GoogleAIModels.Gemini15Flash, 2 * 1024 * 1024, 3.5 * 0.000001, 10.50 * 0.000001, 7.0 * 0.000001, 21.00 * 0.000001);
6 changes: 5 additions & 1 deletion src/Providers/Google/test/Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using GenerativeAI.Models;
using GeminiProModel = LangChain.Providers.Google.Predefined.GeminiProModel;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

The GetWeather method implementations across different providers have been successfully extracted. The Google provider's test method includes the new assertions for token counts and pricing, which aligns with the PR objectives. However, the other providers' test methods do not include these new assertions.

  • Google Provider: The GetWeather method includes assertions for InputTokens and OutputTokens.
  • Other Providers: The GetWeather methods do not include the new assertions for token counts and pricing.

Locations that need attention:

  • src/Providers/TogetherAI/tests/Tests.cs
  • src/Providers/OpenRouter/tests/Tests.cs
  • src/Providers/OpenAI/test/Tests.cs
  • src/Providers/DeepInfra/tests/Tests.cs
  • src/Providers/Anthropic/tests/Tests.cs

These files should be updated to include the new assertions for token counts and pricing in their respective GetWeather methods.

Analysis chain

Updated import and added assertions for token counts and pricing in the GetWeather test method align with the new features.

Also applies to: 17-17, 29-31

Scripts Executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify integration of `GetWeather` test method with other components.

# Test: Search for the usage of `GetWeather` method. Expect: Correct usage of new assertions.
rg --type csharp $'GetWeather'

Length of output: 467

Script:

#!/bin/bash
# Extract the implementation of the `GetWeather` method from each identified file.

# Define the pattern to match the `GetWeather` method.
pattern=$'public async Task GetWeather() { $$$ }'

# Search and extract the `GetWeather` method implementations.
ast-grep --lang csharp --pattern "$pattern" src/Providers/TogetherAI/tests/Tests.cs
ast-grep --lang csharp --pattern "$pattern" src/Providers/OpenRouter/tests/Tests.cs
ast-grep --lang csharp --pattern "$pattern" src/Providers/OpenAI/test/Tests.cs
ast-grep --lang csharp --pattern "$pattern" src/Providers/Google/test/Tests.cs
ast-grep --lang csharp --pattern "$pattern" src/Providers/DeepInfra/tests/Tests.cs
ast-grep --lang csharp --pattern "$pattern" src/Providers/Anthropic/tests/Tests.cs

Length of output: 734

Script:

#!/bin/bash
# Extract the implementation of the `GetWeather` method from each identified file.

# Define a more flexible pattern to match the `GetWeather` method.
pattern=$'public async Task GetWeather() { $$$ }'

# Search and extract the `GetWeather` method implementations.
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/TogetherAI/tests/Tests.cs
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/OpenRouter/tests/Tests.cs
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/OpenAI/test/Tests.cs
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/Google/test/Tests.cs
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/DeepInfra/tests/Tests.cs
ast-grep --lang csharp --pattern 'public async Task GetWeather() { $$$ }' src/Providers/Anthropic/tests/Tests.cs

Length of output: 734

Script:

#!/bin/bash
# Extract the implementation of the `GetWeather` method from each identified file using a more general approach.

# Search for the `GetWeather` method and extract surrounding lines.
rg 'public async Task GetWeather' -A 20 src/Providers/TogetherAI/tests/Tests.cs
rg 'public async Task GetWeather' -A 20 src/Providers/OpenRouter/tests/Tests.cs
rg 'public async Task GetWeather' -A 20 src/Providers/OpenAI/test/Tests.cs
rg 'public async Task GetWeather' -A 20 src/Providers/Google/test/Tests.cs
rg 'public async Task GetWeather' -A 20 src/Providers/DeepInfra/tests/Tests.cs
rg 'public async Task GetWeather' -A 20 src/Providers/Anthropic/tests/Tests.cs

Length of output: 5376


namespace LangChain.Providers.Google.Tests;

Expand All @@ -13,7 +14,7 @@ public async Task GetWeather()
throw new InvalidOperationException("Gemini_API_Key is not set");


var model = new GoogleChatModel(apiKey, GoogleAIModels.GeminiPro);
var model = new GeminiProModel(apiKey);//, GoogleAIModels.GeminiPro);

var service = new WeatherService();
model.AddGlobalTools(service.AsGoogleFunctions(), service.AsGoogleCalls());
Expand All @@ -25,6 +26,9 @@ public async Task GetWeather()
"Sure! Could you please provide me with your location?".AsAiMessage(),
"Dubai, UAE".AsHumanMessage(),
});
response.Usage.InputTokens.Should().BeGreaterThan(0);
response.Usage.OutputTokens.Should().BeGreaterThan(0);
response.Usage.PriceInUsd.Should().BeGreaterThan(0);

Console.WriteLine(response.Messages.AsHistory());
}
Expand Down
Loading