-
Notifications
You must be signed in to change notification settings - Fork 265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Agent: clean-up assertion of telemetry events within integration tests #5455
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm sorry, I had published pending comments a long time ago but forgot to "comment" the reviwe.
This is a nice cleanup! Only one blocking comment, otherwise LGTM
@@ -30,6 +30,28 @@ const workspace = new TestWorkspace(path.join(__dirname, '__tests__', 'example-t | |||
const mayRecord = | |||
process.env.CODY_RECORDING_MODE === 'record' || process.env.CODY_RECORD_IF_MISSING === 'true' | |||
|
|||
async function returnTelemetryEvents(currentClient: TestClient) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The returnTelemetryEvents
name is a bit unusual, also the return type is missing
async function returnTelemetryEvents(currentClient: TestClient) { | |
async function exportedTelemetryEvents(currentClient: TestClient): string[] { |
// telemetry assertion, to validate the expected events fired during the test run | ||
// Do not remove this assertion, and instead update the expectedEvents list above | ||
expect(await returnTelemetryEvents(client)).toEqual( | ||
expect.arrayContaining(client.expectedEvents) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should remove the mutable TestClient.expectedEvents
property and inline the values into the assertions instead. This is the only blocking comment for the PR
Previously, the assertion of telemetry events for agent integration tests was happening within an
afterEach
, and this was not the cleanest approach, as the error messages would not point to the relevant code causing the error. In addition, this PR also addsparameters
as a property that gets sent with the testing telemetry events.Now, we manually add the assertion at the end of test cases where we want to assert what events got exported. As @olafurpg mentioned, the benefits of this approach are:
Addresses feedback here
Test plan
CI, specifically
pnpm run test src/index.test.ts
Changelog