handle data streaming for new responses API #807

ASteinheiser · 2025-07-07T22:11:41Z

Jira: https://jira.mongodb.org/browse/EAI-1154

Changes

implement data streaming for responses api
matches openai spec for response data and errors

Notes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.ts

packages/mongodb-rag-core/src/DataStreamer.ts

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts

mongodben

definitely looks directionally correct. left a few notes.

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts

packages/mongodb-chatbot-server/src/test/testHelpers.ts

packages/mongodb-rag-core/src/openai.ts

mongodben

back 2 u. think getting very close. this looks great. very exciting. happy to chat further on any of the points in a call

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts

packages/mongodb-chatbot-server/src/routes/responses/createResponse.ts

packages/mongodb-chatbot-server/src/routes/responses/responsesRouter.test.ts

packages/mongodb-rag-core/src/DataStreamer.ts

mongodben

LGTM % few tiny nits and 1 request for tests. awesome work here! we're getting close.

with this, you've pretty much finished the hard part on the server, and now get to do the fun (and also probably somewhat hard) part integrating the AI

ASteinheiser added 12 commits July 7, 2025 14:48

ensure stream is configurable in chatbot-server-public layer

53d2946

setup data streamer helper

c853971

add skeleton data streaming to createResponse service

50e58ba

move StreamFunction type to chat-server, share with public

5f094df

fix test

5ee9ce3

start streaming in createResponse

685f140

add stream disconnect

e5c600f

move in progress stream message

636a3db

stream event from verifiedAnswer helper

eb511bf

create helper for addMessageToConversationVerifiedAnswerStream

29c18cd

apply stream configs to chat-public

0b02450

update test to use openAI client

e521aff

mongodben reviewed Jul 8, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.ts Show resolved Hide resolved

mongodben reviewed Jul 8, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 8, 2025

View reviewed changes

packages/mongodb-rag-core/src/DataStreamer.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 8, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 8, 2025

View reviewed changes

ASteinheiser added 13 commits July 8, 2025 16:31

update input schema for openai client call to responses

5f41b6d

add test helper for making local server

25ea804

almost finish converting createResposne tests to use openai client

838fc86

more tests

1c1305a

disconnect data streamed on error

f821f8f

update data stream logic for create response

9822ecf

update data streamer sendResponsesEvent write type

94fdc7b

i think this still needs to be a string

9c706a2

mapper for streamError

503b2df

export openai shim types

5a879f8

mostly working tests with reading the response stream from openai client

c4d1cf6

create test helper

dd736b0

fix test helper for reading entire stream

1c34c76

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/routes/responses/createResponse.test.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/test/testHelpers.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-chatbot-server/src/test/testHelpers.ts Outdated Show resolved Hide resolved

mongodben reviewed Jul 14, 2025

View reviewed changes

packages/mongodb-rag-core/src/openai.ts Outdated Show resolved Hide resolved

mongodben requested changes Jul 14, 2025

View reviewed changes

update tests per review

b7a1424

ASteinheiser requested a review from mongodben July 15, 2025 08:01

ASteinheiser added 8 commits July 15, 2025 13:32

update test utils

e976991

fix test type

1ccd448

update openai rag-core to 5.9

7c6c712

fix data streamer for responses events to be SSE compliant

bf0571a

cleanup responses tests

87d7b7e

cleanup createResponse tests

e0dc84d

cleanup error handling to match openai spec

c1629fe

fix tests for standard openai exceptions

e1f3cad