(EAI-1159) Implement middleware in new `contentRouter` #816

mmeigs · 2025-07-14T19:39:02Z

Jira: https://jira.mongodb.org/browse/EAI-1159

Changes

Generically type requireValidIpAddress and requireRequestOrigin
Move custom data constants and utils without change to file in /processors
Use custom data within contentRouter and searchContent route
Persist custom data in db

… add test for limit

…SearchFilter types

…t.ts, remove zod checks where unnecessary in MongoDbSearchResultsStore

…ds function outside of file to mock

…versationRouter

…, added to tests

cbush

Passing TS-trivia comment

cbush · 2025-07-15T21:22:49Z

packages/mongodb-chatbot-server/src/processors/addCustomData.ts

+  Has access to the Express.js request and response plus the values
+  from the {@link Response.locals} object.
+ */
+export type AddCustomDataFunc<Req extends Request = Request, Res extends Response = Response> = (


Is this truly generic or is this falling back to the default type parameters of Request?

For example, try

AddCustomDataFunc<Request<{test: true}>>

You get:

Type 'Request<{ test: true; }, any, any, ParsedQs, Record<string, any>>' does not satisfy the constraint 'Request<ParamsDictionary, any, any, ParsedQs, Record<string, any>>'.

Because Req extends Request is actually Req extends Request<core.ParamsDictionary, ...>.

If customizing Request isn't really going to be used, you can skip the generics here and just use the concrete type (i.e. Request).
Otherwise you can pass the type parameters: Req extends Request<unknown, ...>...

Wow, I really went way farther into the weeds on this one than I needed. Thanks!

METAPROGRAMMING IS A TEMPTING POISON APPLE
-Chris Bush

mongodben

looks pretty good. couple small things before approval

packages/chatbot-server-mongodb-public/src/config.ts

mongodben · 2025-07-15T21:56:15Z

packages/mongodb-chatbot-server/src/routes/content/searchContent.ts

 }: MakeSearchContentRouteParams) {
  return async (
-    req: ExpressRequest<SearchContentRequest["params"]>,
+    req: ExpressRequest<ParamsDictionary>,


why this change? seems more generic than what you had before.

also this makes me realize that wen must do an input validation. see next comment

Ah, I see! The main issue was that all of our base Express Request types (created from this) have params as an optional property, but Express Requests always have at least an empty object. I've taken away optional from this zod type and now can keep the above type to the more specific!

mongodben · 2025-07-15T21:58:43Z

packages/mongodb-chatbot-server/src/routes/content/searchContent.ts

 }: MakeSearchContentRouteParams) {
  return async (
-    req: ExpressRequest<SearchContentRequest["params"]>,
+    req: ExpressRequest<ParamsDictionary>,
    res: ExpressResponse<SearchContentResponseBody, SearchContentRouterLocals>
  ) => {
    try {


should validate input with SearchContentRequestBody.safeParse(req.body). and if safeParse encounters an error, you should throw it accordingly. Andrew implemented this pattern in the new Responses API route which you can use as reference https://github.com/mongodb/chatbot/blob/responses_api/packages/mongodb-chatbot-server/src/routes/responses/createResponse.ts#L189-L196

mongodben · 2025-07-15T22:00:45Z

packages/mongodb-chatbot-server/src/routes/content/searchContent.ts

+  } catch (error) {
+    throw makeRequestError({
+      message: "Error parsing custom data from the request",
+      stack: (error as Error).stack,


do you need to include stack here? also i don't like this as, what if the thing being thrown doesn't have stack?

You're right! Removing

For future reference one safe way to do this is:

stack: (error as Partial<Error>)?.stack

You can be almost certain that the thrown object is an Error, but just in case it isn't it won't further blow up your app.

Actually... just the question mark is needed... if error is anything but an Error, it ?.stack will just return undefined.

How do we feel about needing to log the stack trace? In most of our error logs we don't - but we have added it in 5 other places.

mongodben · 2025-07-15T22:01:54Z

packages/mongodb-chatbot-server/src/middleware/requireValidIpAddress.ts

 import { isValidIp } from "../routes/conversations/utils";

-export function requireValidIpAddress(): ConversationsMiddleware {
+export function requireValidIpAddress<


related to chris' comment, if the generics aren't ever being used here (which they dont seem to be), lets remove them

I've added the usage of the generics to all the invocations of these middlewares! Otherwise, the function definitions would need an unknown for the locals values.

mmeigs added 30 commits July 2, 2025 15:21

Create MongoDbSearchResultsStore, add limit to DefaultFindContent and…

9664693

… add test for limit

Implement saveSearchResult, create MongoDbSearchResultsStore.test.ts

f35d9a7

lint, format

b07d9d9

Check entire returned document in MongoDbSearchResultsStore.test.ts

3a1060b

Create ResultChunk type and zod check

f62e36c

Correct usage of limit in makeDefaultFindContent

b6f9aec

PR feedback: cast badSearchResultRecord as any

1c10dfc

Starting structure of searchContent

3a9b937

Use unknown instead of any for ResultChunk additional metadata

f479ea0

Merge branch 'add_search_results_store' into expose_search_endpoint

fe1d015

Add searchContent test file, broaden QueryFilter & MongoDbAtlasVector…

4df3c20

…SearchFilter types

Work on clarity of comments in contentRouter

baad5bd

PR feedback: Combine describe blocks in MongoDbSearchResultsStore.tes…

b3ad758

…t.ts, remove zod checks where unnecessary in MongoDbSearchResultsStore

Merge branch 'add_search_results_store' into expose_search_endpoint

d2b9375

Use generics on middleware: requireRequestOrigin, requireValidIpAddress

370bd86

Structure out contentRouter test file

a9d1910

Combine describes

dd3c316

Merge branch 'add_search_results_store' into expose_search_endpoint

969e3b6

Clean

5ebefb5

makeFindContentWithMongoDbMetadata

647144f

config.test.ts

c65e8e6

Merge branch 'search_content_route' into expose_search_endpoint

a7919c1

Clean

147aea0

PR feedback

5dabbf6

Correct types

6f377f5

Correct test

31710a4

Merge branch 'search_content_route' into expose_search_endpoint

310a607

Fix test return type

fd66498

lint

2f1df27

Revert move of classifyMongoDbProgrammingLanguageAndProduct, jest nee…

8efc4c6

…ds function outside of file to mock

mmeigs added 10 commits July 11, 2025 13:28

Created addCustomData.ts, generics, use in both contentRouter and con…

2c9a21b

…versationRouter

Clean

ff75594

Remove unnecessary tests and comments

19ac7d4

Merge branch 'expose_search_endpoint' into implement_search_middleware

3446476

Added custom middleware to contentRouter, used in searchContent route…

cadae06

…, added to tests

Merge branch 'search_content_route' into implement_search_middleware

c8c38ca

Add customData to db...

25e2c5c

Clean: allow undefined customData value

db98f56

Alter types for createConversationsMiddlewareReq

d435d8f

Rerun tests

3a58520

mmeigs marked this pull request as ready for review July 15, 2025 19:39

mmeigs requested a review from mongodben July 15, 2025 19:39

Merge branch 'search_content_route' into implement_search_middleware

896194c

cbush reviewed Jul 15, 2025

View reviewed changes

mongodben requested changes Jul 15, 2025

View reviewed changes

mmeigs added 2 commits July 16, 2025 10:57

PR feedback

2812d4e

Add Locals types to middleware invocations

9e9c44d

mmeigs requested a review from mongodben July 16, 2025 19:00

Lint, fix trace name, remove unnecessary import

4db7567

(EAI-1159) Implement middleware in new contentRouter #816

Are you sure you want to change the base?

(EAI-1159) Implement middleware in new contentRouter #816

Uh oh!

Conversation

mmeigs commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

cbush left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mongodben left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmeigs Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

(EAI-1159) Implement middleware in new `contentRouter` #816

(EAI-1159) Implement middleware in new `contentRouter` #816

mmeigs commented Jul 14, 2025 •

edited

Loading

mmeigs Jul 16, 2025 •

edited

Loading