diff --git a/.gitignore b/.gitignore index c8ebd78..0e182fd 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,6 @@ dist # VS Code .vscode + +# Intellij +.idea diff --git a/README.md b/README.md index f2c1ce8..6ea65d2 100644 --- a/README.md +++ b/README.md @@ -11,16 +11,17 @@ SendGrid-Mock serves as a simple server mocking the sendgrid-apis for developmen * Send mails `POST /v3/mail/send` * Retrieve sent mails `GET /api/mails` - * Filter capabilities are included and can be combined: - * **To**: `GET /api/mails?to=email@address.com` - * **Subject**: - * `GET /api/mails?subject=The subject` (*exact match*) - * `GET /api/mails?subject=%subject%` (*contains*) - * **Datetime**: `GET /api/mails?dateTimeSince=2020-12-06T10:00:00Z` (*[ISO-8601 format](https://en.wikipedia.org/wiki/ISO_8601)*) + * Filter capabilities are included and can be combined: + * **To**: `GET /api/mails?to=email@address.com` + * **Subject**: + * `GET /api/mails?subject=The subject` (*exact match*) + * `GET /api/mails?subject=%subject%` (*contains*) + * **Datetime**: `GET /api/mails?dateTimeSince=2020-12-06T10:00:00Z` ( + *[ISO-8601 format](https://en.wikipedia.org/wiki/ISO_8601)*) * Delete sent mails `DELETE /api/mails` - * Filter capabilities are included and can be combined: - * **To**: `DELETE /api/mails?to=email@address.com` + * Filter capabilities are included and can be combined: + * **To**: `DELETE /api/mails?to=email@address.com` ### UI @@ -29,21 +30,31 @@ SendGrid-Mock serves as a simple server mocking the sendgrid-apis for developmen ### Extras -* Basic authentication support: Add basic authentication credentials by specifying environment variable `AUTHENTICATION` to the following format: `user1:passwordForUser1;user2:passwordForUser2` +* Basic authentication support: Add basic authentication credentials by specifying environment variable `AUTHENTICATION` + to the following format: `user1:passwordForUser1;user2:passwordForUser2` -* Request rate limiting: Both the actual SendGrid API server as well as the SSL server can be rate limited by specifying environment variables: - * `RATE_LIMIT_ENABLED`: `true` or `false` (default) - * `RATE_LIMIT_WINDOW_IN_MS`: The time window in milliseconds (default: `60000`) - * `RATE_LIMIT_MAX_REQUESTS`: The maximum number of requests allowed in the time window (default: `100`) - * `SSL_RATE_LIMIT_ENABLED`: `true` or `false` (default) - * `SSL_RATE_LIMIT_WINDOW_IN_MS`: The time window in milliseconds (default: `60000`) - * `SSL_RATE_LIMIT_MAX_REQUESTS`: The maximum number of requests allowed in the time window (default: `100`) +* Request rate limiting: Both the actual SendGrid API server as well as the SSL server can be rate limited by specifying + environment variables: + * `RATE_LIMIT_ENABLED`: `true` or `false` (default) + * `RATE_LIMIT_WINDOW_IN_MS`: The time window in milliseconds (default: `60000`) + * `RATE_LIMIT_MAX_REQUESTS`: The maximum number of requests allowed in the time window (default: `100`) + * `SSL_RATE_LIMIT_ENABLED`: `true` or `false` (default) + * `SSL_RATE_LIMIT_WINDOW_IN_MS`: The time window in milliseconds (default: `60000`) + * `SSL_RATE_LIMIT_MAX_REQUESTS`: The maximum number of requests allowed in the time window (default: `100`) -* By default, all emails older than 24 hours will be deleted. This can be configured using environment variable `MAIL_HISTORY_DURATION` which uses [ISO-8601 Duration format](https://en.wikipedia.org/wiki/ISO_8601#Durations) such as *'PT24H'*. +* By default, all emails older than 24 hours will be deleted. This can be configured using environment + variable `MAIL_HISTORY_DURATION` which + uses [ISO-8601 Duration format](https://en.wikipedia.org/wiki/ISO_8601#Durations) such as *'PT24H'*. + +* Event support: Add basic [event](https://www.twilio.com/docs/sendgrid/for-developers/tracking-events/event#events) + support by specifying the environment variable `EVENT_DELIVERY_URL`. When set, + [delivered](https://www.twilio.com/docs/sendgrid/for-developers/tracking-events/event#delivered) events will be + sent to the specified webhook URL when an email is sent. ## Dockerized -The SendGrid-Mock server and the UI are both contained in the same docker-image which you can pull from [Docker Hub](https://cloud.docker.com/u/ghashange/repository/docker/ghashange/sendgrid-mock) and start it via: +The SendGrid-Mock server and the UI are both contained in the same docker-image which you can pull +from [Docker Hub](https://cloud.docker.com/u/ghashange/repository/docker/ghashange/sendgrid-mock) and start it via: ```shell docker run -it -p 3000:3000 -e "API_KEY=sendgrid-api-key" ghashange/sendgrid-mock:1.9.2 @@ -63,7 +74,8 @@ docker run -it -p 3000:3000 -e "API_KEY=sendgrid-api-key" -e "CERT_DOMAINNAMES=[ ## Development -Setup with `npm ci` and start both server and UI concurrently with `npm run dev`. Per default the server is reachable via and the UI via . +Setup with `npm ci` and start both server and UI concurrently with `npm run dev`. Per default the server is reachable +via and the UI via . Some prepared HTTP calls can be found [here](./http-calls). @@ -81,4 +93,5 @@ Create docker image with `docker build -t ghashange/sendgrid-mock:1.9.2 .`. 3. Merge PR -4. Create GitHub release and update [Docker Hub description](https://hub.docker.com/repository/docker/ghashange/sendgrid-mock) +4. Create GitHub release and + update [Docker Hub description](https://hub.docker.com/repository/docker/ghashange/sendgrid-mock) diff --git a/__mocks__/axios.js b/__mocks__/axios.js new file mode 100644 index 0000000..8940615 --- /dev/null +++ b/__mocks__/axios.js @@ -0,0 +1,6 @@ +const axios = { + post: jest.fn(() => Promise.resolve({ data: {} })), + // Add other axios methods you use here +}; + +module.exports = axios; \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index aa3911b..8cebb99 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,6 +9,7 @@ "version": "1.9.2", "license": "MIT", "dependencies": { + "axios": "^1.7.2", "body-parser": "^1.20.1", "express": "^4.19.2", "express-basic-auth": "^1.2.0", @@ -3648,8 +3649,30 @@ "node_modules/asynckit": { "version": "0.4.0", "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", - "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=", - "dev": true + "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=" + }, + "node_modules/axios": { + "version": "1.7.2", + "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.2.tgz", + "integrity": "sha512-2A8QhOMrbomlDuiLeK9XibIBzuHeRcqqNOHp0Cyp5EoJ1IFDh+XZH3A6BkXtv0K4gFGCI0Y4BM7B1wOEi0Rmgw==", + "dependencies": { + "follow-redirects": "^1.15.6", + "form-data": "^4.0.0", + "proxy-from-env": "^1.1.0" + } + }, + "node_modules/axios/node_modules/form-data": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz", + "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "mime-types": "^2.1.12" + }, + "engines": { + "node": ">= 6" + } }, "node_modules/babel-eslint": { "version": "10.0.2", @@ -4136,7 +4159,6 @@ "version": "1.0.8", "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", - "dev": true, "dependencies": { "delayed-stream": "~1.0.0" }, @@ -4724,7 +4746,6 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=", - "dev": true, "engines": { "node": ">=0.4.0" } @@ -5804,6 +5825,25 @@ "resolved": "https://registry.npmjs.org/flatted/-/flatted-3.2.6.tgz", "integrity": "sha512-0sQoMh9s0BYsm+12Huy/rkKxVu4R1+r96YX5cG44rHV0pQ6iC3Q+mkoMFaGWObMFYQxCVT+ssG1ksneA2MI9KQ==" }, + "node_modules/follow-redirects": { + "version": "1.15.6", + "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.6.tgz", + "integrity": "sha512-wWN62YITEaOpSK584EZXJafH1AGpO8RVgElfkuXbTOrPX4fIfOyEpW/CsiNd8JdYrAoOvafRTOEnvsO++qCqFA==", + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/RubenVerborgh" + } + ], + "engines": { + "node": ">=4.0" + }, + "peerDependenciesMeta": { + "debug": { + "optional": true + } + } + }, "node_modules/form-data": { "version": "3.0.1", "resolved": "https://registry.npmjs.org/form-data/-/form-data-3.0.1.tgz", @@ -9392,6 +9432,11 @@ "node": ">= 0.10" } }, + "node_modules/proxy-from-env": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz", + "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==" + }, "node_modules/psl": { "version": "1.9.0", "resolved": "https://registry.npmjs.org/psl/-/psl-1.9.0.tgz", @@ -13250,8 +13295,29 @@ "asynckit": { "version": "0.4.0", "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", - "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=", - "dev": true + "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=" + }, + "axios": { + "version": "1.7.2", + "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.2.tgz", + "integrity": "sha512-2A8QhOMrbomlDuiLeK9XibIBzuHeRcqqNOHp0Cyp5EoJ1IFDh+XZH3A6BkXtv0K4gFGCI0Y4BM7B1wOEi0Rmgw==", + "requires": { + "follow-redirects": "^1.15.6", + "form-data": "^4.0.0", + "proxy-from-env": "^1.1.0" + }, + "dependencies": { + "form-data": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz", + "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==", + "requires": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "mime-types": "^2.1.12" + } + } + } }, "babel-eslint": { "version": "10.0.2", @@ -13632,7 +13698,6 @@ "version": "1.0.8", "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", - "dev": true, "requires": { "delayed-stream": "~1.0.0" } @@ -14074,8 +14139,7 @@ "delayed-stream": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", - "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=", - "dev": true + "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=" }, "depd": { "version": "2.0.0", @@ -14892,6 +14956,11 @@ "resolved": "https://registry.npmjs.org/flatted/-/flatted-3.2.6.tgz", "integrity": "sha512-0sQoMh9s0BYsm+12Huy/rkKxVu4R1+r96YX5cG44rHV0pQ6iC3Q+mkoMFaGWObMFYQxCVT+ssG1ksneA2MI9KQ==" }, + "follow-redirects": { + "version": "1.15.6", + "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.6.tgz", + "integrity": "sha512-wWN62YITEaOpSK584EZXJafH1AGpO8RVgElfkuXbTOrPX4fIfOyEpW/CsiNd8JdYrAoOvafRTOEnvsO++qCqFA==" + }, "form-data": { "version": "3.0.1", "resolved": "https://registry.npmjs.org/form-data/-/form-data-3.0.1.tgz", @@ -17575,6 +17644,11 @@ "ipaddr.js": "1.9.1" } }, + "proxy-from-env": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz", + "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==" + }, "psl": { "version": "1.9.0", "resolved": "https://registry.npmjs.org/psl/-/psl-1.9.0.tgz", diff --git a/package.json b/package.json index 9d7aee3..549dfc4 100644 --- a/package.json +++ b/package.json @@ -28,6 +28,7 @@ }, "license": "MIT", "dependencies": { + "axios": "^1.7.2", "body-parser": "^1.20.1", "express": "^4.19.2", "express-basic-auth": "^1.2.0", diff --git a/src/server/RequestHandler.js b/src/server/RequestHandler.js index 8b2ca05..b7d5ef2 100644 --- a/src/server/RequestHandler.js +++ b/src/server/RequestHandler.js @@ -95,11 +95,12 @@ const RequestHandler = (app, apiAuthenticationKey, mailHandler) => { const reqApiKey = req.headers.authorization; if (reqApiKey === `Bearer ${apiAuthenticationKey}`) { + const messageId = crypto.randomUUID(); - mailHandler.addMail(req.body); + mailHandler.addMail(req.body, messageId); res.status(202).header({ - 'X-Message-ID': crypto.randomUUID(), + 'X-Message-ID': messageId, }).send(); } else { res.status(403).send({ diff --git a/src/server/handler/MailHandler.js b/src/server/handler/MailHandler.js index 055fbf7..424ffa8 100644 --- a/src/server/handler/MailHandler.js +++ b/src/server/handler/MailHandler.js @@ -1,5 +1,8 @@ const {loggerFactory} = require('../logger/log4js'); +const axios = require('axios'); +const crypto = require('crypto'); + const logger = loggerFactory('MailHandler'); const mailWithTimestamp = (mail) => { @@ -123,7 +126,7 @@ class MailHandler { .slice(paginationStart, paginationStart + paginationSize); } - addMail(mail) { + addMail(mail, messageId = crypto.randomUUID()) { this.#mails = [mailWithTimestamp(mail), ...this.#mails]; @@ -132,9 +135,34 @@ class MailHandler { return Date.parse(mail.datetime).valueOf() >= maxRetentionTime; }); + if (process.env.EVENT_DELIVERY_URL) { + this.sendDeliveryEvents(mail, messageId); + } + logMemoryUsage(this.#mails); } + sendDeliveryEvents(mail, messageId) { + const datetime = new Date(); + const deliveredEvents = mail.personalizations + .flatMap(personalization => personalization.to) + .map(to => { + let event = { + email: to.email, + timestamp: datetime.getTime(), + event: 'delivered', + sg_event_id: crypto.randomUUID(), + sg_message_id: messageId, + }; + event['smtp-id'] = crypto.randomUUID(); + return event; + }); + + axios.post(process.env.EVENT_DELIVERY_URL, deliveredEvents) + .then(() => logger.debug(`Delivery events sent successfully to ${process.env.EVENT_DELIVERY_URL}`)) + .catch((error) => logger.debug(`Failed to send delivery events to ${process.env.EVENT_DELIVERY_URL}`, error)); + } + clear(filterCriteria) { const filters = [ diff --git a/test/server/handler/MailHandler.test.js b/test/server/handler/MailHandler.test.js index ee78682..387cb85 100644 --- a/test/server/handler/MailHandler.test.js +++ b/test/server/handler/MailHandler.test.js @@ -1,5 +1,10 @@ const {withMockedDate} = require('../../MockDate'); +const axios = require('axios'); +jest.mock('axios'); + +const crypto = require('crypto'); + const MailHandler = require('../../../src/server/handler/MailHandler'); const testMail = { @@ -54,6 +59,55 @@ describe('MailHandler', () => { expect(addedMails.length).toBe(3); }); + describe('add mail when EVENT_DELIVERY_URL is set', () => { + + beforeAll(() => { + process.env.EVENT_DELIVERY_URL = 'http://example.com'; + }); + + test('send delivery events', () => { + const sut = new MailHandler(); + + axios.post.mockResolvedValue({data: {message: 'success'}}); + + const messageId = crypto.randomUUID(); + + sut.addMail(testMail, messageId); + + expect(axios.post.mock.calls[0][0]).toEqual(process.env.EVENT_DELIVERY_URL); + const eventData = axios.post.mock.calls[0][1]; + expect(eventData.length).toBe(2); + expect(eventData[0]).toMatchObject({ + email: testMail.personalizations[0].to[0].email, + event: 'delivered', + timestamp: expect.any(Number), + sg_event_id: expect.any(String), + sg_message_id: messageId, + 'smtp-id': expect.any(String), + }); + }); + + test('send delivery events with defaulted messageId', () => { + const sut = new MailHandler(); + + axios.post.mockResolvedValue({data: {message: 'success'}}); + + sut.addMail(testMail); + + expect(axios.post.mock.calls[0][0]).toEqual(process.env.EVENT_DELIVERY_URL); + const eventData = axios.post.mock.calls[0][1]; + expect(eventData.length).toBe(2); + expect(eventData[0]).toMatchObject({ + email: testMail.personalizations[0].to[0].email, + event: 'delivered', + timestamp: expect.any(Number), + sg_event_id: expect.any(String), + sg_message_id: expect.any(String), + 'smtp-id': expect.any(String), + }); + }); + }); + describe('delete old mails', () => { test('if not configured, per default after 24 hours', () => {