:read` permission is required.
+
+
+If the authentication was successful and the client has read access to at least one collection, `success` = `true` is set in the response object and the field `collections` contains an array of collections which were actually subscribed (granted).
+In other case `success` = `false` is set in the response object and the field `message` contains an error message.
+
+```javascript
+function (data) {
+ if (data.success) {
+ console.log('subscribed for collections', data.collections);
+ }
+ else {
+ console.error(data.message);
+ }
+ });
+});
+```
+
+### Receiving events
+After the successful subscription the client can start listening to `create`, `update` and/or `delete` events of the socket.
+
+
+##### create
+`create` event fires each time a new document is inserted into the storage, regardless of whether it was CREATE or UPDATE operation of APIv3 (both of these operations are upserting/deduplicating, so they are "insert capable"). If the document already existed in the storage, the `update` event would be fired instead.
+
+The received object contains:
+* `colName` field with the name of the affected collection
+* the inserted document in `doc` field
+
+```javascript
+socket.on('create', function (data) {
+ console.log(`${data.colName}:created document`, data.doc);
+});
+```
+
+
+##### update
+`update` event fires each time an existing document is modified in the storage, regardless of whether it was CREATE, UPDATE or PATCH operation of APIv3 (all of these operations are "update capable"). If the document did not yet exist in the storage, the `create` event would be fired instead.
+
+The received object contains:
+* `colName` field with the name of the affected collection
+* the new version of the modified document in `doc` field
+
+```javascript
+socket.on('update', function (data) {
+ console.log(`${data.colName}:updated document`, data.doc);
+});
+```
+
+
+##### delete
+`delete` event fires each time an existing document is deleted in the storage, regardless of whether it was "soft" (marking as invalid) or permanent deleting.
+
+The received object contains:
+* `colName` field with the name of the affected collection
+* the identifier of the deleted document in the `identifier` field
+
+```javascript
+socket.on('delete', function (data) {
+ console.log(`${data.colName}:deleted document with identifier`, data.identifier);
+});
+```
\ No newline at end of file
diff --git a/lib/api3/doc/tutorial.md b/lib/api3/doc/tutorial.md
new file mode 100644
index 00000000000..3d8c656dfbd
--- /dev/null
+++ b/lib/api3/doc/tutorial.md
@@ -0,0 +1,329 @@
+# APIv3: Basics tutorial
+
+Nightscout API v3 is a component of [cgm-remote-monitor](https://github.com/nightscout/cgm-remote-monitor) project.
+It aims to provide lightweight, secured and HTTP REST compliant interface for your T1D treatment data exchange.
+
+There is a list of REST operations that the API v3 offers (inside `/api/v3` relative URL namespace), we will briefly introduce them in this file.
+
+Each NS instance with API v3 contains self-included OpenAPI specification at [/api/v3/swagger-ui-dist/](https://nsapiv3.herokuapp.com/api/v3/swagger-ui-dist/) relative URL.
+
+
+---
+### VERSION
+
+[VERSION](https://nsapiv3.herokuapp.com/api/v3/swagger-ui-dist/#/other/get_version) operation gets you basic information about software packages versions.
+It is public (there is no need to add authorization parameters/headers).
+
+Sample GET `/version` client code (to get actual versions):
+```javascript
+const request = require('request');
+
+request('https://nsapiv3.herokuapp.com/api/v3/version',
+ (error, response, body) => console.log(body));
+```
+Sample result:
+```javascript
+{
+ "version":"0.12.2",
+ "apiVersion":"3.0.0-alpha",
+ "srvDate":1564386001772,
+ "storage":{
+ "storage":"mongodb",
+ "version":"3.6.12"
+ }
+}
+```
+
+
+---
+### STATUS
+
+[STATUS](https://nsapiv3.herokuapp.com/api/v3/swagger-ui-dist/#/other/get_status) operation gets you basic information about software packages versions.
+It is public (there is no need to add authorization parameters/headers).
+
+Sample GET `/status` client code (to get my actual permissions):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+
+request(`https://nsapiv3.herokuapp.com/api/v3/status?${auth}`,
+ (error, response, body) => console.log(body));
+```
+Sample result:
+```javascript
+{
+ "version":"0.12.2",
+ "apiVersion":"3.0.0-alpha",
+ "srvDate":1564391740738,
+ "storage":{
+ "storage":"mongodb",
+ "version":"3.6.12"
+ },
+ "apiPermissions":{
+ "devicestatus":"crud",
+ "entries":"crud",
+ "food":"crud",
+ "profile":"crud",
+ "settings":"crud",
+ "treatments":"crud"
+ }
+}
+```
+`"crud"` represents create + read + update + delete permissions for the collection.
+
+
+---
+### SEARCH
+
+[SEARCH](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/generic/SEARCH) operation filters, sorts, paginates and projects documents from the collection.
+
+Sample GET `/entries` client code (to retrieve last 3 BG values):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+
+request(`https://nsapiv3.herokuapp.com/api/v3/entries?${auth}&sort$desc=date&limit=3&fields=dateString,sgv,direction`,
+ (error, response, body) => console.log(body));
+```
+Sample result:
+```
+[
+ {
+ "dateString":"2019-07-30T02:24:50.434+0200",
+ "sgv":115,
+ "direction":"FortyFiveDown"
+ },
+ {
+ "dateString":"2019-07-30T02:19:50.374+0200",
+ "sgv":121,
+ "direction":"FortyFiveDown"
+ },
+ {
+ "dateString":"2019-07-30T02:14:50.450+0200",
+ "sgv":129,
+ "direction":"FortyFiveDown"
+ }
+]
+```
+
+
+---
+### CREATE
+
+[CREATE](https://nsapiv3.herokuapp.com/api/v3/swagger-ui-dist/#/generic/post__collection_) operation inserts a new document into the collection.
+
+Sample POST `/treatments` client code:
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const doc = {
+ date: 1564591511232, // (new Date()).getTime(),
+ app: 'AndroidAPS',
+ device: 'Samsung XCover 4-861536030196001',
+ eventType: 'Correction Bolus',
+ insulin: 0.3
+};
+request({
+ method: 'post',
+ body: doc,
+ json: true,
+ url: `https://nsapiv3.herokuapp.com/api/v3/treatments?${auth}`
+ },
+ (error, response, body) => console.log(response.headers.location));
+```
+Sample result:
+```
+/api/v3/treatments/95e1a6e3-1146-5d6a-a3f1-41567cae0895
+```
+
+
+---
+### READ
+
+[READ](https://nsapiv3.herokuapp.com/api/v3/swagger-ui-dist/#/generic/get__collection___identifier_) operation retrieves you a single document from the collection by its identifier.
+
+Sample GET `/treatments/{identifier}` client code:
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const identifier = '95e1a6e3-1146-5d6a-a3f1-41567cae0895';
+
+request(`https://nsapiv3.herokuapp.com/api/v3/treatments/${identifier}?${auth}`,
+ (error, response, body) => console.log(body));
+```
+Sample result:
+```
+{
+ "date":1564591511232,
+ "app":"AndroidAPS",
+ "device":"Samsung XCover 4-861536030196001",
+ "eventType":"Correction Bolus",
+ "insulin":0.3,
+ "identifier":"95e1a6e3-1146-5d6a-a3f1-41567cae0895",
+ "utcOffset":0,
+ "created_at":"2019-07-31T16:45:11.232Z",
+ "srvModified":1564591627732,
+ "srvCreated":1564591511711,
+ "subject":"test-admin"
+}
+```
+
+
+---
+### LAST MODIFIED
+
+[LAST MODIFIED](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/other/LAST-MODIFIED) operation finds the date of last modification for each collection.
+
+Sample GET `/lastModified` client code (to get latest modification dates):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+
+request(`https://nsapiv3.herokuapp.com/api/v3/lastModified?${auth}`,
+ (error, response, body) => console.log(body));
+```
+Sample result:
+```javascript
+{
+ "srvDate":1564591783202,
+ "collections":{
+ "devicestatus":1564591490074,
+ "entries":1564591486801,
+ "profile":1548524042744,
+ "treatments":1564591627732
+ }
+}
+```
+
+
+---
+### UPDATE
+
+[UPDATE](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/generic/put__collection___identifier_) operation updates existing document in the collection.
+
+Sample PUT `/treatments/{identifier}` client code (to update `insulin` from 0.3 to 0.4):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const identifier = '95e1a6e3-1146-5d6a-a3f1-41567cae0895';
+const doc = {
+ date: 1564591511232,
+ app: 'AndroidAPS',
+ device: 'Samsung XCover 4-861536030196001',
+ eventType: 'Correction Bolus',
+ insulin: 0.4
+};
+
+request({
+ method: 'put',
+ body: doc,
+ json: true,
+ url: `https://nsapiv3.herokuapp.com/api/v3/treatments/${identifier}?${auth}`
+ },
+ (error, response, body) => console.log(response.statusCode));
+```
+Sample result:
+```
+204
+```
+
+
+---
+### PATCH
+
+[PATCH](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/generic/patch__collection___identifier_) operation partially updates existing document in the collection.
+
+Sample PATCH `/treatments/{identifier}` client code (to update `insulin` from 0.4 to 0.5):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const identifier = '95e1a6e3-1146-5d6a-a3f1-41567cae0895';
+const doc = {
+ insulin: 0.5
+};
+
+request({
+ method: 'patch',
+ body: doc,
+ json: true,
+ url: `https://nsapiv3.herokuapp.com/api/v3/treatments/${identifier}?${auth}`
+ },
+ (error, response, body) => console.log(response.statusCode));
+```
+Sample result:
+```
+204
+```
+
+
+---
+### DELETE
+
+[DELETE](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/generic/delete__collection___identifier_) operation deletes existing document from the collection.
+
+Sample DELETE `/treatments/{identifier}` client code (to update `insulin` from 0.4 to 0.5):
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const identifier = '95e1a6e3-1146-5d6a-a3f1-41567cae0895';
+
+request({
+ method: 'delete',
+ url: `https://nsapiv3.herokuapp.com/api/v3/treatments/${identifier}?${auth}`
+ },
+ (error, response, body) => console.log(response.statusCode));
+```
+Sample result:
+```
+204
+```
+
+
+---
+### HISTORY
+
+[HISTORY](https://nsapiv3insecure.herokuapp.com/api/v3/swagger-ui-dist/index.html#/generic/HISTORY2) operation queries all changes since the timestamp.
+
+Sample HISTORY `/treatments/history/{lastModified}` client code:
+```javascript
+const request = require('request');
+const auth = `token=testadmin-ad3b1f9d7b3f59d5&now=${new Date().getTime()}`;
+const lastModified = 1564521267421;
+
+request(`https://nsapiv3.herokuapp.com/api/v3/treatments/history/${lastModified}?${auth}`,
+ (error, response, body) => console.log(response.body));
+```
+Sample result:
+```
+[
+ {
+ "date":1564521267421,
+ "app":"AndroidAPS",
+ "device":"Samsung XCover 4-861536030196001",
+ "eventType":"Correction Bolus",
+ "insulin":0.5,
+ "utcOffset":0,
+ "created_at":"2019-07-30T21:14:27.421Z",
+ "identifier":"95e1a6e3-1146-5d6a-a3f1-41567cae0895",
+ "srvModified":1564592440416,
+ "srvCreated":1564592334853,
+ "subject":"test-admin",
+ "modifiedBy":"test-admin",
+ "isValid":false
+ },
+ {
+ "date":1564592545299,
+ "app":"AndroidAPS",
+ "device":"Samsung XCover 4-861536030196001",
+ "eventType":"Snack Bolus",
+ "carbs":10,
+ "identifier":"267c43c2-f629-5191-a542-4f410c69e486",
+ "utcOffset":0,
+ "created_at":"2019-07-31T17:02:25.299Z",
+ "srvModified":1564592545781,
+ "srvCreated":1564592545781,
+ "subject":"test-admin"
+ }
+]
+```
+Notice the `"isValid":false` field marking the deletion of the document.
\ No newline at end of file
diff --git a/lib/api3/generic/collection.js b/lib/api3/generic/collection.js
new file mode 100644
index 00000000000..0a1a29b3915
--- /dev/null
+++ b/lib/api3/generic/collection.js
@@ -0,0 +1,193 @@
+'use strict';
+
+const apiConst = require('../const.json')
+ , _ = require('lodash')
+ , dateTools = require('../shared/dateTools')
+ , opTools = require('../shared/operationTools')
+ , stringTools = require('../shared/stringTools')
+ , CollectionStorage = require('../storage/mongoCollection')
+ , searchOperation = require('./search/operation')
+ , createOperation = require('./create/operation')
+ , readOperation = require('./read/operation')
+ , updateOperation = require('./update/operation')
+ , patchOperation = require('./patch/operation')
+ , deleteOperation = require('./delete/operation')
+ , historyOperation = require('./history/operation')
+ ;
+
+/**
+ * Generic collection (abstraction over each collection specifics)
+ * @param {string} colName - name of the collection inside the storage system
+ * @param {function} fallbackGetDate - function that tries to create srvModified virtually from other fields of document
+ * @param {Array} dedupFallbackFields - fields that all need to be matched to identify document via fallback deduplication
+ * @param {function} fallbackHistoryFilter - function that creates storage filter for all newer records (than the timestamp from first function parameter)
+ */
+function Collection ({ ctx, env, app, colName, storageColName, fallbackGetDate, dedupFallbackFields,
+ fallbackDateField }) {
+
+ const self = this;
+
+ self.colName = colName;
+ self.fallbackGetDate = fallbackGetDate;
+ self.dedupFallbackFields = app.get('API3_DEDUP_FALLBACK_ENABLED') ? dedupFallbackFields : [];
+ self.autoPruneDays = app.setENVTruthy('API3_AUTOPRUNE_' + colName.toUpperCase());
+ self.nextAutoPrune = new Date();
+ self.storage = new CollectionStorage(ctx, env, storageColName);
+ self.fallbackDateField = fallbackDateField;
+
+ self.mapRoutes = function mapRoutes () {
+ const prefix = '/' + colName
+ , prefixId = prefix + '/:identifier'
+ , prefixHistory = prefix + '/history'
+ ;
+
+
+ // GET /{collection}
+ app.get(prefix, searchOperation(ctx, env, app, self));
+
+ // POST /{collection}
+ app.post(prefix, createOperation(ctx, env, app, self));
+
+ // GET /{collection}/history
+ app.get(prefixHistory, historyOperation(ctx, env, app, self));
+
+ // GET /{collection}/history
+ app.get(prefixHistory + '/:lastModified', historyOperation(ctx, env, app, self));
+
+ // GET /{collection}/{identifier}
+ app.get(prefixId, readOperation(ctx, env, app, self));
+
+ // PUT /{collection}/{identifier}
+ app.put(prefixId, updateOperation(ctx, env, app, self));
+
+ // PATCH /{collection}/{identifier}
+ app.patch(prefixId, patchOperation(ctx, env, app, self));
+
+ // DELETE /{collection}/{identifier}
+ app.delete(prefixId, deleteOperation(ctx, env, app, self));
+ };
+
+
+ /**
+ * Parse limit (max document count) from query string
+ */
+ self.parseLimit = function parseLimit (req, res) {
+ const maxLimit = app.get('API3_MAX_LIMIT');
+ let limit = maxLimit;
+
+ if (req.query.limit) {
+ if (!isNaN(req.query.limit) && req.query.limit > 0 && req.query.limit <= maxLimit) {
+ limit = parseInt(req.query.limit);
+ }
+ else {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_LIMIT);
+ return null;
+ }
+ }
+
+ return limit;
+ };
+
+
+
+ /**
+ * Fetch modified date from document (with possible fallback and back-fill to srvModified/srvCreated)
+ * @param {Object} doc - document loaded from database
+ */
+ self.resolveDates = function resolveDates (doc) {
+ let modifiedDate;
+ try {
+ if (doc.srvModified) {
+ modifiedDate = new Date(doc.srvModified);
+ }
+ else {
+ if (typeof (self.fallbackGetDate) === 'function') {
+ modifiedDate = self.fallbackGetDate(doc);
+ if (modifiedDate) {
+ doc.srvModified = modifiedDate.getTime();
+ }
+ }
+ }
+
+ if (doc.srvModified && !doc.srvCreated) {
+ doc.srvCreated = modifiedDate.getTime();
+ }
+ }
+ catch (error) {
+ console.warn(error);
+ }
+ return modifiedDate;
+ };
+
+
+ /**
+ * Deletes old documents from the collection if enabled (for this collection)
+ * in the background (asynchronously)
+ * */
+ self.autoPrune = function autoPrune () {
+
+ if (!stringTools.isNumberInString(self.autoPruneDays))
+ return;
+
+ const autoPruneDays = parseFloat(self.autoPruneDays);
+ if (autoPruneDays <= 0)
+ return;
+
+ if (new Date() > self.nextAutoPrune) {
+
+ const deleteBefore = new Date(new Date().getTime() - (autoPruneDays * 24 * 3600 * 1000));
+
+ const filter = [
+ { field: 'srvCreated', operator: 'lt', value: deleteBefore.getTime() },
+ { field: 'created_at', operator: 'lt', value: deleteBefore.toISOString() },
+ { field: 'date', operator: 'lt', value: deleteBefore.getTime() }
+ ];
+
+ // let's autoprune asynchronously (we won't wait for the result)
+ self.storage.deleteManyOr(filter, function deleteDone (err, result) {
+ if (err || !result) {
+ console.error(err);
+ }
+
+ if (result.deleted) {
+ console.info('Auto-pruned ' + result.deleted + ' documents from ' + self.colName + ' collection ');
+ }
+ });
+
+ self.nextAutoPrune = new Date(new Date().getTime() + (3600 * 1000));
+ }
+ };
+
+
+ /**
+ * Parse date and utcOffset + optional created_at fallback
+ * @param {Object} doc
+ */
+ self.parseDate = function parseDate (doc) {
+ if (!_.isEmpty(doc)) {
+
+ let values = app.get('API3_CREATED_AT_FALLBACK_ENABLED')
+ ? [doc.date, doc.created_at]
+ : [doc.date];
+
+ let m = dateTools.parseToMoment(values);
+ if (m && m.isValid()) {
+ doc.date = m.valueOf();
+
+ if (typeof doc.utcOffset === 'undefined') {
+ doc.utcOffset = m.utcOffset();
+ }
+
+ if (app.get('API3_CREATED_AT_FALLBACK_ENABLED')) {
+ doc.created_at = m.toISOString();
+ }
+ else {
+ if (doc.created_at)
+ delete doc.created_at;
+ }
+ }
+ }
+ }
+}
+
+module.exports = Collection;
\ No newline at end of file
diff --git a/lib/api3/generic/create/insert.js b/lib/api3/generic/create/insert.js
new file mode 100644
index 00000000000..4ac80a37e94
--- /dev/null
+++ b/lib/api3/generic/create/insert.js
@@ -0,0 +1,45 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , security = require('../../security')
+ , validate = require('./validate.js')
+ ;
+
+/**
+ * Insert new document into the collection
+ * @param {Object} opCtx
+ * @param {Object} doc
+ */
+async function insert (opCtx, doc) {
+
+ const { ctx, auth, col, req, res } = opCtx;
+
+ await security.demandPermission(opCtx, `api:${col.colName}:create`);
+
+ if (validate(opCtx, doc) !== true)
+ return;
+
+ const now = new Date;
+ doc.srvModified = now.getTime();
+ doc.srvCreated = doc.srvModified;
+
+ if (auth && auth.subject && auth.subject.name) {
+ doc.subject = auth.subject.name;
+ }
+
+ const identifier = await col.storage.insertOne(doc);
+
+ if (!identifier)
+ throw new Error('empty identifier');
+
+ res.setHeader('Last-Modified', now.toUTCString());
+ res.setHeader('Location', `${req.baseUrl}${req.path}/${identifier}`);
+ res.status(apiConst.HTTP.CREATED).send({ });
+
+ ctx.bus.emit('storage-socket-create', { colName: col.colName, doc });
+ col.autoPrune();
+ ctx.bus.emit('data-received');
+}
+
+
+module.exports = insert;
\ No newline at end of file
diff --git a/lib/api3/generic/create/operation.js b/lib/api3/generic/create/operation.js
new file mode 100644
index 00000000000..39986a87ebd
--- /dev/null
+++ b/lib/api3/generic/create/operation.js
@@ -0,0 +1,63 @@
+'use strict';
+
+const _ = require('lodash')
+ , apiConst = require('../../const.json')
+ , security = require('../../security')
+ , insert = require('./insert')
+ , replace = require('../update/replace')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+
+/**
+ * CREATE: Inserts a new document into the collection
+ */
+async function create (opCtx) {
+
+ const { col, req, res } = opCtx;
+ const doc = req.body;
+
+ if (_.isEmpty(doc)) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_REQUEST_BODY);
+ }
+
+ col.parseDate(doc);
+ opTools.resolveIdentifier(doc);
+ const identifyingFilter = col.storage.identifyingFilter(doc.identifier, doc, col.dedupFallbackFields);
+
+ const result = await col.storage.findOneFilter(identifyingFilter, { });
+
+ if (!result)
+ throw new Error('empty result');
+
+ if (result.length > 0) {
+ const storageDoc = result[0];
+ await replace(opCtx, doc, storageDoc, { isDeduplication: true });
+ }
+ else {
+ await insert(opCtx, doc);
+ }
+}
+
+
+function createOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await create(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = createOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/create/validate.js b/lib/api3/generic/create/validate.js
new file mode 100644
index 00000000000..e978a3955e5
--- /dev/null
+++ b/lib/api3/generic/create/validate.js
@@ -0,0 +1,26 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , stringTools = require('../../shared/stringTools')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+
+/**
+ * Validation of document to create
+ * @param {Object} opCtx
+ * @param {Object} doc
+ * @returns string with error message if validation fails, true in case of success
+ */
+function validate (opCtx, doc) {
+
+ const { res } = opCtx;
+
+ if (typeof(doc.identifier) !== 'string' || stringTools.isNullOrWhitespace(doc.identifier)) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_FIELD_IDENTIFIER);
+ }
+
+ return opTools.validateCommon(doc, res);
+}
+
+module.exports = validate;
\ No newline at end of file
diff --git a/lib/api3/generic/delete/operation.js b/lib/api3/generic/delete/operation.js
new file mode 100644
index 00000000000..535ca1a7620
--- /dev/null
+++ b/lib/api3/generic/delete/operation.js
@@ -0,0 +1,93 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , security = require('../../security')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+/**
+ * DELETE: Deletes a document from the collection
+ */
+async function doDelete (opCtx) {
+
+ const { col, req } = opCtx;
+
+ await security.demandPermission(opCtx, `api:${col.colName}:delete`);
+
+ if (req.query.permanent && req.query.permanent === "true") {
+ await deletePermanently(opCtx);
+ } else {
+ await markAsDeleted(opCtx);
+ }
+}
+
+
+async function deletePermanently (opCtx) {
+
+ const { ctx, col, req, res } = opCtx;
+
+ const identifier = req.params.identifier;
+ const result = await col.storage.deleteOne(identifier);
+
+ if (!result)
+ throw new Error('empty result');
+
+ if (!result.deleted) {
+ return res.status(apiConst.HTTP.NOT_FOUND).end();
+ }
+
+ col.autoPrune();
+ ctx.bus.emit('storage-socket-delete', { colName: col.colName, identifier });
+ ctx.bus.emit('data-received');
+ return res.status(apiConst.HTTP.NO_CONTENT).end();
+}
+
+
+async function markAsDeleted (opCtx) {
+
+ const { ctx, col, req, res, auth } = opCtx;
+
+ const identifier = req.params.identifier;
+ const setFields = { 'isValid': false, 'srvModified': (new Date).getTime() };
+
+ if (auth && auth.subject && auth.subject.name) {
+ setFields.modifiedBy = auth.subject.name;
+ }
+
+ const result = await col.storage.updateOne(identifier, setFields);
+
+ if (!result)
+ throw new Error('empty result');
+
+ if (!result.updated) {
+ return res.status(apiConst.HTTP.NOT_FOUND).end();
+ }
+
+ ctx.bus.emit('storage-socket-delete', { colName: col.colName, identifier });
+ col.autoPrune();
+ ctx.bus.emit('data-received');
+ return res.status(apiConst.HTTP.NO_CONTENT).end();
+}
+
+
+function deleteOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await doDelete(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = deleteOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/history/operation.js b/lib/api3/generic/history/operation.js
new file mode 100644
index 00000000000..0929a09cc4f
--- /dev/null
+++ b/lib/api3/generic/history/operation.js
@@ -0,0 +1,151 @@
+'use strict';
+
+const dateTools = require('../../shared/dateTools')
+ , apiConst = require('../../const.json')
+ , security = require('../../security')
+ , opTools = require('../../shared/operationTools')
+ , FieldsProjector = require('../../shared/fieldsProjector')
+ , _ = require('lodash')
+ ;
+
+/**
+ * HISTORY: Retrieves incremental changes since timestamp
+ */
+async function history (opCtx, fieldsProjector) {
+
+ const { req, res, col } = opCtx;
+
+ let filter = parseFilter(opCtx)
+ , limit = col.parseLimit(req, res)
+ , projection = fieldsProjector.storageProjection()
+ , sort = prepareSort()
+ , skip = 0
+ , onlyValid = false
+ , logicalOperator = 'or'
+ ;
+
+ if (filter !== null && limit !== null && projection !== null) {
+
+ const result = await col.storage.findMany(filter
+ , sort
+ , limit
+ , skip
+ , projection
+ , onlyValid
+ , logicalOperator);
+
+ if (!result)
+ throw new Error('empty result');
+
+ if (result.length === 0) {
+ return res.status(apiConst.HTTP.NO_CONTENT).end();
+ }
+
+ _.each(result, col.resolveDates);
+
+ const srvModifiedValues = _.map(result, function mapSrvModified (item) {
+ return item.srvModified;
+ })
+ , maxSrvModified = _.max(srvModifiedValues);
+
+ res.setHeader('Last-Modified', (new Date(maxSrvModified)).toUTCString());
+ res.setHeader('ETag', 'W/"' + maxSrvModified + '"');
+
+ _.each(result, fieldsProjector.applyProjection);
+
+ res.status(apiConst.HTTP.OK).send(result);
+ }
+}
+
+
+/**
+ * Parse history filtering criteria from Last-Modified header
+ */
+function parseFilter (opCtx) {
+
+ const { req, res } = opCtx;
+
+ let lastModified = null
+ , lastModifiedParam = req.params.lastModified
+ , operator = null;
+
+ if (lastModifiedParam) {
+
+ // using param in URL as a source of timestamp
+ const m = dateTools.parseToMoment(lastModifiedParam);
+
+ if (m === null || !m.isValid()) {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_LAST_MODIFIED);
+ return null;
+ }
+
+ lastModified = m.toDate();
+ operator = 'gt';
+ }
+ else {
+ // using request HTTP header as a source of timestamp
+ const lastModifiedHeader = req.get('Last-Modified');
+ if (!lastModifiedHeader) {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_LAST_MODIFIED);
+ return null;
+ }
+
+ try {
+ lastModified = dateTools.floorSeconds(new Date(lastModifiedHeader));
+ } catch (err) {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_LAST_MODIFIED);
+ return null;
+ }
+ operator = 'gte';
+ }
+
+ return [
+ { field: 'srvModified', operator: operator, value: lastModified.getTime() },
+ { field: 'created_at', operator: operator, value: lastModified.toISOString() },
+ { field: 'date', operator: operator, value: lastModified.getTime() }
+ ];
+}
+
+
+
+/**
+ * Prepare sorting for storage query
+ */
+function prepareSort () {
+ return {
+ srvModified: 1,
+ created_at: 1,
+ date: 1
+ };
+}
+
+
+function historyOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ if (col.colName === 'settings') {
+ await security.demandPermission(opCtx, `api:${col.colName}:admin`);
+ } else {
+ await security.demandPermission(opCtx, `api:${col.colName}:read`);
+ }
+
+ const fieldsProjector = new FieldsProjector(req.query.fields);
+
+ await history(opCtx, fieldsProjector);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = historyOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/patch/operation.js b/lib/api3/generic/patch/operation.js
new file mode 100644
index 00000000000..d7bb5fc2b4d
--- /dev/null
+++ b/lib/api3/generic/patch/operation.js
@@ -0,0 +1,118 @@
+'use strict';
+
+const _ = require('lodash')
+ , apiConst = require('../../const.json')
+ , security = require('../../security')
+ , validate = require('./validate.js')
+ , opTools = require('../../shared/operationTools')
+ , dateTools = require('../../shared/dateTools')
+ , FieldsProjector = require('../../shared/fieldsProjector')
+ ;
+
+/**
+ * PATCH: Partially updates document in the collection
+ */
+async function patch (opCtx) {
+
+ const { req, res, col } = opCtx;
+ const doc = req.body;
+
+ if (_.isEmpty(doc)) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_REQUEST_BODY);
+ }
+
+ await security.demandPermission(opCtx, `api:${col.colName}:update`);
+
+ col.parseDate(doc);
+ const identifier = req.params.identifier
+ , identifyingFilter = col.storage.identifyingFilter(identifier);
+
+ const result = await col.storage.findOneFilter(identifyingFilter, { });
+
+ if (!result)
+ throw new Error('result empty');
+
+ if (result.length > 0) {
+
+ const storageDoc = result[0];
+ if (storageDoc.isValid === false) {
+ return res.status(apiConst.HTTP.GONE).end();
+ }
+
+ const modifiedDate = col.resolveDates(storageDoc)
+ , ifUnmodifiedSince = req.get('If-Unmodified-Since');
+
+ if (ifUnmodifiedSince
+ && dateTools.floorSeconds(modifiedDate) > dateTools.floorSeconds(new Date(ifUnmodifiedSince))) {
+ return res.status(apiConst.HTTP.PRECONDITION_FAILED).end();
+ }
+
+ await applyPatch(opCtx, identifier, doc, storageDoc);
+ }
+ else {
+ return res.status(apiConst.HTTP.NOT_FOUND).end();
+ }
+}
+
+
+/**
+ * Patch existing document in the collection
+ * @param {Object} opCtx
+ * @param {string} identifier
+ * @param {Object} doc - fields and values to patch
+ * @param {Object} storageDoc - original (database) version of document
+ */
+async function applyPatch (opCtx, identifier, doc, storageDoc) {
+
+ const { ctx, res, col, auth } = opCtx;
+
+ if (validate(opCtx, doc, storageDoc) !== true)
+ return;
+
+ const now = new Date;
+ doc.srvModified = now.getTime();
+
+ if (auth && auth.subject && auth.subject.name) {
+ doc.modifiedBy = auth.subject.name;
+ }
+
+ const matchedCount = await col.storage.updateOne(identifier, doc);
+
+ if (!matchedCount)
+ throw new Error('matchedCount empty');
+
+ res.setHeader('Last-Modified', now.toUTCString());
+ res.status(apiConst.HTTP.NO_CONTENT).send({ });
+
+ const fieldsProjector = new FieldsProjector('_all');
+ const patchedDocs = await col.storage.findOne(identifier, fieldsProjector);
+ const patchedDoc = patchedDocs[0];
+ fieldsProjector.applyProjection(patchedDoc);
+ ctx.bus.emit('storage-socket-update', { colName: col.colName, doc: patchedDoc });
+
+ col.autoPrune();
+ ctx.bus.emit('data-received');
+}
+
+
+function patchOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await patch(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = patchOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/patch/validate.js b/lib/api3/generic/patch/validate.js
new file mode 100644
index 00000000000..057bb5c39e8
--- /dev/null
+++ b/lib/api3/generic/patch/validate.js
@@ -0,0 +1,19 @@
+'use strict';
+
+const updateValidate = require('../update/validate')
+ ;
+
+
+/**
+ * Validate document to patch
+ * @param {Object} opCtx
+ * @param {Object} doc
+ * @param {Object} storageDoc
+ * @returns string - null if validation fails
+ */
+function validate (opCtx, doc, storageDoc) {
+
+ return updateValidate(opCtx, doc, storageDoc, { isPatching: true });
+}
+
+module.exports = validate;
\ No newline at end of file
diff --git a/lib/api3/generic/read/operation.js b/lib/api3/generic/read/operation.js
new file mode 100644
index 00000000000..04d6f03bc70
--- /dev/null
+++ b/lib/api3/generic/read/operation.js
@@ -0,0 +1,75 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , security = require('../../security')
+ , opTools = require('../../shared/operationTools')
+ , dateTools = require('../../shared/dateTools')
+ , FieldsProjector = require('../../shared/fieldsProjector')
+ ;
+
+/**
+ * READ: Retrieves a single document from the collection
+ */
+async function read (opCtx) {
+
+ const { req, res, col } = opCtx;
+
+ await security.demandPermission(opCtx, `api:${col.colName}:read`);
+
+ const fieldsProjector = new FieldsProjector(req.query.fields);
+
+ const result = await col.storage.findOne(req.params.identifier
+ , fieldsProjector.storageProjection());
+
+ if (!result)
+ throw new Error('empty result');
+
+ if (result.length === 0) {
+ return res.status(apiConst.HTTP.NOT_FOUND).end();
+ }
+
+ const doc = result[0];
+ if (doc.isValid === false) {
+ return res.status(apiConst.HTTP.GONE).end();
+ }
+
+
+ const modifiedDate = col.resolveDates(doc);
+ if (modifiedDate) {
+ res.setHeader('Last-Modified', modifiedDate.toUTCString());
+
+ const ifModifiedSince = req.get('If-Modified-Since');
+
+ if (ifModifiedSince
+ && dateTools.floorSeconds(modifiedDate) <= dateTools.floorSeconds(new Date(ifModifiedSince))) {
+ return res.status(apiConst.HTTP.NOT_MODIFIED).end();
+ }
+ }
+
+ fieldsProjector.applyProjection(doc);
+
+ res.status(apiConst.HTTP.OK).send(doc);
+}
+
+
+function readOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await read(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = readOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/search/input.js b/lib/api3/generic/search/input.js
new file mode 100644
index 00000000000..dbd37356760
--- /dev/null
+++ b/lib/api3/generic/search/input.js
@@ -0,0 +1,140 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , dateTools = require('../../shared/dateTools')
+ , stringTools = require('../../shared/stringTools')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+const filterRegex = /(.*)\$([a-zA-Z]+)/;
+
+
+/**
+ * Parse value of the parameter (to the correct data type)
+ */
+function parseValue(param, value) {
+
+ value = stringTools.isNumberInString(value) ? parseFloat(value) : value; // convert number from string
+
+ // convert boolean from string
+ if (value === 'true')
+ value = true;
+
+ if (value === 'false')
+ value = false;
+
+ // unwrap string in single quotes
+ if (typeof(value) === 'string' && value.startsWith('\'') && value.endsWith('\'')) {
+ value = value.substr(1, value.length - 2);
+ }
+
+ if (['date', 'srvModified', 'srvCreated'].includes(param)) {
+ let m = dateTools.parseToMoment(value);
+ if (m && m.isValid()) {
+ value = m.valueOf();
+ }
+ }
+
+ if (param === 'created_at') {
+ let m = dateTools.parseToMoment(value);
+ if (m && m.isValid()) {
+ value = m.toISOString();
+ }
+ }
+
+ return value;
+}
+
+
+/**
+ * Parse filtering criteria from query string
+ */
+function parseFilter (req, res) {
+ const filter = []
+ , reservedParams = ['token', 'sort', 'sort$desc', 'limit', 'skip', 'fields', 'now']
+ , operators = ['eq', 'ne', 'gt', 'gte', 'lt', 'lte', 'in', 'nin', 're']
+ ;
+
+ for (let param in req.query) {
+ if (!Object.prototype.hasOwnProperty.call(req.query, param)
+ || reservedParams.includes(param)) continue;
+
+ let field = param
+ , operator = 'eq'
+ ;
+
+ const match = filterRegex.exec(param);
+ if (match != null) {
+ operator = match[2];
+ field = match[1];
+
+ if (!operators.includes(operator)) {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST,
+ apiConst.MSG.HTTP_400_UNSUPPORTED_FILTER_OPERATOR.replace('{0}', operator));
+ return null;
+ }
+ }
+ const value = parseValue(field, req.query[param]);
+
+ filter.push({ field, operator, value });
+ }
+
+ return filter;
+}
+
+
+/**
+ * Parse sorting from query string
+ */
+function parseSort (req, res) {
+ let sort = {}
+ , sortDirection = 1;
+
+ if (req.query.sort && req.query.sort$desc) {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_SORT_SORT_DESC);
+ return null;
+ }
+
+ if (req.query.sort$desc) {
+ sortDirection = -1;
+ sort[req.query.sort$desc] = sortDirection;
+ }
+ else {
+ if (req.query.sort) {
+ sort[req.query.sort] = sortDirection;
+ }
+ }
+
+ sort.identifier = sortDirection;
+ sort.created_at = sortDirection;
+ sort.date = sortDirection;
+
+ return sort;
+}
+
+
+/**
+ * Parse skip (offset) from query string
+ */
+function parseSkip (req, res) {
+ let skip = 0;
+
+ if (req.query.skip) {
+ if (!isNaN(req.query.skip) && req.query.skip >= 0) {
+ skip = parseInt(req.query.skip);
+ }
+ else {
+ opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_SKIP);
+ return null;
+ }
+ }
+
+ return skip;
+}
+
+
+module.exports = {
+ parseFilter,
+ parseSort,
+ parseSkip
+};
\ No newline at end of file
diff --git a/lib/api3/generic/search/operation.js b/lib/api3/generic/search/operation.js
new file mode 100644
index 00000000000..074f864d58a
--- /dev/null
+++ b/lib/api3/generic/search/operation.js
@@ -0,0 +1,77 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , security = require('../../security')
+ , opTools = require('../../shared/operationTools')
+ , input = require('./input')
+ , _each = require('lodash/each')
+ , FieldsProjector = require('../../shared/fieldsProjector')
+ ;
+
+
+/**
+ * SEARCH: Search documents from the collection
+ */
+async function search (opCtx) {
+
+ const { req, res, col } = opCtx;
+
+ if (col.colName === 'settings') {
+ await security.demandPermission(opCtx, `api:${col.colName}:admin`);
+ } else {
+ await security.demandPermission(opCtx, `api:${col.colName}:read`);
+ }
+
+ const fieldsProjector = new FieldsProjector(req.query.fields);
+
+ const filter = input.parseFilter(req, res)
+ , sort = input.parseSort(req, res)
+ , limit = col.parseLimit(req, res)
+ , skip = input.parseSkip(req, res)
+ , projection = fieldsProjector.storageProjection()
+ , onlyValid = true
+ ;
+
+
+ if (filter !== null && sort !== null && limit !== null && skip !== null && projection !== null) {
+
+ const result = await col.storage.findMany(filter
+ , sort
+ , limit
+ , skip
+ , projection
+ , onlyValid);
+
+ if (!result)
+ throw new Error('empty result');
+
+ _each(result, col.resolveDates);
+
+ _each(result, fieldsProjector.applyProjection);
+
+ res.status(apiConst.HTTP.OK).send(result);
+ }
+}
+
+
+function searchOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await search(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = searchOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/setup.js b/lib/api3/generic/setup.js
new file mode 100644
index 00000000000..17e118658dd
--- /dev/null
+++ b/lib/api3/generic/setup.js
@@ -0,0 +1,103 @@
+'use strict';
+
+const _ = require('lodash')
+ , dateTools = require('../shared/dateTools')
+ , Collection = require('./collection')
+ ;
+
+
+function fallbackDate (doc) {
+ const m = dateTools.parseToMoment(doc.date);
+ return m == null || !m.isValid()
+ ? null
+ : m.toDate();
+}
+
+
+function fallbackCreatedAt (doc) {
+ const m = dateTools.parseToMoment(doc.created_at);
+ return m == null || !m.isValid()
+ ? null
+ : m.toDate();
+}
+
+
+function setupGenericCollections (ctx, env, app) {
+ const cols = { }
+ , enabledCols = app.get('enabledCollections');
+
+ if (_.includes(enabledCols, 'devicestatus')) {
+ cols.devicestatus = new Collection({
+ ctx, env, app,
+ colName: 'devicestatus',
+ storageColName: env.devicestatus_collection || 'devicestatus',
+ fallbackGetDate: fallbackCreatedAt,
+ dedupFallbackFields: ['created_at', 'device'],
+ fallbackDateField: 'created_at'
+ });
+ }
+
+ const entriesCollection = new Collection({
+ ctx, env, app,
+ colName: 'entries',
+ storageColName: env.entries_collection || 'entries',
+ fallbackGetDate: fallbackDate,
+ dedupFallbackFields: ['date', 'type'],
+ fallbackDateField: 'date'
+ });
+ app.set('entriesCollection', entriesCollection);
+
+ if (_.includes(enabledCols, 'entries')) {
+ cols.entries = entriesCollection;
+ }
+
+ if (_.includes(enabledCols, 'food')) {
+ cols.food = new Collection({
+ ctx, env, app,
+ colName: 'food',
+ storageColName: env.food_collection || 'food',
+ fallbackGetDate: fallbackCreatedAt,
+ dedupFallbackFields: ['created_at'],
+ fallbackDateField: 'created_at'
+ });
+ }
+
+ if (_.includes(enabledCols, 'profile')) {
+ cols.profile = new Collection({
+ ctx, env, app,
+ colName: 'profile',
+ storageColName: env.profile_collection || 'profile',
+ fallbackGetDate: fallbackCreatedAt,
+ dedupFallbackFields: ['created_at'],
+ fallbackDateField: 'created_at'
+ });
+ }
+
+ if (_.includes(enabledCols, 'settings')) {
+ cols.settings = new Collection({
+ ctx, env, app,
+ colName: 'settings',
+ storageColName: env.settings_collection || 'settings'
+ });
+ }
+
+ if (_.includes(enabledCols, 'treatments')) {
+ cols.treatments = new Collection({
+ ctx, env, app,
+ colName: 'treatments',
+ storageColName: env.treatments_collection || 'treatments',
+ fallbackGetDate: fallbackCreatedAt,
+ dedupFallbackFields: ['created_at', 'eventType'],
+ fallbackDateField: 'created_at'
+ });
+ }
+
+ _.forOwn(cols, function forMember (col) {
+ col.mapRoutes();
+ });
+
+ app.set('collections', cols);
+}
+
+
+module.exports = setupGenericCollections;
diff --git a/lib/api3/generic/update/operation.js b/lib/api3/generic/update/operation.js
new file mode 100644
index 00000000000..3e517a32d11
--- /dev/null
+++ b/lib/api3/generic/update/operation.js
@@ -0,0 +1,86 @@
+'use strict';
+
+const _ = require('lodash')
+ , dateTools = require('../../shared/dateTools')
+ , apiConst = require('../../const.json')
+ , security = require('../../security')
+ , insert = require('../create/insert')
+ , replace = require('./replace')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+/**
+ * UPDATE: Updates a document in the collection
+ */
+async function update (opCtx) {
+
+ const { col, req, res } = opCtx;
+ const doc = req.body;
+
+ if (_.isEmpty(doc)) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_REQUEST_BODY);
+ }
+
+ col.parseDate(doc);
+ opTools.resolveIdentifier(doc);
+
+ const identifier = req.params.identifier
+ , identifyingFilter = col.storage.identifyingFilter(identifier);
+
+ const result = await col.storage.findOneFilter(identifyingFilter, { });
+
+ if (!result)
+ throw new Error('empty result');
+
+ doc.identifier = identifier;
+
+ if (result.length > 0) {
+ await updateConditional(opCtx, doc, result[0]);
+ }
+ else {
+ await insert(opCtx, doc);
+ }
+}
+
+
+async function updateConditional (opCtx, doc, storageDoc) {
+
+ const { col, req, res } = opCtx;
+
+ if (storageDoc.isValid === false) {
+ return res.status(apiConst.HTTP.GONE).end();
+ }
+
+ const modifiedDate = col.resolveDates(storageDoc)
+ , ifUnmodifiedSince = req.get('If-Unmodified-Since');
+
+ if (ifUnmodifiedSince
+ && dateTools.floorSeconds(modifiedDate) > dateTools.floorSeconds(new Date(ifUnmodifiedSince))) {
+ return res.status(apiConst.HTTP.PRECONDITION_FAILED).end();
+ }
+
+ await replace(opCtx, doc, storageDoc);
+}
+
+
+function updateOperation (ctx, env, app, col) {
+
+ return async function operation (req, res) {
+
+ const opCtx = { app, ctx, env, col, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await update(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ };
+}
+
+module.exports = updateOperation;
\ No newline at end of file
diff --git a/lib/api3/generic/update/replace.js b/lib/api3/generic/update/replace.js
new file mode 100644
index 00000000000..ca490b31136
--- /dev/null
+++ b/lib/api3/generic/update/replace.js
@@ -0,0 +1,52 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , security = require('../../security')
+ , validate = require('./validate.js')
+ ;
+
+/**
+ * Replace existing document in the collection
+ * @param {Object} opCtx
+ * @param {any} doc - new version of document to set
+ * @param {any} storageDoc - old version of document (existing in the storage)
+ * @param {Object} options
+ */
+async function replace (opCtx, doc, storageDoc, options) {
+
+ const { ctx, auth, col, req, res } = opCtx;
+ const { isDeduplication } = options || {};
+
+ await security.demandPermission(opCtx, `api:${col.colName}:update`);
+
+ if (validate(opCtx, doc, storageDoc, { isDeduplication }) !== true)
+ return;
+
+ const now = new Date;
+ doc.srvModified = now.getTime();
+ doc.srvCreated = storageDoc.srvCreated || doc.srvModified;
+
+ if (auth && auth.subject && auth.subject.name) {
+ doc.subject = auth.subject.name;
+ }
+
+ const matchedCount = await col.storage.replaceOne(storageDoc.identifier, doc);
+
+ if (!matchedCount)
+ throw new Error('empty matchedCount');
+
+ res.setHeader('Last-Modified', now.toUTCString());
+
+ if (storageDoc.identifier !== doc.identifier || isDeduplication) {
+ res.setHeader('Location', `${req.baseUrl}${req.path}/${doc.identifier}`);
+ }
+
+ res.status(apiConst.HTTP.NO_CONTENT).send({ });
+
+ ctx.bus.emit('storage-socket-update', { colName: col.colName, doc });
+ col.autoPrune();
+ ctx.bus.emit('data-received');
+}
+
+
+module.exports = replace;
\ No newline at end of file
diff --git a/lib/api3/generic/update/validate.js b/lib/api3/generic/update/validate.js
new file mode 100644
index 00000000000..e68eb2955f8
--- /dev/null
+++ b/lib/api3/generic/update/validate.js
@@ -0,0 +1,43 @@
+'use strict';
+
+const apiConst = require('../../const.json')
+ , opTools = require('../../shared/operationTools')
+ ;
+
+
+/**
+ * Validation of document to update
+ * @param {Object} opCtx
+ * @param {Object} doc
+ * @param {Object} storageDoc
+ * @param {Object} options
+ * @returns string with error message if validation fails, true in case of success
+ */
+function validate (opCtx, doc, storageDoc, options) {
+
+ const { res } = opCtx;
+ const { isPatching, isDeduplication } = options || {};
+
+ const immutable = ['identifier', 'date', 'utcOffset', 'eventType', 'device', 'app',
+ 'srvCreated', 'subject', 'srvModified', 'modifiedBy', 'isValid'];
+
+ for (const field of immutable) {
+
+ // change of identifier is allowed in deduplication (for APIv1 documents)
+ if (field === 'identifier' && isDeduplication)
+ continue;
+
+ // changing deleted document is without restrictions
+ if (storageDoc.isValid === false)
+ continue;
+
+ if (typeof(doc[field]) !== 'undefined' && doc[field] !== storageDoc[field]) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST,
+ apiConst.MSG.HTTP_400_IMMUTABLE_FIELD.replace('{0}', field));
+ }
+ }
+
+ return opTools.validateCommon(doc, res, { isPatching });
+}
+
+module.exports = validate;
\ No newline at end of file
diff --git a/lib/api3/index.js b/lib/api3/index.js
new file mode 100644
index 00000000000..70fb6a4d59e
--- /dev/null
+++ b/lib/api3/index.js
@@ -0,0 +1,106 @@
+'use strict';
+
+const express = require('express')
+ , bodyParser = require('body-parser')
+ , StorageSocket = require('./storageSocket')
+ , apiConst = require('./const.json')
+ , security = require('./security')
+ , genericSetup = require('./generic/setup')
+ , swaggerSetup = require('./swagger')
+ ;
+
+function configure (env, ctx) {
+
+ const self = { }
+ , app = express()
+ ;
+
+ self.setENVTruthy = function setENVTruthy (varName, defaultValue) {
+ //for some reason Azure uses this prefix, maybe there is a good reason
+ let value = process.env['CUSTOMCONNSTR_' + varName]
+ || process.env['CUSTOMCONNSTR_' + varName.toLowerCase()]
+ || process.env[varName]
+ || process.env[varName.toLowerCase()];
+
+ value = value != null ? value : defaultValue;
+
+ if (typeof value === 'string' && (value.toLowerCase() === 'on' || value.toLowerCase() === 'true')) { value = true; }
+ if (typeof value === 'string' && (value.toLowerCase() === 'off' || value.toLowerCase() === 'false')) { value = false; }
+
+ app.set(varName, value);
+ return value;
+ };
+ app.setENVTruthy = self.setENVTruthy;
+
+
+ self.setupApiEnvironment = function setupApiEnvironment () {
+
+ app.use(bodyParser.json({
+ limit: 1048576 * 50
+ }), function errorHandler (err, req, res, next) {
+ console.error(err);
+ res.status(500).json({
+ status: 500,
+ message: apiConst.MSG.HTTP_500_INTERNAL_ERROR
+ });
+ if (next) { // we need 4th parameter next to behave like error handler, but we have to use it to prevent "unused variable" message
+ }
+ });
+
+ // we don't need these here
+ app.set('etag', false);
+ app.set('x-powered-by', false); // this seems to be unreliable
+ app.use(function (req, res, next) {
+ res.removeHeader('x-powered-by');
+ next();
+ });
+
+ app.set('name', env.name);
+ app.set('version', env.version);
+ app.set('apiVersion', apiConst.API3_VERSION);
+ app.set('units', env.DISPLAY_UNITS);
+ app.set('enabledCollections', ['devicestatus', 'entries', 'food', 'profile', 'settings', 'treatments']);
+
+ self.setENVTruthy('API3_SECURITY_ENABLE', apiConst.API3_SECURITY_ENABLE);
+ self.setENVTruthy('API3_TIME_SKEW_TOLERANCE', apiConst.API3_TIME_SKEW_TOLERANCE);
+ self.setENVTruthy('API3_DEDUP_FALLBACK_ENABLED', apiConst.API3_DEDUP_FALLBACK_ENABLED);
+ self.setENVTruthy('API3_CREATED_AT_FALLBACK_ENABLED', apiConst.API3_CREATED_AT_FALLBACK_ENABLED);
+ self.setENVTruthy('API3_MAX_LIMIT', apiConst.API3_MAX_LIMIT);
+ };
+
+
+ self.setupApiRoutes = function setupApiRoutes () {
+
+ app.get('/version', require('./specific/version')(app, ctx, env));
+
+ if (app.get('env') === 'development') { // for development and testing purposes only
+ app.get('/test', async function test (req, res) {
+
+ try {
+ const opCtx = {app, ctx, env, req, res};
+ opCtx.auth = await security.authenticate(opCtx);
+ await security.demandPermission(opCtx, 'api:entries:read');
+ res.status(200).end();
+ } catch (error) {
+ console.error(error);
+ }
+ });
+ }
+
+ app.get('/lastModified', require('./specific/lastModified')(app, ctx, env));
+
+ app.get('/status', require('./specific/status')(app, ctx, env));
+ };
+
+
+ self.setupApiEnvironment();
+ genericSetup(ctx, env, app);
+ self.setupApiRoutes();
+ swaggerSetup(app);
+
+ ctx.storageSocket = new StorageSocket(app, env, ctx);
+
+ return app;
+}
+
+module.exports = configure;
diff --git a/lib/api3/security.js b/lib/api3/security.js
new file mode 100644
index 00000000000..33099d88f12
--- /dev/null
+++ b/lib/api3/security.js
@@ -0,0 +1,122 @@
+'use strict';
+
+const moment = require('moment')
+ , apiConst = require('./const.json')
+ , _ = require('lodash')
+ , shiroTrie = require('shiro-trie')
+ , dateTools = require('./shared/dateTools')
+ , opTools = require('./shared/operationTools')
+ ;
+
+
+/**
+ * Check if Date header in HTTP request (or 'now' query parameter) is present and valid (with error response sending)
+ */
+function checkDateHeader (opCtx) {
+
+ const { app, req, res } = opCtx;
+
+ let dateString = req.header('Date');
+ if (!dateString) {
+ dateString = req.query.now;
+ }
+
+ if (!dateString) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.UNAUTHORIZED, apiConst.MSG.HTTP_401_MISSING_DATE);
+ }
+
+ let dateMoment = dateTools.parseToMoment(dateString);
+ if (!dateMoment) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.UNAUTHORIZED, apiConst.MSG.HTTP_401_BAD_DATE);
+ }
+
+ let nowMoment = moment(new Date());
+ let diffMinutes = moment.duration(nowMoment.diff(dateMoment)).asMinutes();
+
+ if (Math.abs(diffMinutes) > app.get('API3_TIME_SKEW_TOLERANCE')) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.UNAUTHORIZED, apiConst.MSG.HTTP_401_DATE_OUT_OF_TOLERANCE);
+ }
+
+ return true;
+}
+
+
+function authenticate (opCtx) {
+ return new Promise(function promise (resolve, reject) {
+
+ let { app, ctx, req, res } = opCtx;
+
+ if (!app.get('API3_SECURITY_ENABLE')) {
+ const adminShiro = shiroTrie.new();
+ adminShiro.add('*');
+ return resolve({ shiros: [ adminShiro ] });
+ }
+
+ if (req.protocol !== 'https') {
+ return reject(
+ opTools.sendJSONStatus(res, apiConst.HTTP.FORBIDDEN, apiConst.MSG.HTTP_403_NOT_USING_HTTPS));
+ }
+
+ const checkDateResult = checkDateHeader(opCtx);
+ if (checkDateResult !== true) {
+ return checkDateResult;
+ }
+
+ let token = ctx.authorization.extractToken(req);
+ if (!token) {
+ return reject(
+ opTools.sendJSONStatus(res, apiConst.HTTP.UNAUTHORIZED, apiConst.MSG.HTTP_401_MISSING_OR_BAD_TOKEN));
+ }
+
+ ctx.authorization.resolve({ token }, function resolveFinish (err, result) {
+ if (err) {
+ return reject(
+ opTools.sendJSONStatus(res, apiConst.HTTP.UNAUTHORIZED, apiConst.MSG.HTTP_401_BAD_TOKEN));
+ }
+ else {
+ return resolve(result);
+ }
+ });
+ });
+}
+
+
+/**
+ * Checks for the permission from the authorization without error response sending
+ * @param {any} auth
+ * @param {any} permission
+ */
+function checkPermission (auth, permission) {
+
+ if (auth) {
+ const found = _.find(auth.shiros, function checkEach (shiro) {
+ return shiro && shiro.check(permission);
+ });
+ return _.isObject(found);
+ }
+ else {
+ return false;
+ }
+}
+
+
+
+function demandPermission (opCtx, permission) {
+ return new Promise(function promise (resolve, reject) {
+ const { auth, res } = opCtx;
+
+ if (checkPermission(auth, permission)) {
+ return resolve(true);
+ } else {
+ return reject(
+ opTools.sendJSONStatus(res, apiConst.HTTP.FORBIDDEN, apiConst.MSG.HTTP_403_MISSING_PERMISSION.replace('{0}', permission)));
+ }
+ });
+}
+
+
+module.exports = {
+ authenticate,
+ checkPermission,
+ demandPermission
+};
\ No newline at end of file
diff --git a/lib/api3/shared/dateTools.js b/lib/api3/shared/dateTools.js
new file mode 100644
index 00000000000..14b67f9e109
--- /dev/null
+++ b/lib/api3/shared/dateTools.js
@@ -0,0 +1,78 @@
+'use strict';
+
+const moment = require('moment')
+ , stringTools = require('./stringTools')
+ , apiConst = require('../const.json')
+ ;
+
+
+/**
+ * Floor date to whole seconds (cut off milliseconds)
+ * @param {Date} date
+ */
+function floorSeconds (date) {
+ let ms = date.getTime();
+ ms -= ms % 1000;
+ return new Date(ms);
+}
+
+
+/**
+ * Parse date as moment object from value or array of values.
+ * @param {any} value
+ */
+function parseToMoment (value)
+{
+ if (!value)
+ return null;
+
+ if (Array.isArray(value)) {
+ for (let item of value) {
+ let m = parseToMoment(item);
+
+ if (m !== null)
+ return m;
+ }
+ }
+ else {
+
+ if (typeof value === 'string' && stringTools.isNumberInString(value)) {
+ value = parseFloat(value);
+ }
+
+ if (typeof value === 'number') {
+ let m = moment(value);
+
+ if (!m.isValid())
+ return null;
+
+ if (m.valueOf() < apiConst.MIN_TIMESTAMP)
+ m = moment.unix(m);
+
+ if (!m.isValid() || m.valueOf() < apiConst.MIN_TIMESTAMP)
+ return null;
+
+ return m;
+ }
+
+ if (typeof value === 'string') {
+ let m = moment.parseZone(value, moment.ISO_8601);
+
+ if (!m.isValid())
+ m = moment.parseZone(value, moment.RFC_2822);
+
+ if (!m.isValid() || m.valueOf() < apiConst.MIN_TIMESTAMP)
+ return null;
+
+ return m;
+ }
+ }
+
+ // no parsing option succeeded => failure
+ return null;
+}
+
+module.exports = {
+ floorSeconds,
+ parseToMoment
+};
diff --git a/lib/api3/shared/fieldsProjector.js b/lib/api3/shared/fieldsProjector.js
new file mode 100644
index 00000000000..921c7cc6df8
--- /dev/null
+++ b/lib/api3/shared/fieldsProjector.js
@@ -0,0 +1,82 @@
+'use strict';
+
+const _each = require('lodash/each');
+
+/**
+ * Decoder of 'fields' parameter providing storage projections
+ * @param {string} fieldsString - fields parameter from user
+ */
+function FieldsProjector (fieldsString) {
+
+ const self = this
+ , exclude = [];
+ let specific = null;
+
+ switch (fieldsString)
+ {
+ case '_all':
+ break;
+
+ default:
+ if (fieldsString) {
+ specific = fieldsString.split(',');
+ }
+ }
+
+ const systemFields = ['identifier', 'srvCreated', 'created_at', 'date'];
+
+ /**
+ * Prepare projection definition for storage query
+ * */
+ self.storageProjection = function storageProjection () {
+ const projection = { };
+
+ if (specific) {
+ _each(specific, function include (field) {
+ projection[field] = 1;
+ });
+
+ _each(systemFields, function include (field) {
+ projection[field] = 1;
+ });
+ }
+ else {
+ _each(exclude, function exclude (field) {
+ projection[field] = 0;
+ });
+
+ _each(exclude, function exclude (field) {
+ if (systemFields.indexOf(field) >= 0) {
+ delete projection[field];
+ }
+ });
+ }
+
+ return projection;
+ };
+
+
+ /**
+ * Cut off unwanted fields from given document
+ * @param {Object} doc
+ */
+ self.applyProjection = function applyProjection (doc) {
+
+ if (specific) {
+ for(const field in doc) {
+ if (specific.indexOf(field) === -1) {
+ delete doc[field];
+ }
+ }
+ }
+ else {
+ _each(exclude, function include (field) {
+ if (typeof(doc[field]) !== 'undefined') {
+ delete doc[field];
+ }
+ });
+ }
+ };
+}
+
+module.exports = FieldsProjector;
\ No newline at end of file
diff --git a/lib/api3/shared/operationTools.js b/lib/api3/shared/operationTools.js
new file mode 100644
index 00000000000..1955b9c2068
--- /dev/null
+++ b/lib/api3/shared/operationTools.js
@@ -0,0 +1,111 @@
+'use strict';
+
+const apiConst = require('../const.json')
+ , stringTools = require('./stringTools')
+ , uuidv5 = require('uuid/v5')
+ , uuidNamespace = [...Buffer.from("NightscoutRocks!", "ascii")] // official namespace for NS :-)
+ ;
+
+function sendJSONStatus (res, status, title, description, warning) {
+
+ const json = {
+ status: status,
+ message: title,
+ description: description
+ };
+
+ // Add optional warning message.
+ if (warning) { json.warning = warning; }
+
+ res.status(status).json(json);
+
+ return title;
+}
+
+
+/**
+ * Validate document's common fields
+ * @param {Object} doc
+ * @param {any} res
+ * @param {Object} options
+ * @returns {any} - string with error message if validation fails, true in case of success
+ */
+function validateCommon (doc, res, options) {
+
+ const { isPatching } = options || {};
+
+
+ if ((!isPatching || typeof(doc.date) !== 'undefined')
+
+ && (typeof(doc.date) !== 'number'
+ || doc.date <= apiConst.MIN_TIMESTAMP)
+ ) {
+ return sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_FIELD_DATE);
+ }
+
+
+ if ((!isPatching || typeof(doc.utcOffset) !== 'undefined')
+
+ && (typeof(doc.utcOffset) !== 'number'
+ || doc.utcOffset < apiConst.MIN_UTC_OFFSET
+ || doc.utcOffset > apiConst.MAX_UTC_OFFSET)
+ ) {
+ return sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_FIELD_UTC);
+ }
+
+
+ if ((!isPatching || typeof(doc.app) !== 'undefined')
+
+ && (typeof(doc.app) !== 'string'
+ || stringTools.isNullOrWhitespace(doc.app))
+ ) {
+ return sendJSONStatus(res, apiConst.HTTP.BAD_REQUEST, apiConst.MSG.HTTP_400_BAD_FIELD_APP);
+ }
+
+ return true;
+}
+
+
+/**
+ * Calculate identifier for the document
+ * @param {Object} doc
+ * @returns string
+ */
+function calculateIdentifier (doc) {
+ if (!doc)
+ return undefined;
+
+ let key = doc.device + '_' + doc.date;
+ if (doc.eventType) {
+ key += '_' + doc.eventType;
+ }
+
+ return uuidv5(key, uuidNamespace);
+}
+
+
+/**
+ * Validate identifier in the document
+ * @param {Object} doc
+ */
+function resolveIdentifier (doc) {
+
+ let identifier = calculateIdentifier(doc);
+ if (doc.identifier) {
+ if (doc.identifier !== identifier) {
+ console.warn(`APIv3: Identifier mismatch (expected: ${identifier}, received: ${doc.identifier})`);
+ console.log(doc);
+ }
+ }
+ else {
+ doc.identifier = identifier;
+ }
+}
+
+
+module.exports = {
+ sendJSONStatus,
+ validateCommon,
+ calculateIdentifier,
+ resolveIdentifier
+};
\ No newline at end of file
diff --git a/lib/api3/shared/storageTools.js b/lib/api3/shared/storageTools.js
new file mode 100644
index 00000000000..b7d9dca6776
--- /dev/null
+++ b/lib/api3/shared/storageTools.js
@@ -0,0 +1,63 @@
+'use strict';
+
+function getStorageVersion (app) {
+
+ return new Promise(function (resolve, reject) {
+
+ try {
+ const storage = app.get('entriesCollection').storage;
+ let storageVersion = app.get('storageVersion');
+
+ if (storageVersion) {
+ process.nextTick(() => {
+ resolve(storageVersion);
+ });
+ } else {
+ storage.version()
+ .then(storageVersion => {
+
+ app.set('storageVersion', storageVersion);
+ resolve(storageVersion);
+ }, reject);
+ }
+ } catch (error) {
+ reject(error);
+ }
+ });
+}
+
+
+function getVersionInfo(app) {
+
+ return new Promise(function (resolve, reject) {
+
+ try {
+ const srvDate = new Date()
+ , info = { version: app.get('version')
+ , apiVersion: app.get('apiVersion')
+ , srvDate: srvDate.getTime()
+ };
+
+ getStorageVersion(app)
+ .then(storageVersion => {
+
+ if (!storageVersion)
+ throw new Error('empty storageVersion');
+
+ info.storage = storageVersion;
+
+ resolve(info);
+
+ }, reject);
+
+ } catch(error) {
+ reject(error);
+ }
+ });
+}
+
+
+module.exports = {
+ getStorageVersion,
+ getVersionInfo
+};
diff --git a/lib/api3/shared/stringTools.js b/lib/api3/shared/stringTools.js
new file mode 100644
index 00000000000..b71a4b4f1a6
--- /dev/null
+++ b/lib/api3/shared/stringTools.js
@@ -0,0 +1,28 @@
+'use strict';
+
+/**
+ * Check the string for strictly valid number (no other characters present)
+ * @param {any} str
+ */
+function isNumberInString (str) {
+ return !isNaN(parseFloat(str)) && isFinite(str);
+}
+
+
+/**
+ * Check the string for non-whitespace characters presence
+ * @param {any} input
+ */
+function isNullOrWhitespace (input) {
+
+ if (typeof input === 'undefined' || input == null) return true;
+
+ return input.replace(/\s/g, '').length < 1;
+}
+
+
+
+module.exports = {
+ isNumberInString,
+ isNullOrWhitespace
+};
diff --git a/lib/api3/specific/lastModified.js b/lib/api3/specific/lastModified.js
new file mode 100644
index 00000000000..b27ecaca852
--- /dev/null
+++ b/lib/api3/specific/lastModified.js
@@ -0,0 +1,101 @@
+'use strict';
+
+function configure (app, ctx, env) {
+ const express = require('express')
+ , api = express.Router( )
+ , apiConst = require('../const.json')
+ , security = require('../security')
+ , opTools = require('../shared/operationTools')
+ ;
+
+ api.get('/lastModified', async function getLastModified (req, res) {
+
+ async function getLastModified (col) {
+
+ let result;
+ const lastModified = await col.storage.getLastModified('srvModified');
+
+ if (lastModified) {
+ result = lastModified.srvModified ? lastModified.srvModified : null;
+ }
+
+ if (col.fallbackDateField) {
+
+ const lastModified = await col.storage.getLastModified(col.fallbackDateField);
+
+ if (lastModified && lastModified[col.fallbackDateField]) {
+ let timestamp = lastModified[col.fallbackDateField];
+ if (typeof(timestamp) === 'string') {
+ timestamp = (new Date(timestamp)).getTime();
+ }
+
+ if (result === null || result < timestamp) {
+ result = timestamp;
+ }
+ }
+ }
+
+ return { colName: col.colName, lastModified: result };
+ }
+
+
+ async function collectionsAsync (auth) {
+
+ const cols = app.get('collections')
+ , promises = []
+ , output = {}
+ ;
+
+ for (const colName in cols) {
+ const col = cols[colName];
+
+ if (security.checkPermission(auth, 'api:' + col.colName + ':read')) {
+ promises.push(getLastModified(col));
+ }
+ }
+
+ const results = await Promise.all(promises);
+
+ for (const result of results) {
+ if (result.lastModified)
+ output[result.colName] = result.lastModified;
+ }
+
+ return output;
+ }
+
+
+ async function operation (opCtx) {
+
+ const { res, auth } = opCtx;
+ const srvDate = new Date();
+
+ let info = {
+ srvDate: srvDate.getTime(),
+ collections: { }
+ };
+
+ info.collections = await collectionsAsync(auth);
+
+ res.json(info);
+ }
+
+
+ const opCtx = { app, ctx, env, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await operation(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ });
+
+ return api;
+}
+module.exports = configure;
diff --git a/lib/api3/specific/status.js b/lib/api3/specific/status.js
new file mode 100644
index 00000000000..7b70b24ab71
--- /dev/null
+++ b/lib/api3/specific/status.js
@@ -0,0 +1,71 @@
+'use strict';
+
+function configure (app, ctx, env) {
+ const express = require('express')
+ , api = express.Router( )
+ , apiConst = require('../const.json')
+ , storageTools = require('../shared/storageTools')
+ , security = require('../security')
+ , opTools = require('../shared/operationTools')
+ ;
+
+ api.get('/status', async function getStatus (req, res) {
+
+ function permsForCol (col, auth) {
+ let colPerms = '';
+
+ if (security.checkPermission(auth, 'api:' + col.colName + ':create')) {
+ colPerms += 'c';
+ }
+
+ if (security.checkPermission(auth, 'api:' + col.colName + ':read')) {
+ colPerms += 'r';
+ }
+
+ if (security.checkPermission(auth, 'api:' + col.colName + ':update')) {
+ colPerms += 'u';
+ }
+
+ if (security.checkPermission(auth, 'api:' + col.colName + ':delete')) {
+ colPerms += 'd';
+ }
+
+ return colPerms;
+ }
+
+
+ async function operation (opCtx) {
+ const cols = app.get('collections');
+
+ let info = await storageTools.getVersionInfo(app);
+
+ info.apiPermissions = {};
+ for (let col in cols) {
+ const colPerms = permsForCol(col, opCtx.auth);
+ if (colPerms) {
+ info.apiPermissions[col] = colPerms;
+ }
+ }
+
+ res.json(info);
+ }
+
+
+ const opCtx = { app, ctx, env, req, res };
+
+ try {
+ opCtx.auth = await security.authenticate(opCtx);
+
+ await operation(opCtx);
+
+ } catch (err) {
+ console.error(err);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ });
+
+ return api;
+}
+module.exports = configure;
diff --git a/lib/api3/specific/version.js b/lib/api3/specific/version.js
new file mode 100644
index 00000000000..25392fe99d7
--- /dev/null
+++ b/lib/api3/specific/version.js
@@ -0,0 +1,28 @@
+'use strict';
+
+function configure (app) {
+ const express = require('express')
+ , api = express.Router( )
+ , apiConst = require('../const.json')
+ , storageTools = require('../shared/storageTools')
+ , opTools = require('../shared/operationTools')
+ ;
+
+ api.get('/version', async function getVersion (req, res) {
+
+ try {
+ const versionInfo = await storageTools.getVersionInfo(app);
+
+ res.json(versionInfo);
+
+ } catch(error) {
+ console.error(error);
+ if (!res.headersSent) {
+ return opTools.sendJSONStatus(res, apiConst.HTTP.INTERNAL_ERROR, apiConst.MSG.STORAGE_ERROR);
+ }
+ }
+ });
+
+ return api;
+}
+module.exports = configure;
diff --git a/lib/api3/storage/mongoCollection/find.js b/lib/api3/storage/mongoCollection/find.js
new file mode 100644
index 00000000000..bc399dbce98
--- /dev/null
+++ b/lib/api3/storage/mongoCollection/find.js
@@ -0,0 +1,93 @@
+'use strict';
+
+const utils = require('./utils')
+ , _ = require('lodash')
+ ;
+
+
+/**
+ * Find single document by identifier
+ * @param {Object} col
+ * @param {string} identifier
+ * @param {Object} projection
+ */
+function findOne (col, identifier, projection) {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.filterForOne(identifier);
+
+ col.find(filter)
+ .project(projection)
+ .sort({ identifier: -1 }) // document with identifier first (not the fallback one)
+ .toArray(function mongoDone (err, result) {
+
+ if (err) {
+ reject(err);
+ } else {
+ _.each(result, utils.normalizeDoc);
+ resolve(result);
+ }
+ });
+ });
+}
+
+
+/**
+ * Find single document by query filter
+ * @param {Object} col
+ * @param {Object} filter specific filter
+ * @param {Object} projection
+ */
+function findOneFilter (col, filter, projection) {
+
+ return new Promise(function (resolve, reject) {
+
+ col.find(filter)
+ .project(projection)
+ .sort({ identifier: -1 }) // document with identifier first (not the fallback one)
+ .toArray(function mongoDone (err, result) {
+
+ if (err) {
+ reject(err);
+ } else {
+ _.each(result, utils.normalizeDoc);
+ resolve(result);
+ }
+ });
+ });
+}
+
+
+/**
+ * Find many documents matching the filtering criteria
+ */
+function findMany (col, filterDef, sort, limit, skip, projection, onlyValid, logicalOperator = 'and') {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.parseFilter(filterDef, logicalOperator, onlyValid);
+
+ col.find(filter)
+ .sort(sort)
+ .limit(limit)
+ .skip(skip)
+ .project(projection)
+ .toArray(function mongoDone (err, result) {
+
+ if (err) {
+ reject(err);
+ } else {
+ _.each(result, utils.normalizeDoc);
+ resolve(result);
+ }
+ });
+ });
+}
+
+
+module.exports = {
+ findOne,
+ findOneFilter,
+ findMany
+};
\ No newline at end of file
diff --git a/lib/api3/storage/mongoCollection/index.js b/lib/api3/storage/mongoCollection/index.js
new file mode 100644
index 00000000000..e6ad0a6cf8b
--- /dev/null
+++ b/lib/api3/storage/mongoCollection/index.js
@@ -0,0 +1,90 @@
+'use strict';
+
+/**
+ * Storage implementation using mongoDB
+ * @param {Object} ctx
+ * @param {Object} env
+ * @param {string} colName - name of the collection in mongo database
+ */
+function MongoCollection (ctx, env, colName) {
+
+ const self = this
+ , utils = require('./utils')
+ , find = require('./find')
+ , modify = require('./modify')
+ ;
+
+ self.colName = colName;
+
+ self.col = ctx.store.collection(colName);
+
+ ctx.store.ensureIndexes(self.col, [ 'identifier',
+ 'srvModified',
+ 'isValid'
+ ]);
+
+
+ self.identifyingFilter = utils.identifyingFilter;
+
+ self.findOne = (...args) => find.findOne(self.col, ...args);
+
+ self.findOneFilter = (...args) => find.findOneFilter(self.col, ...args);
+
+ self.findMany = (...args) => find.findMany(self.col, ...args);
+
+ self.insertOne = (...args) => modify.insertOne(self.col, ...args);
+
+ self.replaceOne = (...args) => modify.replaceOne(self.col, ...args);
+
+ self.updateOne = (...args) => modify.updateOne(self.col, ...args);
+
+ self.deleteOne = (...args) => modify.deleteOne(self.col, ...args);
+
+ self.deleteManyOr = (...args) => modify.deleteManyOr(self.col, ...args);
+
+
+ /**
+ * Get server version
+ */
+ self.version = function version () {
+
+ return new Promise(function (resolve, reject) {
+
+ ctx.store.db.admin().buildInfo({}, function mongoDone (err, result) {
+
+ err
+ ? reject(err)
+ : resolve({
+ storage: 'mongodb',
+ version: result.version
+ });
+ });
+ });
+ };
+
+
+ /**
+ * Get timestamp (e.g. srvModified) of the last modified document
+ */
+ self.getLastModified = function getLastModified (fieldName) {
+
+ return new Promise(function (resolve, reject) {
+
+ self.col.find()
+
+ .sort({ [fieldName]: -1 })
+
+ .limit(1)
+
+ .project({ [fieldName]: 1 })
+
+ .toArray(function mongoDone (err, [ result ]) {
+ err
+ ? reject(err)
+ : resolve(result);
+ });
+ });
+ }
+}
+
+module.exports = MongoCollection;
\ No newline at end of file
diff --git a/lib/api3/storage/mongoCollection/modify.js b/lib/api3/storage/mongoCollection/modify.js
new file mode 100644
index 00000000000..6552fe40e8c
--- /dev/null
+++ b/lib/api3/storage/mongoCollection/modify.js
@@ -0,0 +1,123 @@
+'use strict';
+
+const utils = require('./utils');
+
+
+/**
+ * Insert single document
+ * @param {Object} col
+ * @param {Object} doc
+ */
+function insertOne (col, doc) {
+
+ return new Promise(function (resolve, reject) {
+
+ col.insertOne(doc, function mongoDone(err, result) {
+
+ if (err) {
+ reject(err);
+ } else {
+ const identifier = doc.identifier || result.insertedId.toString();
+ delete doc._id;
+ resolve(identifier);
+ }
+ });
+ });
+}
+
+
+/**
+ * Replace single document
+ * @param {Object} col
+ * @param {string} identifier
+ * @param {Object} doc
+ */
+function replaceOne (col, identifier, doc) {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.filterForOne(identifier);
+
+ col.replaceOne(filter, doc, { }, function mongoDone(err, result) {
+ if (err) {
+ reject(err);
+ } else {
+ resolve(result.matchedCount);
+ }
+ });
+ });
+}
+
+
+/**
+ * Update single document by identifier
+ * @param {Object} col
+ * @param {string} identifier
+ * @param {object} setFields
+ */
+function updateOne (col, identifier, setFields) {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.filterForOne(identifier);
+
+ col.updateOne(filter, { $set: setFields }, function mongoDone(err, result) {
+ if (err) {
+ reject(err);
+ } else {
+ resolve({ updated: result.result.nModified });
+ }
+ });
+ });
+}
+
+
+/**
+ * Permanently remove single document by identifier
+ * @param {Object} col
+ * @param {string} identifier
+ */
+function deleteOne (col, identifier) {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.filterForOne(identifier);
+
+ col.deleteOne(filter, function mongoDone(err, result) {
+ if (err) {
+ reject(err);
+ } else {
+ resolve({ deleted: result.result.n });
+ }
+ });
+ });
+}
+
+
+/**
+ * Permanently remove many documents matching any of filtering criteria
+ */
+function deleteManyOr (col, filterDef) {
+
+ return new Promise(function (resolve, reject) {
+
+ const filter = utils.parseFilter(filterDef, 'or');
+
+ col.deleteMany(filter, function mongoDone(err, result) {
+ if (err) {
+ reject(err);
+ } else {
+ resolve({ deleted: result.deletedCount });
+ }
+ });
+ });
+}
+
+
+module.exports = {
+ insertOne,
+ replaceOne,
+ updateOne,
+ deleteOne,
+ deleteManyOr
+};
\ No newline at end of file
diff --git a/lib/api3/storage/mongoCollection/utils.js b/lib/api3/storage/mongoCollection/utils.js
new file mode 100644
index 00000000000..1b2ab5610d7
--- /dev/null
+++ b/lib/api3/storage/mongoCollection/utils.js
@@ -0,0 +1,178 @@
+'use strict';
+
+const _ = require('lodash')
+ , checkForHexRegExp = new RegExp("^[0-9a-fA-F]{24}$")
+ , ObjectID = require('mongodb').ObjectID
+;
+
+
+/**
+ * Normalize document (make it mongoDB independent)
+ * @param {Object} doc - document loaded from mongoDB
+ */
+function normalizeDoc (doc) {
+ if (!doc.identifier) {
+ doc.identifier = doc._id.toString();
+ }
+
+ delete doc._id;
+}
+
+
+/**
+ * Parse filter definition array into mongoDB filtering object
+ * @param {any} filterDef
+ * @param {string} logicalOperator
+ * @param {bool} onlyValid
+ */
+function parseFilter (filterDef, logicalOperator, onlyValid) {
+
+ let filter = { };
+ if (!filterDef)
+ return filter;
+
+ if (!_.isArray(filterDef)) {
+ return filterDef;
+ }
+
+ let clauses = [];
+
+ for (const itemDef of filterDef) {
+ let item;
+
+ switch (itemDef.operator) {
+ case 'eq':
+ item = itemDef.value;
+ break;
+
+ case 'ne':
+ item = { $ne: itemDef.value };
+ break;
+
+ case 'gt':
+ item = { $gt: itemDef.value };
+ break;
+
+ case 'gte':
+ item = { $gte: itemDef.value };
+ break;
+
+ case 'lt':
+ item = { $lt: itemDef.value };
+ break;
+
+ case 'lte':
+ item = { $lte: itemDef.value };
+ break;
+
+ case 'in':
+ item = { $in: itemDef.value.toString().split('|') };
+ break;
+
+ case 'nin':
+ item = { $nin: itemDef.value.toString().split('|') };
+ break;
+
+ case 're':
+ item = { $regex: itemDef.value.toString() };
+ break;
+
+ default:
+ throw new Error('Unsupported or missing filter operator ' + itemDef.operator);
+ }
+
+ if (logicalOperator === 'or') {
+ let clause = { };
+ clause[itemDef.field] = item;
+ clauses.push(clause);
+ }
+ else {
+ filter[itemDef.field] = item;
+ }
+ }
+
+ if (logicalOperator === 'or') {
+ filter = { $or: clauses };
+ }
+
+ if (onlyValid) {
+ filter.isValid = { $ne: false };
+ }
+
+ return filter;
+}
+
+
+/**
+ * Create query filter for single document with identifier fallback
+ * @param {string} identifier
+ */
+function filterForOne (identifier) {
+
+ const filterOpts = [ { identifier } ];
+
+ // fallback to "identifier = _id"
+ if (checkForHexRegExp.test(identifier)) {
+ filterOpts.push({ _id: ObjectID(identifier) });
+ }
+
+ return { $or: filterOpts };
+}
+
+
+/**
+ * Create query filter to check whether the document already exists in the storage.
+ * This function resolves eventual fallback deduplication.
+ * @param {string} identifier - identifier of document to check its existence in the storage
+ * @param {Object} doc - document to check its existence in the storage
+ * @param {Array} dedupFallbackFields - fields that all need to be matched to identify document via fallback deduplication
+ * @returns {Object} - query filter for mongo or null in case of no identifying possibility
+ */
+function identifyingFilter (identifier, doc, dedupFallbackFields) {
+
+ const filterItems = [];
+
+ if (identifier) {
+ // standard identifier field (APIv3)
+ filterItems.push({ identifier: identifier });
+
+ // fallback to "identifier = _id" (APIv1)
+ if (checkForHexRegExp.test(identifier)) {
+ filterItems.push({ identifier: { $exists: false }, _id: ObjectID(identifier) });
+ }
+ }
+
+ // let's deal with eventual fallback deduplication
+ if (!_.isEmpty(doc) && _.isArray(dedupFallbackFields) && dedupFallbackFields.length > 0) {
+ let dedupFilterItems = [];
+
+ _.each(dedupFallbackFields, function addDedupField (field) {
+
+ if (doc[field] !== undefined) {
+
+ let dedupFilterItem = { };
+ dedupFilterItem[field] = doc[field];
+ dedupFilterItems.push(dedupFilterItem);
+ }
+ });
+
+ if (dedupFilterItems.length === dedupFallbackFields.length) { // all dedup fields are present
+
+ dedupFilterItems.push({ identifier: { $exists: false } }); // force not existing identifier for fallback deduplication
+ filterItems.push({ $and: dedupFilterItems });
+ }
+ }
+
+ if (filterItems.length > 0)
+ return { $or: filterItems };
+ else
+ return null; // we don't have any filtering rule to identify the document in the storage
+}
+
+
+module.exports = {
+ normalizeDoc,
+ parseFilter,
+ filterForOne,
+ identifyingFilter
+};
\ No newline at end of file
diff --git a/lib/api3/storageSocket.js b/lib/api3/storageSocket.js
new file mode 100644
index 00000000000..e8c08310d2b
--- /dev/null
+++ b/lib/api3/storageSocket.js
@@ -0,0 +1,145 @@
+'use strict';
+
+const apiConst = require('./const');
+
+/**
+ * Socket.IO broadcaster of any storage change
+ */
+function StorageSocket (app, env, ctx) {
+
+ const self = this;
+
+ const LOG_GREEN = '\x1B[32m'
+ , LOG_MAGENTA = '\x1B[35m'
+ , LOG_RESET = '\x1B[0m'
+ , LOG = LOG_GREEN + 'STORAGE SOCKET: ' + LOG_RESET
+ , LOG_ERROR = LOG_MAGENTA + 'STORAGE SOCKET: ' + LOG_RESET
+ , NAMESPACE = '/storage'
+ ;
+
+
+ /**
+ * Initialize socket namespace and bind the events
+ * @param {Object} io Socket.IO object to multiplex namespaces
+ */
+ self.init = function init (io) {
+ self.io = io;
+
+ self.namespace = io.of(NAMESPACE);
+ self.namespace.on('connection', function onConnected (socket) {
+
+ const remoteIP = socket.request.headers['x-forwarded-for'] || socket.request.connection.remoteAddress;
+ console.log(LOG + 'Connection from client ID: ', socket.client.id, ' IP: ', remoteIP);
+
+ socket.on('disconnect', function onDisconnect () {
+ console.log(LOG + 'Disconnected client ID: ', socket.client.id);
+ });
+
+ socket.on('subscribe', function onSubscribe (message, returnCallback) {
+ self.subscribe(socket, message, returnCallback);
+ });
+ });
+
+ ctx.bus.on('storage-socket-create', self.emitCreate);
+ ctx.bus.on('storage-socket-update', self.emitUpdate);
+ ctx.bus.on('storage-socket-delete', self.emitDelete);
+ };
+
+
+ /**
+ * Authorize Socket.IO client and subscribe him to authorized rooms
+ * @param {Object} socket
+ * @param {Object} message input message from the client
+ * @param {Function} returnCallback function for returning a value back to the client
+ */
+ self.subscribe = function subscribe (socket, message, returnCallback) {
+ const shouldCallBack = typeof(returnCallback) === 'function';
+
+ if (message && message.accessToken) {
+ return ctx.authorization.resolveAccessToken(message.accessToken, function resolveFinish (err, auth) {
+ if (err) {
+ console.log(`${LOG_ERROR} Authorization failed for accessToken:`, message.accessToken);
+
+ if (shouldCallBack) {
+ returnCallback({ success: false, message: apiConst.MSG.SOCKET_MISSING_OR_BAD_ACCESS_TOKEN });
+ }
+ return err;
+ }
+ else {
+ return self.subscribeAuthorized(socket, message, auth, returnCallback);
+ }
+ });
+ }
+
+ console.log(`${LOG_ERROR} Authorization failed for message:`, message);
+ if (shouldCallBack) {
+ returnCallback({ success: false, message: apiConst.MSG.SOCKET_MISSING_OR_BAD_ACCESS_TOKEN});
+ }
+ };
+
+
+ /**
+ * Subscribe already authorized Socket.IO client to his rooms
+ * @param {Object} socket
+ * @param {Object} message input message from the client
+ * @param {Object} auth authorization of the client
+ * @param {Function} returnCallback function for returning a value back to the client
+ */
+ self.subscribeAuthorized = function subscribeAuthorized (socket, message, auth, returnCallback) {
+ const shouldCallBack = typeof(returnCallback) === 'function';
+ const enabledCols = app.get('enabledCollections');
+ const cols = Array.isArray(message.collections) ? message.collections : enabledCols;
+ const subscribed = [];
+
+ for (const col of cols) {
+ if (enabledCols.includes(col)) {
+ const permission = (col === 'settings') ? `api:${col}:admin` : `api:${col}:read`;
+
+ if (ctx.authorization.checkMultiple(permission, auth.shiros)) {
+ socket.join(col);
+ subscribed.push(col);
+ }
+ }
+ }
+
+ const doc = subscribed.length > 0
+ ? { success: true, collections: subscribed }
+ : { success: false, message: apiConst.MSG.SOCKET_UNAUTHORIZED_TO_ANY };
+ if (shouldCallBack) {
+ returnCallback(doc);
+ }
+ return doc;
+ };
+
+
+ /**
+ * Emit create event to the subscribers (of the collection's room)
+ * @param {Object} event
+ */
+ self.emitCreate = function emitCreate (event) {
+ self.namespace.to(event.colName)
+ .emit('create', event);
+ };
+
+
+ /**
+ * Emit update event to the subscribers (of the collection's room)
+ * @param {Object} event
+ */
+ self.emitUpdate = function emitUpdate (event) {
+ self.namespace.to(event.colName)
+ .emit('update', event);
+ };
+
+
+ /**
+ * Emit delete event to the subscribers (of the collection's room)
+ * @param {Object} event
+ */
+ self.emitDelete = function emitDelete (event) {
+ self.namespace.to(event.colName)
+ .emit('delete', event);
+ }
+}
+
+module.exports = StorageSocket;
\ No newline at end of file
diff --git a/lib/api3/swagger.js b/lib/api3/swagger.js
new file mode 100644
index 00000000000..2d434e97f53
--- /dev/null
+++ b/lib/api3/swagger.js
@@ -0,0 +1,41 @@
+'use strict';
+
+const express = require('express')
+ , fs = require('fs')
+ ;
+
+
+function setupSwaggerUI (app) {
+
+ const serveSwaggerDef = function serveSwaggerDef (req, res) {
+ res.sendFile(__dirname + '/swagger.yaml');
+ };
+ app.get('/swagger.yaml', serveSwaggerDef);
+
+ const swaggerUiAssetPath = require('swagger-ui-dist').getAbsoluteFSPath();
+ const swaggerFiles = express.static(swaggerUiAssetPath);
+
+ const urlRegex = /url: "[^"]*",/;
+
+ const patchIndex = function patchIndex (req, res) {
+ const indexContent = fs.readFileSync(`${swaggerUiAssetPath}/index.html`)
+ .toString()
+ .replace(urlRegex, 'url: "../swagger.yaml",');
+ res.send(indexContent);
+ };
+
+ app.get('/swagger-ui-dist', function getSwaggerRoot (req, res) {
+ let targetUrl = req.originalUrl;
+ if (!targetUrl.endsWith('/')) {
+ targetUrl += '/';
+ }
+ targetUrl += 'index.html';
+ res.redirect(targetUrl);
+ });
+ app.get('/swagger-ui-dist/index.html', patchIndex);
+
+ app.use('/swagger-ui-dist', swaggerFiles);
+}
+
+
+module.exports = setupSwaggerUI;
\ No newline at end of file
diff --git a/lib/api3/swagger.yaml b/lib/api3/swagger.yaml
new file mode 100644
index 00000000000..32ac1815aee
--- /dev/null
+++ b/lib/api3/swagger.yaml
@@ -0,0 +1,1592 @@
+openapi: 3.0.0
+servers:
+ - url: '/api/v3'
+info:
+ version: '3.0.0'
+ title: Nightscout API
+ contact:
+ name: NS development discussion channel
+ url: https://gitter.im/nightscout/public
+ license:
+ name: AGPL 3
+ url: 'https://www.gnu.org/licenses/agpl.txt'
+ description:
+ Nightscout API v3 is a component of cgm-remote-monitor project. It aims to provide lightweight, secured and HTTP REST compliant interface for your T1D treatment data exchange.
+
+
+ API v3 uses these environment variables, among other things:
+
+ - Security switch (optional, default = `true`)
+ API3_SECURITY_ENABLE=true
+ You can turn the whole security mechanism off, e.g. for debugging or development purposes,
+ but this should never be set to false in production.
+
+
+ - Number of minutes of acceptable time skew between client's and server's clock (optional, default = 5)
+ API3_TIME_SKEW_TOLERANCE=5
+ This security parameter is used for preventing anti-replay attacks, specifically when checking the time from `Date` header.
+
+
+ - Maximum limit count of documents retrieved from single query
+ API3_MAX_LIMIT=1000
+
+
+ - Autopruning of obsolete documents (optional, default is only `DEVICESTATUS`=60)
+ API3_AUTOPRUNE_DEVICESTATUS=60
+
+ API3_AUTOPRUNE_ENTRIES=365
+
+ API3_AUTOPRUNE_TREATMENTS=120
+
+ You can specify for which collections autopruning will be activated and length of retention period in days, e.g. "Hold 60 days of devicestatus, automatically delete older documents, hold 365 days of treatments and entries, automatically delete older documents."
+
+
+ - Fallback deduplication switch (optional, default = true)
+ API3_DEDUP_FALLBACK_ENABLED=true
+ API3 uses the `identifier` field for document identification and mutual distinction within a single collection. There is automatic deduplication implemented matching the equal `identifier` field. E.g. `CREATE` operation for document having the same `identifier` as another one existing in the database is automatically transformed into `UPDATE` operation of the document found in the database.
+
+ Documents not created via API v3 usually does not have any `identifier` field, but we would like to have some form of deduplication for them, too. This fallback deduplication is turned on by having set `API3_DEDUP_FALLBACK_ENABLED` to `true`.
+ When searching the collection in database, the document is found to be a duplicate only when either he has equal `identifier` or he has no `identifier` and meets:
+ `devicestatus` collection: equal combination of `created_at` and `device`
+
+ `entries` collection: equal combination of `date` and `type`
+
+ `food` collection: equal `created_at`
+
+ `profile` collection: equal `created_at`
+
+ `treatments` collection: equal combination of `created_at` and `eventType`
+
+
+
+ - Fallback switch for adding `created_at` field along the `date` field (optional, default = true)
+ API3_CREATED_AT_FALLBACK_ENABLED=true
+ Standard APIv3 document model uses only `date` field for storing a timestamp of the event recorded by the document. But there is a fallback option to fill `created_at` field as well automatically on each insert/update, just to keep all older components working.
+
+tags:
+ - name: generic
+ description: Generic operations with each database collection (devicestatus, entries, food, profile, settings, treatments)
+ - name: other
+ description: All other various operations
+
+
+paths:
+ /{collection}:
+ parameters:
+ - in: path
+ name: collection
+ description: Collection to which the operation is targeted
+ required: true
+ schema:
+ $ref: '#/components/schemas/paramCollection'
+
+ - $ref: '#/components/parameters/dateHeader'
+ - $ref: '#/components/parameters/nowParam'
+ - $ref: '#/components/parameters/tokenParam'
+
+ ######################################################################################
+ get:
+ tags:
+ - generic
+ summary: 'SEARCH: Search documents from the collection'
+ operationId: SEARCH
+ description: General search operation through documents of one collection, matching the specified filtering criteria. You can apply:
+
+
+ 1) filtering - combining any number of filtering parameters
+
+
+ 2) ordering - using `sort` or `sort$desc` parameter
+
+
+ 3) paging - using `limit` and `skip` parameters
+
+
+ When there is no document matching the filtering criteria, HTTP status 204 is returned with empty response content. Otherwise HTTP 200 code is returned with JSON array of matching documents as a response content.
+
+
+ This operation requires `read` permission for the API and the collection (e.g. `*:*:read`, `api:*:read`, `*:treatments:read`, `api:treatments:read`).
+
+
+ The only exception is the `settings` collection which requires `admin` permission (`api:settings:admin`), because the settings of each application should be isolated and kept secret. You need to know the concrete identifier to access the app's settings.
+
+
+ parameters:
+ - $ref: '#/components/parameters/filterParams'
+ - $ref: '#/components/parameters/sortParam'
+ - $ref: '#/components/parameters/sortDescParam'
+ - $ref: '#/components/parameters/limitParam'
+ - $ref: '#/components/parameters/skipParam'
+ - $ref: '#/components/parameters/fieldsParam'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ $ref: '#/components/responses/search200'
+ 204:
+ $ref: '#/components/responses/search204'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+
+
+ ######################################################################################
+ post:
+ tags:
+ - generic
+ summary: 'CREATE: Inserts a new document into the collection'
+ description:
+ Using this operation you can insert new documents into collection. Normally the operation ends with 201 HTTP status code, `Last-Modified` and `Location` headers specified and with an empty response content. `identifier` can be parsed from the `Location` response header.
+
+
+ When the document to post is marked as a duplicate (using rules described at `API3_DEDUP_FALLBACK_ENABLED` switch), the update operation takes place instead of inserting. In this case the original document in the collection is found and it gets updated by the actual operation POST body. Finally the operation ends with 204 HTTP status code along with `Last-Modified` and correct `Location` headers.
+
+
+ This operation provides autopruning of the collection (if autopruning is enabled).
+
+
+ This operation requires `create` (and/or `update` for deduplication) permission for the API and the collection (e.g. `api:treatments:create` and `api:treatments:update`)
+
+ requestBody:
+ description: JSON with new document to insert
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/DocumentToPost'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 201:
+ $ref: '#/components/responses/201CreatedLocation'
+ 204:
+ $ref: '#/components/responses/204NoContentLocation'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+
+
+ #return HTTP STATUS 400 for all other verbs (PUT, PATCH, DELETE,...)
+
+
+ /{collection}/{identifier}:
+ parameters:
+ - in: path
+ name: collection
+ description: Collection to which the operation is targeted
+ required: true
+ schema:
+ $ref: '#/components/schemas/paramCollection'
+ - in: path
+ name: identifier
+ description: Identifier of the document to which the operation is targeted
+ required: true
+ schema:
+ $ref: '#/components/schemas/paramIdentifier'
+
+ - $ref: '#/components/parameters/dateHeader'
+ - $ref: '#/components/parameters/nowParam'
+ - $ref: '#/components/parameters/tokenParam'
+
+ ######################################################################################
+ get:
+ tags:
+ - generic
+ summary: 'READ: Retrieves a single document from the collection'
+ description:
+ Basically this operation looks for a document matching the `identifier` field returning 200 or 404 HTTP status code.
+
+
+ If the document has been found in the collection but it had already been deleted, 410 HTTP status code with empty response content is to be returned.
+
+
+ When `If-Modified-Since` header is used and its value is greater than the timestamp of the document in the collection, 304 HTTP status code with empty response content is returned. It means that the document has not been modified on server since the last retrieval to client side.
+ With `If-Modified-Since` header and less or equal timestamp `srvModified` a normal 200 HTTP status with full response is returned.
+
+
+ This operation requires `read` permission for the API and the collection (e.g. `api:treatments:read`)
+
+ parameters:
+ - $ref: '#/components/parameters/ifModifiedSinceHeader'
+ - $ref: '#/components/parameters/fieldsParam'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ $ref: '#/components/responses/read200'
+ 304:
+ $ref: '#/components/responses/304NotModified'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+ 410:
+ $ref: '#/components/responses/410Gone'
+
+
+ ######################################################################################
+ put:
+ tags:
+ - generic
+ summary: 'UPDATE: Updates a document in the collection'
+ description:
+ Normally the document with the matching `identifier` will be replaced in the collection by the whole JSON request body and 204 HTTP status code will be returned with empty response body.
+
+
+ If the document has been found in the collection but it had already been deleted, 410 HTTP status code with empty response content is to be returned.
+
+
+ When no document with `identifier` has been found in the collection, then an insert operation takes place instead of updating. Finally 201 HTTP status code is returned with only `Last-Modified` header (`identifier` is already known from the path parameter).
+
+
+ You can also specify `If-Unmodified-Since` request header including your timestamp of document's last modification. If the document has been modified by somebody else on the server afterwards (and you do not know about it), the 412 HTTP status code is returned cancelling the update operation. You can use this feature to prevent race condition problems.
+
+
+ This operation provides autopruning of the collection (if autopruning is enabled).
+
+
+ This operation requires `update` (and/or `create`) permission for the API and the collection (e.g. `api:treatments:update` and `api:treatments:create`)
+
+ parameters:
+ - $ref: '#/components/parameters/ifUnmodifiedSinceHeader'
+
+ requestBody:
+ description: JSON of new version of document (`identifier` in JSON is ignored if present)
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/DocumentToPost'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 201:
+ $ref: '#/components/responses/201Created'
+ 204:
+ $ref: '#/components/responses/204NoContentLocation'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+ 412:
+ $ref: '#/components/responses/412PreconditionFailed'
+ 410:
+ $ref: '#/components/responses/410Gone'
+
+
+ ######################################################################################
+ patch:
+ tags:
+ - generic
+ summary: 'PATCH: Partially updates document in the collection'
+ description:
+ Normally the document with the matching `identifier` will be retrieved from the collection and it will be patched by all specified fields from the JSON request body. Finally 204 HTTP status code will be returned with empty response body.
+
+
+ If the document has been found in the collection but it had already been deleted, 410 HTTP status code with empty response content is to be returned.
+
+
+ When no document with `identifier` has been found in the collection, then the operation ends with 404 HTTP status code.
+
+
+ You can also specify `If-Unmodified-Since` request header including your timestamp of document's last modification. If the document has been modified by somebody else on the server afterwards (and you do not know about it), the 412 HTTP status code is returned cancelling the update operation. You can use this feature to prevent race condition problems.
+
+
+ `PATCH` operation can save some bandwidth for incremental document updates in comparison with `GET` - `UPDATE` operation sequence.
+
+
+ While patching the document, the field `modifiedBy` is automatically set to the authorized subject's name.
+
+
+ This operation provides autopruning of the collection (if autopruning is enabled).
+
+
+ This operation requires `update` permission for the API and the collection (e.g. `api:treatments:update`)
+
+ parameters:
+ - $ref: '#/components/parameters/ifUnmodifiedSinceHeader'
+
+ requestBody:
+ description: JSON of new version of document (`identifier` in JSON is ignored if present)
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/DocumentToPost'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 204:
+ $ref: '#/components/responses/204NoContentLocation'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+ 412:
+ $ref: '#/components/responses/412PreconditionFailed'
+ 410:
+ $ref: '#/components/responses/410Gone'
+
+
+ ######################################################################################
+ delete:
+ tags:
+ - generic
+ summary: 'DELETE: Deletes a document from the collection'
+ description:
+ If the document has already been deleted, the operation will succeed anyway. Normally, documents are not really deleted from the collection but they are only marked as deleted. For special cases the deletion can be irreversible using `permanent` parameter.
+
+
+ This operation provides autopruning of the collection (if autopruning is enabled).
+
+
+ This operation requires `delete` permission for the API and the collection (e.g. `api:treatments:delete`)
+
+
+ parameters:
+ - $ref: '#/components/parameters/permanentParam'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 204:
+ description: Successful operation - empty response
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+
+
+ ######################################################################################
+ /{collection}/history:
+ parameters:
+ - in: path
+ name: collection
+ description: Collection to which the operation is targeted
+ required: true
+ schema:
+ $ref: '#/components/schemas/paramCollection'
+
+ - $ref: '#/components/parameters/dateHeader'
+ - $ref: '#/components/parameters/nowParam'
+ - $ref: '#/components/parameters/tokenParam'
+
+ get:
+ tags:
+ - generic
+ summary: 'HISTORY: Retrieves incremental changes since timestamp'
+ operationId: HISTORY
+ description:
+ HISTORY operation is intended for continuous data synchronization with other systems.
+
+ Every insertion, update and deletion will be included in the resulting JSON array of documents (since timestamp in `Last-Modified` request header value). All changes are listed chronologically in response with 200 HTTP status code. The maximum listed `srvModified` timestamp is also stored in `Last-Modified` and `ETag` response headers that you can use for future, directly following synchronization. You can also limit the array's length using `limit` parameter.
+
+
+ Deleted documents will appear with `isValid` = `false` field.
+
+
+ When there is no change detected since the timestamp the operation ends with 204 HTTP status code and empty response content.
+
+
+ HISTORY operation has a fallback mechanism in place for documents, which were not created by API v3. For such documents `srvModified` is virtually assigned from the `date` field (for `entries` collection) or from the `created_at` field (for other collections).
+
+
+ This operation requires `read` permission for the API and the collection (e.g. `api:treatments:read`)
+
+
+ The only exception is the `settings` collection which requires `admin` permission (`api:settings:admin`), because the settings of each application should be isolated and kept secret. You need to know the concrete identifier to access the app's settings.
+
+
+ parameters:
+ - $ref: '#/components/parameters/lastModifiedRequiredHeader'
+ - $ref: '#/components/parameters/limitParam'
+ - $ref: '#/components/parameters/fieldsParam'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ $ref: '#/components/responses/history200'
+ 204:
+ $ref: '#/components/responses/history204'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+
+
+ ######################################################################################
+ /{collection}/history/{lastModified}:
+ parameters:
+ - in: path
+ name: collection
+ description: Collection to which the operation is targeted
+ required: true
+ schema:
+ $ref: '#/components/schemas/paramCollection'
+
+ - in: path
+ name: lastModified
+ description: Starting timestamp (in UNIX epoch format, defined with respect to server's clock) since which the changes in documents are to be listed. Query for modified documents is made using "greater than" operator (not including equal timestamps).
+ required: true
+ schema:
+ type: integer
+ format: int64
+
+ - $ref: '#/components/parameters/dateHeader'
+ - $ref: '#/components/parameters/nowParam'
+ - $ref: '#/components/parameters/tokenParam'
+
+ get:
+ tags:
+ - generic
+ summary: 'HISTORY: Retrieves incremental changes since timestamp'
+ operationId: HISTORY2
+ description:
+ This HISTORY operation variant is more precise than the previous one with `Last-Modified` request HTTP header), because it does not loose milliseconds precision.
+
+
+ Since this variant queries for changed documents by timestamp precisely and exclusively, the last modified document does not repeat itself in following calls. That is the reason why is this variant more suitable for continuous synchronization with other systems.
+
+
+ This variant behaves quite the same as the previous one in all other aspects.
+
+
+ parameters:
+ - $ref: '#/components/parameters/limitParam'
+ - $ref: '#/components/parameters/fieldsParam'
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ $ref: '#/components/responses/history200'
+ 204:
+ $ref: '#/components/responses/history204'
+ 400:
+ $ref: '#/components/responses/400BadRequest'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+ 404:
+ $ref: '#/components/responses/404NotFound'
+
+
+ ######################################################################################
+ /version:
+
+ get:
+ tags:
+ - other
+ summary: 'VERSION: Returns actual version information'
+ description: No authentication is needed for this commnad (it is public)
+ responses:
+ 200:
+ description: Successful response
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Version'
+
+
+ ######################################################################################
+ /status:
+
+ get:
+ tags:
+ - other
+ summary: 'STATUS: Returns actual version information and all permissions granted for API'
+ description:
+ This operation requires authorization in contrast with VERSION operation.
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ description: Successful response
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Status'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+
+ ######################################################################################
+ /lastModified:
+ parameters:
+ - $ref: '#/components/parameters/dateHeader'
+ - $ref: '#/components/parameters/nowParam'
+ - $ref: '#/components/parameters/tokenParam'
+
+ get:
+ tags:
+ - other
+ summary: 'LAST MODIFIED: Retrieves timestamp of the last modification of every collection'
+ operationId: LAST-MODIFIED
+ description:
+ LAST MODIFIED operation inspects collections separately (in parallel) and for each of them it finds the date of any last modification (insertion, update, deletion).
+
+ Not only `srvModified`, but also `date` and `created_at` fields are inspected (as a fallback to previous API).
+
+
+ This operation requires `read` permission for the API and the collections (e.g. `api:treatments:read`). For each collection the permission is checked separately, you will get timestamps only for those collections that you have access to.
+
+ security:
+ - apiKeyAuth: []
+
+ responses:
+ 200:
+ $ref: '#/components/responses/lastModified200'
+ 401:
+ $ref: '#/components/responses/401Unauthorized'
+ 403:
+ $ref: '#/components/responses/403Forbidden'
+
+######################################################################################
+components:
+
+ parameters:
+
+ dateHeader:
+ in: header
+ name: Date
+ schema:
+ type: string
+ required: false
+ description:
+ Timestamp (defined by client's clock) when the HTTP request was constructed on client.
+ This mandatory header serves as an anti-replay precaution. After a period of time (specified by `API3_TIME_SKEW_TOLERANCE`) the message won't be valid any more and it will be denied with HTTP 401 Unauthorized code.
+ This can be set alternatively in `now` query parameter.
+
+ Example:
+
+
+ Date: Wed, 17 Oct 2018 05:13:00 GMT
+
+
+ nowParam:
+ in: query
+ name: now
+ schema:
+ type: integer
+ format: int64
+ required: false
+ description:
+ Timestamp (defined by client's clock) when the HTTP request was constructed on client.
+ This mandatory parameter serves as an anti-replay precaution. After a period of time (specified by `API3_TIME_SKEW_TOLERANCE`) the message won't be valid any more and it will be denied with HTTP 401 Unauthorized code.
+ This can be set alternatively in `Date` header.
+
+
+ Example:
+
+
+ now=1525383610088
+
+
+ tokenParam:
+ in: query
+ name: token
+ schema:
+ type: string
+ required: false
+ description:
+ An alternative way of authorization - passing accessToken in a query parameter.
+
+
+ Example:
+
+
+ token=testadmin-bf2591231bd2c042
+
+
+ limitParam:
+ in: query
+ name: limit
+ schema:
+ type: integer
+ minimum: 1
+ default: stored in API3_MAX_LIMIT environment variable (usually 1000)
+ example: 100
+ description: Maximum number of documents to get in result array
+
+ skipParam:
+ in: query
+ name: skip
+ schema:
+ type: integer
+ minimum: 0
+ default: 0
+ example: 0
+ description:
+ Number of documents to skip from collection query before
+ loading them into result array (used for pagination)
+
+ sortParam:
+ in: query
+ name: sort
+ schema:
+ type: string
+ required: false
+ description:
+ Field name by which the sorting of documents is performed. This parameter cannot be combined with `sort$desc` parameter.
+
+ sortDescParam:
+ in: query
+ name: sort$desc
+ schema:
+ type: string
+ required: false
+ description:
+ Field name by which the descending (reverse) sorting of documents is performed. This parameter cannot be combined with `sort` parameter.
+
+ permanentParam:
+ in: query
+ name: permanent
+ schema:
+ type: boolean
+ required: false
+ description:
+ If true, the deletion will be irreversible and it will not appear in `HISTORY` operation. Normally there is no reason for setting this flag.
+
+
+ fieldsParam:
+ in: query
+ name: fields
+ schema:
+ type: string
+ default: '_all'
+ required: false
+ examples:
+ all:
+ value: '_all'
+ summary: All fields will be returned (default behaviour)
+ customSet:
+ value: 'date,insulin'
+ summary: Only fields date and insulin will be returned
+ description: A chosen set of fields to return in response. Either you can enumerate specific fields of interest or use the predefined set. Sample parameter values:
+
+
+ _all: All fields will be returned (default value)
+
+
+ date,insulin: Only fields `date` and `insulin` will be returned
+
+
+ filterParams:
+ in: query
+ name: filter_parameters
+ schema:
+ type: string
+ description:
+ Any number of filtering operators.
+
+
+ Each filtering operator has name like `$`, e.g. `carbs$gt=2` which represents filtering rule "The field carbs must be present and greater than 2".
+
+
+ You can choose from operators:
+
+
+ `eq`=equals, `insulin$eq=1.5`
+
+
+ `ne`=not equals, `insulin$ne=1.5`
+
+
+ `gt`=greater than, `carbs$gt=30`
+
+
+ `gte`=greater than or equal, `carbs$gte=30`
+
+
+ `lt`=less than, `carbs$lt=30`
+
+
+ `lte`=less than or equal, `carbs$lte=30`
+
+
+ `in`=in specified set, `type$in=sgv|mbg|cal`
+
+
+ `nin`=not in specified set, `eventType$nin=Temp%20Basal|Temporary%20Target`
+
+
+ `re`=regex pattern, `eventType$re=Temp.%2A`
+
+
+ When filtering by field `date`, `created_at`, `srvModified` or `srvCreated`, you can choose from three input formats
+
+ - Unix epoch in milliseconds (1525383610088)
+
+ - Unix epoch in seconds (1525383610)
+
+ - ISO 8601 with optional timezone ('2018-05-03T21:40:10.088Z' or '2018-05-03T23:40:10.088+02:00')
+
+
+ The date is always queried in a normalized form - UTC with zero offset and with the correct format (1525383610088 for `date`, '2018-05-03T21:40:10.088Z' for `created_at`).
+
+ lastModifiedRequiredHeader:
+ in: header
+ name: Last-Modified
+ schema:
+ type: string
+ required: true
+ description:
+ Starting timestamp (defined with respect to server's clock) since which the changes in documents are to be listed, formatted as:
+
+
+ <day-name>, <day> <month> <year> <hour>:<minute>:<second> GMT
+
+
+ Example:
+
+
+ Last-Modified: Wed, 17 Oct 2018 05:13:00 GMT
+
+
+ ifModifiedSinceHeader:
+ in: header
+ name: If-Modified-Since
+ schema:
+ type: string
+ required: false
+ description:
+ Timestamp (defined with respect to server's clock) of the last document modification formatted as:
+
+
+ <day-name>, <day> <month> <year> <hour>:<minute>:<second> GMT
+
+
+ If this header is present, the operation will compare its value with the srvModified timestamp of the document at first and the operation result then may differ. The srvModified timestamp was defined by server's clock.
+
+
+ Example:
+
+
+ If-Modified-Since: Wed, 17 Oct 2018 05:13:00 GMT
+
+
+ ifUnmodifiedSinceHeader:
+ in: header
+ name: If-Unmodified-Since
+ schema:
+ type: string
+ required: false
+ description:
+ Timestamp (defined with respect to server's clock) of the last document modification formatted as:
+
+
+ <day-name>, <day> <month> <year> <hour>:<minute>:<second> GMT
+
+
+ If this header is present, the operation will compare its value with the srvModified timestamp of the document at first and the operation result then may differ. The srvModified timestamp was defined by server's clock.
+
+
+ Example:
+
+
+ If-Unmodified-Since: Wed, 17 Oct 2018 05:13:00 GMT
+
+
+ ######################################################################################
+ responses:
+
+ 201Created:
+ description: Successfully created a new document in collection
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+
+ 201CreatedLocation:
+ description: Successfully created a new document in collection
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+ 'Location':
+ $ref: '#/components/schemas/headerLocation'
+
+ 204NoContent:
+ description: Successfully finished operation
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+
+ 204NoContentLocation:
+ description: Successfully finished operation
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+ 'Location':
+ $ref: '#/components/schemas/headerLocation'
+
+ 304NotModified:
+ description: The document has not been modified on the server since timestamp specified in If-Modified-Since header
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+
+ 400BadRequest:
+ description: The request is malformed. There may be some required parameters missing or there are unrecognized parameters present.
+
+ 401Unauthorized:
+ description: The request was not successfully authenticated using access token or JWT, or the request has missing `Date` header or it contains an expired timestamp, so that the request cannot continue due to the security policy.
+
+ 403Forbidden:
+ description: Insecure HTTP scheme used or the request has been successfully authenticated, but the security subject is not authorized for the operation.
+
+ 404NotFound:
+ description: The collection or document specified was not found.
+
+ 412PreconditionFailed:
+ description: The document has already been modified on the server since specified timestamp (in If-Unmodified-Since header).
+
+ 410Gone:
+ description: The requested document has already been deleted.
+
+ search200:
+ description: Successful operation returning array of documents matching the filtering criteria
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/DocumentArray'
+
+ search204:
+ description: Successful operation - no documents matching the filtering criteria
+
+ read200:
+ description: The document has been succesfully found and its JSON form returned in the response content.
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/Document'
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModified'
+
+ history200:
+ description:
+ Changed documents since specified timestamp
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/DocumentArray'
+ headers:
+ 'Last-Modified':
+ $ref: '#/components/schemas/headerLastModifiedMaximum'
+ 'ETag':
+ $ref: '#/components/schemas/headerEtagLastModifiedMaximum'
+
+ history204:
+ description: No changes detected
+
+ lastModified200:
+ description: Successful operation returning the timestamps
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/LastModifiedResult'
+
+ ######################################################################################
+ schemas:
+
+ headerLocation:
+ type: string
+ description:
+ Location of document - the relative part of URL. This can be used to parse the identifier
+ of just created document.
+
+ Example=/api/v3/treatments/53409478-105f-11e9-ab14-d663bd873d93
+
+ headerLastModified:
+ type: string
+ description:
+ Timestamp of the last document modification on the server, formatted as
+
+ ', :: GMT'.
+
+ This field is relevant only for documents which were somehow modified by API v3
+ (inserted, updated or deleted) and it was generated using server's clock.
+
+ Example='Wed, 17 Oct 2018 05:13:00 GMT'
+
+ headerLastModifiedMaximum:
+ type: string
+ description:
+ The latest (maximum) `srvModified` field of all returning documents, formatted as
+
+ ', :: GMT'.
+
+ Example='Wed, 17 Oct 2018 05:13:00 GMT'
+
+ headerEtagLastModifiedMaximum:
+ type: string
+ description:
+ The latest (maximum) `srvModified` field of all returning documents.
+ This header does not loose milliseconds from the date (unlike the `Last-Modified` header).
+
+ Example='W/"1525383610088"'
+
+ paramCollection:
+ type: string
+ enum:
+ - devicestatus
+ - entries
+ - food
+ - profile
+ - settings
+ - treatments
+ example: 'treatments'
+
+ paramIdentifier:
+ type: string
+ example: '53409478-105f-11e9-ab14-d663bd873d93'
+
+
+ DocumentBase:
+ description: Shared base for all documents
+ properties:
+ identifier:
+ description:
+ Main addressing, required field that identifies document in the collection.
+
+
+ The client should not create the identifier, the server automatically assigns it when the document is inserted.
+
+
+ The server calculates the identifier in such a way that duplicate records are automatically merged (deduplicating is made by `date`, `device` and `eventType` fields).
+
+
+ The best practise for all applications is not to loose identifiers from received documents, but save them carefully for other consumer applications/systems.
+
+
+ API v3 has a fallback mechanism in place, for documents without `identifier` field the `identifier` is set to internal `_id`, when reading or addressing these documents.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ type: string
+ example: '53409478-105f-11e9-ab14-d663bd873d93'
+
+ date:
+ type: integer
+ format: int64
+ description:
+ Required timestamp when the record or event occured, you can choose from three input formats
+
+ - Unix epoch in milliseconds (1525383610088)
+
+ - Unix epoch in seconds (1525383610)
+
+ - ISO 8601 with optional timezone ('2018-05-03T21:40:10.088Z' or '2018-05-03T23:40:10.088+02:00')
+
+
+ The date is always stored in a normalized form - UTC with zero offset. If UTC offset was present, it is going to be set in the `utcOffset` field.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 1525383610088
+
+ utcOffset:
+ type: integer
+ description:
+ Local UTC offset (timezone) of the event in minutes. This field can be set either directly by the client (in the incoming document) or it is automatically parsed from the `date` field.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 120
+
+ app:
+ type: string
+ description:
+ Application or system in which the record was entered by human or device for the first time.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: xdrip
+
+ device:
+ type: string
+ description:
+ The device from which the data originated (including serial number of the device, if it is relevant and safe).
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 'dexcom G5'
+
+ _id:
+ description: Internally assigned database id. This field is for internal server purposes only, clients communicate with API by using identifier field.
+ type: string
+ example: '58e9dfbc166d88cc18683aac'
+
+ srvCreated:
+ type: integer
+ format: int64
+ description:
+ The server's timestamp of document insertion into the database (Unix epoch in ms). This field appears only for documents which were inserted by API v3.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 1525383610088
+
+ subject:
+ type: string
+ description:
+ Name of the security subject (within Nightscout scope) which has created the document. This field is automatically set by the server from the passed token or JWT.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 'uploader'
+
+ srvModified:
+ type: integer
+ format: int64
+ description:
+ The server's timestamp of the last document modification in the database (Unix epoch in ms). This field appears only for documents which were somehow modified by API v3 (inserted, updated or deleted).
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: 1525383610088
+
+ modifiedBy:
+ type: string
+ description:
+ Name of the security subject (within Nightscout scope) which has patched or deleted the document for the last time. This field is automatically set by the server.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: admin
+
+ isValid:
+ type: boolean
+ description:
+ A flag set by the server only for deleted documents. This field appears
+ only within history operation and for documents which were deleted by API v3 (and they always have a false value)
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ example: false
+
+
+ required:
+ - date
+ - app
+
+
+ DeviceStatus:
+ description: State of physical device, which is a technical part of the whole T1D compensation system
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+ some_property:
+ type: string
+ description: ...
+
+
+ Entry:
+ description: Blood glucose measurements and CGM calibrations
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+
+ type:
+ type: string
+ description: 'sgv, mbg, cal, etc'
+
+ sgv:
+ type: number
+ description: The glucose reading. (only available for sgv types)
+
+ direction:
+ type: string
+ description: Direction of glucose trend reported by CGM. (only available for sgv types)
+ example: '"DoubleDown", "SingleDown", "FortyFiveDown", "Flat", "FortyFiveUp", "SingleUp", "DoubleUp", "NOT COMPUTABLE", "RATE OUT OF RANGE" for xdrip'
+
+ noise:
+ type: number
+ description: Noise level at time of reading. (only available for sgv types)
+ example: 'xdrip: 0, 1, 2=high, 3=high_for_predict, 4=very high, 5=extreme'
+
+ filtered:
+ type: number
+ description: The raw filtered value directly from CGM transmitter. (only available for sgv types)
+
+ unfiltered:
+ type: number
+ description: The raw unfiltered value directly from CGM transmitter. (only available for sgv types)
+
+ rssi:
+ type: number
+ description: The signal strength from CGM transmitter. (only available for sgv types)
+
+ units:
+ type: string
+ example: '"mg", "mmol"'
+ description: The units for the glucose value, mg/dl or mmol/l. It is strongly recommended to fill in this field.
+
+
+ Food:
+ description: Nutritional values of food
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+
+ food:
+ type: string
+ description: 'food, quickpick'
+
+ category:
+ type: string
+ description: Name for a group of related records
+
+ subcategory:
+ type: string
+ description: Name for a second level of groupping
+
+ name:
+ type: string
+ description: Name of the food described
+
+ portion:
+ type: number
+ description: Number of units (e.g. grams) of the whole portion described
+
+ unit:
+ type: string
+ example: '"g", "ml", "oz"'
+ description: Unit for the portion
+
+ carbs:
+ type: number
+ description: Amount of carbs in the portion in grams
+
+ fat:
+ type: number
+ description: Amount of fat in the portion in grams
+
+ protein:
+ type: number
+ description: Amount of proteins in the portion in grams
+
+ energy:
+ type: number
+ description: Amount of energy in the portion in kJ
+
+ gi:
+ type: number
+ description: 'Glycemic index (1=low, 2=medium, 3=high)'
+
+ hideafteruse:
+ type: boolean
+ description: Flag used for quickpick
+
+ hidden:
+ type: boolean
+ description: Flag used for quickpick
+
+ position:
+ type: number
+ description: Ordering field for quickpick
+
+ portions:
+ type: number
+ description: component multiplier if defined inside quickpick compound
+
+ foods:
+ type: array
+ description: Neighbour documents (from food collection) that together make a quickpick compound
+ items:
+ $ref: '#/components/schemas/Food'
+
+
+ Profile:
+ description: Parameters describing body functioning relative to T1D + compensation parameters
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+ some_property:
+ type: string
+ description: ...
+
+
+ Settings:
+ description:
+ A document representing persisted settings of some application or system (it could by Nightscout itself as well). This pack of options serves as a backup or as a shared centralized storage for multiple client instances. It is a probably good idea to `PATCH` the document instead of `UPDATE` operation, e.g. when changing one settings option in a client application.
+
+
+ `identifier` represents a client application name here, e.g. `xdrip` or `aaps`.
+
+
+ `Settings` collection has a more specific authorization required. For the `SEARCH` operation within this collection, you need an `admin` permission, such as `api:settings:admin`. The goal is to isolate individual client application settings.
+
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+ some_property:
+ type: string
+ description: ...
+
+
+ Treatment:
+ description: T1D compensation action
+ allOf:
+ - $ref: '#/components/schemas/DocumentBase'
+ - type: object
+ properties:
+ eventType:
+ type: string
+ example: '"BG Check", "Snack Bolus", "Meal Bolus", "Correction Bolus", "Carb Correction", "Combo Bolus", "Announcement", "Note", "Question", "Exercise", "Site Change", "Sensor Start", "Sensor Change", "Pump Battery Change", "Insulin Change", "Temp Basal", "Profile Switch", "D.A.D. Alert", "Temporary Target", "OpenAPS Offline", "Bolus Wizard"'
+ description: The type of treatment event.
+
+
+ Note: this field is immutable by the client (it cannot be updated or patched)
+
+
+ # created_at:
+ # type: string
+ # description: The date of the event, might be set retroactively.
+ glucose:
+ type: string
+ description: Current glucose.
+ glucoseType:
+ type: string
+ example: '"Sensor", "Finger", "Manual"'
+ description: Method used to obtain glucose, Finger or Sensor.
+ units:
+ type: string
+ example: '"mg/dl", "mmol/l"'
+ description: The units for the glucose value, mg/dl or mmol/l. It is strongly recommended to fill in this field when `glucose` is entered.
+ carbs:
+ type: number
+ description: Amount of carbs given.
+ protein:
+ type: number
+ description: Amount of protein given.
+ fat:
+ type: number
+ description: Amount of fat given.
+ insulin:
+ type: number
+ description: Amount of insulin, if any.
+ duration:
+ type: number
+ description: Duration in minutes.
+ preBolus:
+ type: number
+ description: How many minutes the bolus was given before the meal started.
+ splitNow:
+ type: number
+ description: Immediate part of combo bolus (in percent).
+ splitExt:
+ type: number
+ description: Extended part of combo bolus (in percent).
+ percent:
+ type: number
+ description: Eventual basal change in percent.
+ absolute:
+ type: number
+ description: Eventual basal change in absolute value (insulin units per hour).
+ targetTop:
+ type: number
+ description: Top limit of temporary target.
+ targetBottom:
+ type: number
+ description: Bottom limit of temporary target.
+ profile:
+ type: string
+ description: Name of the profile to which the pump has been switched.
+ reason:
+ type: string
+ description: For example the reason why the profile has been switched or why the temporary target has been set.
+ notes:
+ type: string
+ description: Description/notes of treatment.
+ enteredBy:
+ type: string
+ description: Who entered the treatment.
+
+
+ DocumentToPost:
+ description: Single document
+ type: object
+ oneOf:
+ - $ref: '#/components/schemas/DeviceStatus'
+ - $ref: '#/components/schemas/Entry'
+ - $ref: '#/components/schemas/Food'
+ - $ref: '#/components/schemas/Profile'
+ - $ref: '#/components/schemas/Settings'
+ - $ref: '#/components/schemas/Treatment'
+ example:
+ 'identifier': '53409478-105f-11e9-ab14-d663bd873d93'
+ 'date': 1532936118000
+ 'utcOffset': 120
+ 'carbs': 10
+ 'insulin': 1
+ 'eventType': 'Snack Bolus'
+ 'app': 'xdrip'
+ 'subject': 'uploader'
+
+
+ Document:
+ description: Single document
+ type: object
+ oneOf:
+ - $ref: '#/components/schemas/DeviceStatus'
+ - $ref: '#/components/schemas/Entry'
+ - $ref: '#/components/schemas/Food'
+ - $ref: '#/components/schemas/Profile'
+ - $ref: '#/components/schemas/Settings'
+ - $ref: '#/components/schemas/Treatment'
+ example:
+ 'identifier': '53409478-105f-11e9-ab14-d663bd873d93'
+ 'date': 1532936118000
+ 'utcOffset': 120
+ 'carbs': 10
+ 'insulin': 1
+ 'eventType': 'Snack Bolus'
+ 'srvCreated': 1532936218000
+ 'srvModified': 1532936218000
+ 'app': 'xdrip'
+ 'subject': 'uploader'
+ 'modifiedBy': 'admin'
+
+
+ DeviceStatusArray:
+ description: Array of documents
+ type: array
+ items:
+ $ref: '#/components/schemas/DeviceStatus'
+
+
+ EntryArray:
+ description: Array of documents
+ type: array
+ items:
+ $ref: '#/components/schemas/Entry'
+
+
+ FoodArray:
+ description: Array of documents
+ type: array
+ items:
+ $ref: '#/components/schemas/Food'
+
+
+ ProfileArray:
+ description: Array of documents
+ type: array
+ items:
+ $ref: '#/components/schemas/Profile'
+
+
+ SettingsArray:
+ description: Array of settings
+ type: array
+ items:
+ $ref: '#/components/schemas/Settings'
+
+
+ TreatmentArray:
+ description: Array of documents
+ type: array
+ items:
+ $ref: '#/components/schemas/Treatment'
+
+
+ DocumentArray:
+ type: object
+ oneOf:
+ - $ref: '#/components/schemas/DeviceStatusArray'
+ - $ref: '#/components/schemas/EntryArray'
+ - $ref: '#/components/schemas/FoodArray'
+ - $ref: '#/components/schemas/ProfileArray'
+ - $ref: '#/components/schemas/SettingsArray'
+ - $ref: '#/components/schemas/TreatmentArray'
+
+
+ Version:
+ description: Information about versions
+ type: object
+ properties:
+
+ version:
+ description: The whole Nightscout instance version
+ type: string
+ example: '0.10.2-release-20171201'
+
+ apiVersion:
+ description: API v3 subsystem version
+ type: string
+ example: '3.0.0'
+
+ srvDate:
+ description: Actual server date and time in UNIX epoch format
+ type: number
+ example: 1532936118000
+
+ storage:
+ type: object
+ properties:
+
+ type:
+ description: Type of storage engine used
+ type: string
+ example: 'mongodb'
+
+ version:
+ description: Version of the storage engine
+ type: string
+ example: '4.0.6'
+
+
+ Status:
+ description: Information about versions and API permissions
+ allOf:
+ - $ref: '#/components/schemas/Version'
+ - type: object
+ properties:
+
+ apiPermissions:
+ type: object
+ properties:
+ devicestatus:
+ type: string
+ example: 'crud'
+ entries:
+ type: string
+ example: 'r'
+ food:
+ type: string
+ example: 'crud'
+ profile:
+ type: string
+ example: 'r'
+ treatments:
+ type: string
+ example: 'crud'
+
+
+ LastModifiedResult:
+ description: Result of LAST MODIFIED operation
+ properties:
+ srvDate:
+ description:
+ Actual storage server date (Unix epoch in ms).
+ type: integer
+ format: int64
+ example: 1556260878776
+
+ collections:
+ type: object
+ description:
+ Collections which the user have read access to.
+ properties:
+ devicestatus:
+ description:
+ Timestamp of the last modification (Unix epoch in ms), `null` when there is no timestamp found.
+ type: integer
+ format: int64
+ example: 1556260760974
+ treatments:
+ description:
+ Timestamp of the last modification (Unix epoch in ms), `null` when there is no timestamp found.
+ type: integer
+ format: int64
+ example: 1553374184169
+ entries:
+ description:
+ Timestamp of the last modification (Unix epoch in ms), `null` when there is no timestamp found.
+ type: integer
+ format: int64
+ example: 1556260758768
+ profile:
+ description:
+ Timestamp of the last modification (Unix epoch in ms), `null` when there is no timestamp found.
+ type: integer
+ format: int64
+ example: 1548524042744
+
+ ######################################################################################
+ securitySchemes:
+
+ accessToken:
+ type: apiKey
+ name: token
+ in: query
+ description: >-
+ Add token as query item in the URL or as HTTP header. You can manage access token in
+ `/admin`.
+
+ Each operation requires a specific permission that has to be granted (via security role) to the security subject, which was authenticated by `token` parameter/header or `JWT`. E.g. for creating new `devicestatus` document via API you need `api:devicestatus:create` permission.
+
+ jwtoken:
+ type: http
+ scheme: bearer
+ description: Use this if you know the temporary json webtoken.
+ bearerFormat: JWT
\ No newline at end of file
diff --git a/lib/authorization/index.js b/lib/authorization/index.js
index 0c0c6f5f2a7..388fa19dffc 100644
--- a/lib/authorization/index.js
+++ b/lib/authorization/index.js
@@ -55,6 +55,8 @@ function init (env, ctx) {
return token;
}
+ authorization.extractToken = extractToken;
+
function authorizeAccessToken (req) {
var accessToken = req.query.token;
@@ -152,9 +154,7 @@ function init (env, ctx) {
if (err) {
return callback(err, { shiros: [ ] });
} else {
- var resolved = storage.resolveSubjectAndPermissions(verified.accessToken);
- var shiros = resolved.shiros.concat(defaultShiros);
- return callback(null, { shiros: shiros, subject: resolved.subject });
+ authorization.resolveAccessToken (verified.accessToken, callback, defaultShiros);
}
});
} else {
@@ -163,6 +163,21 @@ function init (env, ctx) {
};
+ authorization.resolveAccessToken = function resolveAccessToken (accessToken, callback, defaultShiros) {
+
+ if (!defaultShiros) {
+ defaultShiros = storage.rolesToShiros(defaultRoles);
+ }
+
+ let resolved = storage.resolveSubjectAndPermissions(accessToken);
+ if (!resolved || !resolved.subject) {
+ return callback('Subject not found', null);
+ }
+
+ let shiros = resolved.shiros.concat(defaultShiros);
+ return callback(null, { shiros: shiros, subject: resolved.subject });
+ };
+
authorization.isPermitted = function isPermitted (permission, opts) {
diff --git a/lib/server/bootevent.js b/lib/server/bootevent.js
index 9e53c4f77ac..84b6fd9c304 100644
--- a/lib/server/bootevent.js
+++ b/lib/server/bootevent.js
@@ -115,7 +115,7 @@ function boot (env, language) {
} else {
//TODO assume mongo for now, when there are more storage options add a lookup
require('../storage/mongo-storage')(env, function ready(err, store) {
- // FIXME, error is always null, if there is an error, the storage.js will throw an exception
+ // FIXME, error is always null, if there is an error, the index.js will throw an exception
console.log('Mongo Storage system ready');
ctx.store = store;
diff --git a/lib/server/websocket.js b/lib/server/websocket.js
index fb0aa38b349..3946c129a5d 100644
--- a/lib/server/websocket.js
+++ b/lib/server/websocket.js
@@ -520,6 +520,10 @@ function init (env, ctx, server) {
start( );
listeners( );
+ if (ctx.storageSocket) {
+ ctx.storageSocket.init(io);
+ }
+
return websocket();
}
diff --git a/package.json b/package.json
index 3ffb995d5af..a7dbeaac575 100644
--- a/package.json
+++ b/package.json
@@ -106,6 +106,7 @@
"swagger-ui-express": "^4.0.7",
"terser": "^3.17.0",
"traverse": "^0.6.6",
+ "uuid": "^3.3.2",
"webpack": "^4.39.2",
"webpack-cli": "^3.3.7"
},
diff --git a/tests/api.devicestatus.test.js b/tests/api.devicestatus.test.js
index a618db49056..34a0908610e 100644
--- a/tests/api.devicestatus.test.js
+++ b/tests/api.devicestatus.test.js
@@ -56,6 +56,7 @@ describe('Devicestatus API', function ( ) {
request(self.app)
.get('/api/devicestatus/')
.query('find[created_at][$gte]=2018-12-16')
+ .query('find[created_at][$lte]=2018-12-17')
.set('api-secret', self.env.api_secret || '')
.expect(200)
.expect(function (response) {
diff --git a/tests/api3.basic.test.js b/tests/api3.basic.test.js
new file mode 100644
index 00000000000..8e51b343585
--- /dev/null
+++ b/tests/api3.basic.test.js
@@ -0,0 +1,49 @@
+'use strict';
+
+const request = require('supertest');
+require('should');
+
+describe('Basic REST API3', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ ;
+
+ this.timeout(15000);
+
+ before(async () => {
+ self.instance = await instance.create({});
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ });
+
+
+ after(function after () {
+ self.instance.server.close();
+ });
+
+
+ it('GET /swagger', async () => {
+ let res = await request(self.app)
+ .get('/api/v3/swagger.yaml')
+ .expect(200);
+
+ res.header['content-length'].should.be.above(0);
+ });
+
+
+ it('GET /version', async () => {
+ let res = await request(self.app)
+ .get('/api/v3/version')
+ .expect(200);
+
+ const apiConst = require('../lib/api3/const.json')
+ , software = require('../package.json');
+
+ res.body.version.should.equal(software.version);
+ res.body.apiVersion.should.equal(apiConst.API3_VERSION);
+ res.body.srvDate.should.be.within(testConst.YEAR_2019, testConst.YEAR_2050);
+ });
+
+});
+
diff --git a/tests/api3.create.test.js b/tests/api3.create.test.js
new file mode 100644
index 00000000000..cbd17a3e826
--- /dev/null
+++ b/tests/api3.create.test.js
@@ -0,0 +1,487 @@
+/* eslint require-atomic-updates: 0 */
+/* global should */
+'use strict';
+
+require('should');
+
+describe('API3 CREATE', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , opTools = require('../lib/api3/shared/operationTools')
+ , utils = require('./fixtures/api3/utils')
+ ;
+
+ self.validDoc = {
+ date: (new Date()).getTime(),
+ app: testConst.TEST_APP,
+ device: testConst.TEST_DEVICE,
+ eventType: 'Correction Bolus',
+ insulin: 0.3
+ };
+ self.validDoc.identifier = opTools.calculateIdentifier(self.validDoc);
+
+ self.timeout(20000);
+
+
+ /**
+ * Cleanup after successful creation
+ */
+ self.delete = async function deletePermanent (identifier) {
+ await self.instance.delete(`${self.url}/${identifier}?permanent=true&token=${self.token.delete}`)
+ .expect(204);
+ };
+
+
+ /**
+ * Get document detail for futher processing
+ */
+ self.get = async function get (identifier) {
+ let res = await self.instance.get(`${self.url}/${identifier}?token=${self.token.read}`)
+ .expect(200);
+
+ return res.body;
+ };
+
+
+ /**
+ * Get document detail for futher processing
+ */
+ self.search = async function search (date) {
+ let res = await self.instance.get(`${self.url}?date$eq=${date}&token=${self.token.read}`)
+ .expect(200);
+
+ return res.body;
+ };
+
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/treatments';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}?token=${self.token.create}`;
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.post(`${self.url}`)
+ .send(self.validDoc)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.post(`/api/v3/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should require create permission', async () => {
+ let res = await self.instance.post(`${self.url}?token=${self.token.read}`)
+ .send(self.validDoc)
+ .expect(403);
+
+ res.body.status.should.equal(403);
+ res.body.message.should.equal('Missing permission api:treatments:create');
+ });
+
+
+ it('should reject empty body', async () => {
+ await self.instance.post(self.urlToken)
+ .send({ })
+ .expect(400);
+ });
+
+
+ it('should accept valid document', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+ res.headers.location.should.equal(`${self.url}/${self.validDoc.identifier}`);
+ const lastModified = new Date(res.headers['last-modified']).getTime(); // Last-Modified has trimmed milliseconds
+
+ let body = await self.get(self.validDoc.identifier);
+ body.should.containEql(self.validDoc);
+
+ const ms = body.srvModified % 1000;
+ (body.srvModified - ms).should.equal(lastModified);
+ (body.srvCreated - ms).should.equal(lastModified);
+ body.subject.should.equal(self.subject.apiCreate.name);
+
+ await self.delete(self.validDoc.identifier);
+ });
+
+
+ it('should reject missing date', async () => {
+ let doc = Object.assign({}, self.validDoc);
+ delete doc.date;
+
+ let res = await self.instance.post(self.urlToken)
+ .send(doc)
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+ it('should reject invalid date null', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: null }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+ it('should reject invalid date ABC', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: 'ABC' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+ it('should reject invalid date -1', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: -1 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+
+ it('should reject invalid date 1 (too old)', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: 1 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+ it('should reject invalid date - illegal format', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: '2019-20-60T50:90:90' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing date field');
+ });
+
+
+ it('should reject invalid utcOffset -5000', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: -5000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing utcOffset field');
+ });
+
+
+ it('should reject invalid utcOffset ABC', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: 'ABC' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing utcOffset field');
+ });
+
+
+ it('should accept valid utcOffset', async () => {
+ await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: 120 }))
+ .expect(201);
+
+ let body = await self.get(self.validDoc.identifier);
+ body.utcOffset.should.equal(120);
+ await self.delete(self.validDoc.identifier);
+ });
+
+
+ it('should reject invalid utcOffset null', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: null }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing utcOffset field');
+ });
+
+
+ it('should reject missing app', async () => {
+ let doc = Object.assign({}, self.validDoc);
+ delete doc.app;
+
+ let res = await self.instance.post(self.urlToken)
+ .send(doc)
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing app field');
+ });
+
+
+ it('should reject invalid app null', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { app: null }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing app field');
+ });
+
+
+ it('should reject empty app', async () => {
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { app: '' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Bad or missing app field');
+ });
+
+
+ it('should normalize date and store utcOffset', async () => {
+ await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: '2019-06-10T08:07:08,576+02:00' }))
+ .expect(201);
+
+ let body = await self.get(self.validDoc.identifier);
+ body.date.should.equal(1560146828576);
+ body.utcOffset.should.equal(120);
+ await self.delete(self.validDoc.identifier);
+ });
+
+
+ it('should require update permission for deduplication', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ self.validDoc.identifier = utils.randomString('32', 'aA#');
+
+ const doc = Object.assign({}, self.validDoc);
+
+ await self.instance.post(self.urlToken)
+ .send(doc)
+ .expect(201);
+
+ let createdBody = await self.get(doc.identifier);
+ createdBody.should.containEql(doc);
+
+ const doc2 = Object.assign({}, doc);
+ let res = await self.instance.post(self.urlToken)
+ .send(doc2)
+ .expect(403);
+
+ res.body.status.should.equal(403);
+ res.body.message.should.equal('Missing permission api:treatments:update');
+ await self.delete(doc.identifier);
+ });
+
+
+ it('should deduplicate document by identifier', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ self.validDoc.identifier = utils.randomString('32', 'aA#');
+
+ const doc = Object.assign({}, self.validDoc);
+
+ await self.instance.post(self.urlToken)
+ .send(doc)
+ .expect(201);
+
+ let createdBody = await self.get(doc.identifier);
+ createdBody.should.containEql(doc);
+
+ const doc2 = Object.assign({}, doc, {
+ insulin: 0.5
+ });
+
+ await self.instance.post(`${self.url}?token=${self.token.all}`)
+ .send(doc2)
+ .expect(204);
+
+ let updatedBody = await self.get(doc2.identifier);
+ updatedBody.should.containEql(doc2);
+
+ await self.delete(doc2.identifier);
+ });
+
+
+ it('should deduplicate document by created_at+eventType', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ self.validDoc.identifier = utils.randomString('32', 'aA#');
+
+ const doc = Object.assign({}, self.validDoc, {
+ created_at: new Date(self.validDoc.date).toISOString()
+ });
+ delete doc.identifier;
+
+ self.instance.ctx.treatments.create([doc], async (err) => { // let's insert the document in APIv1's way
+ should.not.exist(err);
+
+ const doc2 = Object.assign({}, doc, {
+ insulin: 0.4,
+ identifier: utils.randomString('32', 'aA#')
+ });
+ delete doc2._id; // APIv1 updates input document, we must get rid of _id for the next round
+
+ await self.instance.post(`${self.url}?token=${self.token.all}`)
+ .send(doc2)
+ .expect(204);
+
+ let updatedBody = await self.get(doc2.identifier);
+ updatedBody.should.containEql(doc2);
+
+ await self.delete(doc2.identifier);
+ });
+ });
+
+
+ it('should not deduplicate treatment only by created_at', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ self.validDoc.identifier = utils.randomString('32', 'aA#');
+
+ const doc = Object.assign({}, self.validDoc, {
+ created_at: new Date(self.validDoc.date).toISOString()
+ });
+ delete doc.identifier;
+
+ self.instance.ctx.treatments.create([doc], async (err) => { // let's insert the document in APIv1's way
+ should.not.exist(err);
+
+ let oldBody = await self.get(doc._id);
+ delete doc._id; // APIv1 updates input document, we must get rid of _id for the next round
+ oldBody.should.containEql(doc);
+
+ const doc2 = Object.assign({}, doc, {
+ eventType: 'Meal Bolus',
+ insulin: 0.4,
+ identifier: utils.randomString('32', 'aA#')
+ });
+
+ await self.instance.post(`${self.url}?token=${self.token.all}`)
+ .send(doc2)
+ .expect(201);
+
+ let updatedBody = await self.get(doc2.identifier);
+ updatedBody.should.containEql(doc2);
+ updatedBody.identifier.should.not.equal(oldBody.identifier);
+
+ await self.delete(doc2.identifier);
+ await self.delete(oldBody.identifier);
+ });
+ });
+
+
+ it('should overwrite deleted document', async () => {
+ const date1 = new Date()
+ , identifier = utils.randomString('32', 'aA#');
+
+ await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { identifier, date: date1.toISOString() }))
+ .expect(201);
+
+ await self.instance.delete(`${self.url}/${identifier}?token=${self.token.delete}`)
+ .expect(204);
+
+ const date2 = new Date();
+ let res = await self.instance.post(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { identifier, date: date2.toISOString() }))
+ .expect(403);
+
+ res.body.status.should.be.equal(403);
+ res.body.message.should.be.equal('Missing permission api:treatments:update');
+
+ res = await self.instance.post(`${self.url}?token=${self.token.all}`)
+ .send(Object.assign({}, self.validDoc, { identifier, date: date2.toISOString() }))
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ let body = await self.get(identifier);
+ body.date.should.equal(date2.getTime());
+ body.identifier.should.equal(identifier);
+ await self.delete(identifier);
+ });
+
+
+ it('should calculate the identifier', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ delete self.validDoc.identifier;
+ const validIdentifier = opTools.calculateIdentifier(self.validDoc);
+
+ let res = await self.instance.post(self.urlToken)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+ res.headers.location.should.equal(`${self.url}/${validIdentifier}`);
+ self.validDoc.identifier = validIdentifier;
+
+ let body = await self.get(validIdentifier);
+ body.should.containEql(self.validDoc);
+ await self.delete(validIdentifier);
+ });
+
+
+ it('should deduplicate by identifier calculation', async () => {
+ self.validDoc.date = (new Date()).getTime();
+ delete self.validDoc.identifier;
+ const validIdentifier = opTools.calculateIdentifier(self.validDoc);
+
+ let res = await self.instance.post(self.urlToken)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+ res.headers.location.should.equal(`${self.url}/${validIdentifier}`);
+ self.validDoc.identifier = validIdentifier;
+
+ let body = await self.get(validIdentifier);
+ body.should.containEql(self.validDoc);
+
+ delete self.validDoc.identifier;
+ res = await self.instance.post(`${self.url}?token=${self.token.update}`)
+ .send(self.validDoc)
+ .expect(204);
+
+ res.body.should.be.empty();
+ res.headers.location.should.equal(`${self.url}/${validIdentifier}`);
+ self.validDoc.identifier = validIdentifier;
+
+ body = await self.search(self.validDoc.date);
+ body.length.should.equal(1);
+
+ await self.delete(validIdentifier);
+ });
+
+});
+
diff --git a/tests/api3.delete.test.js b/tests/api3.delete.test.js
new file mode 100644
index 00000000000..203d32edce8
--- /dev/null
+++ b/tests/api3.delete.test.js
@@ -0,0 +1,53 @@
+/* eslint require-atomic-updates: 0 */
+'use strict';
+
+require('should');
+
+describe('API3 UPDATE', function() {
+ const self = this
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ ;
+
+ self.timeout(15000);
+
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/treatments';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}?token=${self.token.delete}`;
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.delete(`${self.url}/FAKE_IDENTIFIER`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.delete(`/api/v3/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+});
+
diff --git a/tests/api3.generic.workflow.test.js b/tests/api3.generic.workflow.test.js
new file mode 100644
index 00000000000..36c94a00f74
--- /dev/null
+++ b/tests/api3.generic.workflow.test.js
@@ -0,0 +1,257 @@
+/* eslint require-atomic-updates: 0 */
+'use strict';
+
+require('should');
+
+describe('Generic REST API3', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , opTools = require('../lib/api3/shared/operationTools')
+ , utils = require('./fixtures/api3/utils')
+ ;
+
+ utils.randomString('32', 'aA#'); // let's have a brand new identifier for your testing document
+ self.urlLastModified = '/api/v3/lastModified';
+ self.historyTimestamp = 0;
+
+ self.docOriginal = {
+ eventType: 'Correction Bolus',
+ insulin: 1,
+ date: (new Date()).getTime(),
+ app: testConst.TEST_APP,
+ device: testConst.TEST_DEVICE
+ };
+ self.identifier = opTools.calculateIdentifier(self.docOriginal);
+ self.docOriginal.identifier = self.identifier;
+
+ this.timeout(30000);
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.urlCol = '/api/v3/treatments';
+ self.urlResource = self.urlCol + '/' + self.identifier;
+ self.urlHistory = self.urlCol + '/history';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}?token=${self.token.create}`;
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ self.checkHistoryExistence = async function checkHistoryExistence (assertions) {
+
+ let res = await self.instance.get(`${self.urlHistory}/${self.historyTimestamp}?token=${self.token.read}`)
+ .expect(200);
+
+ res.body.length.should.be.above(0);
+ res.body.should.matchAny(value => {
+ value.identifier.should.be.eql(self.identifier);
+ value.srvModified.should.be.above(self.historyTimestamp);
+
+ if (typeof(assertions) === 'function') {
+ assertions(value);
+ }
+
+ self.historyTimestamp = value.srvModified;
+ });
+ };
+
+
+ it('LAST MODIFIED to get actual server timestamp', async () => {
+ let res = await self.instance.get(`${self.urlLastModified}?token=${self.token.read}`)
+ .expect(200);
+
+ self.historyTimestamp = res.body.collections.treatments;
+ if (!self.historyTimestamp) {
+ self.historyTimestamp = res.body.srvDate - (10 * 60 * 1000);
+ }
+ self.historyTimestamp.should.be.aboveOrEqual(testConst.YEAR_2019);
+ });
+
+
+ it('STATUS to get actual server timestamp', async () => {
+ let res = await self.instance.get(`/api/v3/status?token=${self.token.read}`)
+ .expect(200);
+
+ self.historyTimestamp = res.body.srvDate;
+ self.historyTimestamp.should.be.aboveOrEqual(testConst.YEAR_2019);
+ });
+
+
+ it('READ of not existing document is not found', async () => {
+ await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(404);
+ });
+
+
+ it('SEARCH of not existing document (not found)', async () => {
+ let res = await self.instance.get(`${self.urlCol}?token=${self.token.read}`)
+ .query({ 'identifier_eq': self.identifier })
+ .expect(200);
+
+ res.body.should.have.length(0);
+ });
+
+
+ it('DELETE of not existing document is not found', async () => {
+ await self.instance.delete(`${self.urlResource}?token=${self.token.delete}`)
+ .expect(404);
+ });
+
+
+ it('CREATE new document', async () => {
+ await self.instance.post(`${self.urlCol}?token=${self.token.create}`)
+ .send(self.docOriginal)
+ .expect(201);
+ });
+
+
+ it('READ existing document', async () => {
+ let res = await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(200);
+
+ res.body.should.containEql(self.docOriginal);
+ self.docActual = res.body;
+
+ if (self.historyTimestamp >= self.docActual.srvModified) {
+ self.historyTimestamp = self.docActual.srvModified - 1;
+ }
+ });
+
+
+ it('SEARCH existing document (found)', async () => {
+ let res = await self.instance.get(`${self.urlCol}?token=${self.token.read}`)
+ .query({ 'identifier$eq': self.identifier })
+ .expect(200);
+
+ res.body.length.should.be.above(0);
+ res.body.should.matchAny(value => {
+ value.identifier.should.be.eql(self.identifier);
+ });
+ });
+
+
+ it('new document in HISTORY', async () => {
+ await self.checkHistoryExistence();
+ });
+
+
+ it('UPDATE document', async () => {
+ self.docActual.insulin = 0.5;
+
+ await self.instance.put(`${self.urlResource}?token=${self.token.update}`)
+ .send(self.docActual)
+ .expect(204);
+
+ self.docActual.subject = self.subject.apiUpdate.name;
+ });
+
+
+ it('document changed in HISTORY', async () => {
+ await self.checkHistoryExistence();
+ });
+
+
+ it('document changed in READ', async () => {
+ let res = await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(200);
+
+ delete self.docActual.srvModified;
+ res.body.should.containEql(self.docActual);
+ self.docActual = res.body;
+ });
+
+
+ it('PATCH document', async () => {
+ self.docActual.carbs = 5;
+ self.docActual.insulin = 0.4;
+
+ await self.instance.patch(`${self.urlResource}?token=${self.token.update}`)
+ .send({ 'carbs': self.docActual.carbs, 'insulin': self.docActual.insulin })
+ .expect(204);
+ });
+
+
+ it('document changed in HISTORY', async () => {
+ await self.checkHistoryExistence();
+ });
+
+
+ it('document changed in READ', async () => {
+ let res = await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(200);
+
+ delete self.docActual.srvModified;
+ res.body.should.containEql(self.docActual);
+ self.docActual = res.body;
+ });
+
+
+ it('soft DELETE', async () => {
+ await self.instance.delete(`${self.urlResource}?token=${self.token.delete}`)
+ .expect(204);
+ });
+
+
+ it('READ of deleted is gone', async () => {
+ await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(410);
+ });
+
+
+
+ it('SEARCH of deleted document missing it', async () => {
+ let res = await self.instance.get(`${self.urlCol}?token=${self.token.read}`)
+ .query({ 'identifier_eq': self.identifier })
+ .expect(200);
+
+ res.body.should.have.length(0);
+ });
+
+
+ it('document deleted in HISTORY', async () => {
+ await self.checkHistoryExistence(value => {
+ value.isValid.should.be.eql(false);
+ });
+ });
+
+
+ it('permanent DELETE', async () => {
+ await self.instance.delete(`${self.urlResource}?token=${self.token.delete}`)
+ .query({ 'permanent': 'true' })
+ .expect(204);
+ });
+
+
+ it('READ of permanently deleted is not found', async () => {
+ await self.instance.get(`${self.urlResource}?token=${self.token.read}`)
+ .expect(404);
+ });
+
+
+ it('document permanently deleted not in HISTORY', async () => {
+ let res = await self.instance.get(`${self.urlHistory}/${self.historyTimestamp}?token=${self.token.read}`);
+
+ if (res.status === 200) {
+ res.body.should.matchEach(value => {
+ value.identifier.should.not.be.eql(self.identifier);
+ });
+ } else {
+ res.status.should.equal(204);
+ }
+ });
+
+});
+
diff --git a/tests/api3.patch.test.js b/tests/api3.patch.test.js
new file mode 100644
index 00000000000..38850b46ad5
--- /dev/null
+++ b/tests/api3.patch.test.js
@@ -0,0 +1,219 @@
+/* eslint require-atomic-updates: 0 */
+'use strict';
+
+require('should');
+
+describe('API3 PATCH', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , opTools = require('../lib/api3/shared/operationTools')
+ ;
+
+ self.validDoc = {
+ date: (new Date()).getTime(),
+ utcOffset: -180,
+ app: testConst.TEST_APP,
+ device: testConst.TEST_DEVICE,
+ eventType: 'Correction Bolus',
+ insulin: 0.3
+ };
+ self.validDoc.identifier = opTools.calculateIdentifier(self.validDoc);
+
+ self.timeout(15000);
+
+
+ /**
+ * Get document detail for futher processing
+ */
+ self.get = async function get (identifier) {
+ let res = await self.instance.get(`${self.url}/${identifier}?token=${self.token.read}`)
+ .expect(200);
+
+ return res.body;
+ };
+
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/treatments';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}/${self.validDoc.identifier}?token=${self.token.update}`;
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.patch(`${self.url}/FAKE_IDENTIFIER`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.patch(`/api/v3/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should not found not existing document', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+
+ // now let's insert the document for further patching
+ res = await self.instance.post(`${self.url}?token=${self.token.create}`)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should reject identifier alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { identifier: 'MODIFIED'}))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field identifier cannot be modified by the client');
+ });
+
+
+ it('should reject date alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: self.validDoc.date + 10000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field date cannot be modified by the client');
+ });
+
+
+ it('should reject utcOffset alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: self.utcOffset - 120 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field utcOffset cannot be modified by the client');
+ });
+
+
+ it('should reject eventType alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { eventType: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field eventType cannot be modified by the client');
+ });
+
+
+ it('should reject device alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { device: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field device cannot be modified by the client');
+ });
+
+
+ it('should reject app alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { app: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field app cannot be modified by the client');
+ });
+
+
+ it('should reject srvCreated alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { srvCreated: self.validDoc.date - 10000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field srvCreated cannot be modified by the client');
+ });
+
+
+ it('should reject subject alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { subject: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field subject cannot be modified by the client');
+ });
+
+
+ it('should reject srvModified alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { srvModified: self.validDoc.date - 100000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field srvModified cannot be modified by the client');
+ });
+
+
+ it('should reject modifiedBy alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { modifiedBy: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field modifiedBy cannot be modified by the client');
+ });
+
+
+ it('should reject isValid alteration', async () => {
+ let res = await self.instance.patch(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { isValid: false }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field isValid cannot be modified by the client');
+ });
+
+
+ it('should patch document', async () => {
+ self.validDoc.carbs = 10;
+
+ let res = await self.instance.patch(self.urlToken)
+ .send(self.validDoc)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ let body = await self.get(self.validDoc.identifier);
+ body.carbs.should.equal(10);
+ body.insulin.should.equal(0.3);
+ body.subject.should.equal(self.subject.apiCreate.name);
+ body.modifiedBy.should.equal(self.subject.apiUpdate.name);
+ });
+
+});
+
diff --git a/tests/api3.read.test.js b/tests/api3.read.test.js
new file mode 100644
index 00000000000..b18b0225bb9
--- /dev/null
+++ b/tests/api3.read.test.js
@@ -0,0 +1,180 @@
+/* eslint require-atomic-updates: 0 */
+/* global should */
+'use strict';
+
+require('should');
+
+describe('API3 READ', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , opTools = require('../lib/api3/shared/operationTools')
+ ;
+
+ self.validDoc = {
+ date: (new Date()).getTime(),
+ app: testConst.TEST_APP,
+ device: testConst.TEST_DEVICE,
+ uploaderBattery: 58
+ };
+ self.validDoc.identifier = opTools.calculateIdentifier(self.validDoc);
+
+ self.timeout(15000);
+
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/devicestatus';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.get(`${self.url}/FAKE_IDENTIFIER`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.get(`/api/v3/NOT_EXIST/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should not found not existing document', async () => {
+ await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .expect(404);
+ });
+
+
+ it('should read just created document', async () => {
+ let res = await self.instance.post(`${self.url}?token=${self.token.create}`)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+
+ res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .expect(200);
+
+ res.body.should.containEql(self.validDoc);
+ res.body.should.have.property('srvCreated').which.is.a.Number();
+ res.body.should.have.property('srvModified').which.is.a.Number();
+ res.body.should.have.property('subject');
+ self.validDoc.subject = res.body.subject; // let's store subject for later tests
+ });
+
+
+ it('should contain only selected fields', async () => {
+ let res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?fields=date,device,subject&token=${self.token.read}`)
+ .expect(200);
+
+ const correct = {
+ date: self.validDoc.date,
+ device: self.validDoc.device,
+ subject: self.validDoc.subject
+ };
+ res.body.should.eql(correct);
+ });
+
+
+ it('should contain all fields', async () => {
+ let res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?fields=_all&token=${self.token.read}`)
+ .expect(200);
+
+ for (let fieldName of ['app', 'date', 'device', 'identifier', 'srvModified', 'uploaderBattery', 'subject']) {
+ res.body.should.have.property(fieldName);
+ }
+ });
+
+
+ it('should not send unmodified document since', async () => {
+ let res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .set('If-Modified-Since', new Date(new Date().getTime() + 1000).toUTCString())
+ .expect(304);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should send modified document since', async () => {
+ let res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .set('If-Modified-Since', new Date(new Date(self.validDoc.date).getTime() - 1000).toUTCString())
+ .expect(200);
+
+ res.body.should.containEql(self.validDoc);
+ });
+
+
+ it('should recognize softly deleted document', async () => {
+ let res = await self.instance.delete(`${self.url}/${self.validDoc.identifier}?token=${self.token.delete}`)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .expect(410);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should not found permanently deleted document', async () => {
+ let res = await self.instance.delete(`${self.url}/${self.validDoc.identifier}?permanent=true&token=${self.token.delete}`)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ res = await self.instance.get(`${self.url}/${self.validDoc.identifier}?token=${self.token.read}`)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should found document created by APIv1', async () => {
+
+ const doc = Object.assign({}, self.validDoc, {
+ created_at: new Date(self.validDoc.date).toISOString()
+ });
+ delete doc.identifier;
+
+ self.instance.ctx.devicestatus.create([doc], async (err) => { // let's insert the document in APIv1's way
+ should.not.exist(err);
+ const identifier = doc._id.toString();
+ delete doc._id;
+
+ let res = await self.instance.get(`${self.url}/${identifier}?token=${self.token.read}`)
+ .expect(200);
+
+ res.body.should.containEql(doc);
+
+ res = await self.instance.delete(`${self.url}/${identifier}?permanent=true&token=${self.token.delete}`)
+ .expect(204);
+
+ res.body.should.be.empty();
+ });
+ });
+
+
+});
+
diff --git a/tests/api3.search.test.js b/tests/api3.search.test.js
new file mode 100644
index 00000000000..dae0ebaaf34
--- /dev/null
+++ b/tests/api3.search.test.js
@@ -0,0 +1,261 @@
+/* eslint require-atomic-updates: 0 */
+/* global should */
+'use strict';
+
+require('should');
+
+describe('API3 SEARCH', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , opTools = require('../lib/api3/shared/operationTools')
+ ;
+
+ self.docs = testConst.SAMPLE_ENTRIES;
+
+ self.timeout(15000);
+
+
+ /**
+ * Get document detail for futher processing
+ */
+ self.get = function get (identifier, done) {
+ self.instance.get(`${self.url}/${identifier}?token=${self.token.read}`)
+ .expect(200)
+ .end((err, res) => {
+ should.not.exist(err);
+ done(res.body);
+ });
+ };
+
+
+ /**
+ * Create given document in a promise
+ */
+ self.create = (doc) => new Promise((resolve) => {
+ doc.identifier = opTools.calculateIdentifier(doc);
+ self.instance.post(`${self.url}?token=${self.token.all}`)
+ .send(doc)
+ .end((err) => {
+ should.not.exist(err);
+ self.get(doc.identifier, resolve);
+ });
+ });
+
+
+ before(async () => {
+ self.testStarted = new Date();
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/entries';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}?token=${self.token.read}`;
+ self.urlTest = `${self.urlToken}&srvModified$gte=${self.testStarted.getTime()}`;
+
+ const promises = testConst.SAMPLE_ENTRIES.map(doc => self.create(doc));
+ self.docs = await Promise.all(promises);
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.get(self.url)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.get(`/api/v3/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should found at least 10 documents', async () => {
+ let res = await self.instance.get(self.urlToken)
+ .expect(200);
+
+ res.body.length.should.be.aboveOrEqual(self.docs.length);
+ });
+
+
+ it('should found at least 10 documents from test start', async () => {
+ let res = await self.instance.get(self.urlTest)
+ .expect(200);
+
+ res.body.length.should.be.aboveOrEqual(self.docs.length);
+ });
+
+
+ it('should reject invalid limit - not a number', async () => {
+ let res = await self.instance.get(`${self.urlToken}&limit=INVALID`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter limit out of tolerance');
+ });
+
+
+ it('should reject invalid limit - negative number', async () => {
+ let res = await self.instance.get(`${self.urlToken}&limit=-1`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter limit out of tolerance');
+ });
+
+
+ it('should reject invalid limit - zero', async () => {
+ let res = await self.instance.get(`${self.urlToken}&limit=0`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter limit out of tolerance');
+ });
+
+
+ it('should accept valid limit', async () => {
+ let res = await self.instance.get(`${self.urlToken}&limit=3`)
+ .expect(200);
+
+ res.body.length.should.be.equal(3);
+ });
+
+
+ it('should reject invalid skip - not a number', async () => {
+ let res = await self.instance.get(`${self.urlToken}&skip=INVALID`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter skip out of tolerance');
+ });
+
+
+ it('should reject invalid skip - negative number', async () => {
+ let res = await self.instance.get(`${self.urlToken}&skip=-5`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter skip out of tolerance');
+ });
+
+
+ it('should reject both sort and sort$desc', async () => {
+ let res = await self.instance.get(`${self.urlToken}&sort=date&sort$desc=created_at`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameters sort and sort_desc cannot be combined');
+ });
+
+
+ it('should sort well by date field', async () => {
+ let res = await self.instance.get(`${self.urlTest}&sort=date`)
+ .expect(200);
+
+ const ascending = res.body;
+ const length = ascending.length;
+ length.should.be.aboveOrEqual(self.docs.length);
+
+ res = await self.instance.get(`${self.urlTest}&sort$desc=date`)
+ .expect(200);
+
+ const descending = res.body;
+ descending.length.should.equal(length);
+
+ for (let i in ascending) {
+ ascending[i].should.eql(descending[length - i - 1]);
+
+ if (i > 0) {
+ ascending[i - 1].date.should.be.lessThanOrEqual(ascending[i].date);
+ }
+ }
+ });
+
+
+ it('should skip documents', async () => {
+ let res = await self.instance.get(`${self.urlToken}&sort=date&limit=8`)
+ .expect(200);
+
+ const fullDocs = res.body;
+ fullDocs.length.should.be.equal(8);
+
+ res = await self.instance.get(`${self.urlToken}&sort=date&skip=3&limit=5`)
+ .expect(200);
+
+ const skipDocs = res.body;
+ skipDocs.length.should.be.equal(5);
+
+ for (let i = 0; i < 3; i++) {
+ skipDocs[i].should.be.eql(fullDocs[i + 3]);
+ }
+ });
+
+
+ it('should project selected fields', async () => {
+ let res = await self.instance.get(`${self.urlToken}&fields=date,app,subject`)
+ .expect(200);
+
+ res.body.forEach(doc => {
+ const docFields = Object.getOwnPropertyNames(doc);
+ docFields.sort().should.be.eql(['app', 'date', 'subject']);
+ });
+ });
+
+
+ it('should project all fields', async () => {
+ let res = await self.instance.get(`${self.urlToken}&fields=_all`)
+ .expect(200);
+
+ res.body.forEach(doc => {
+ Object.getOwnPropertyNames(doc).length.should.be.aboveOrEqual(10);
+ Object.prototype.hasOwnProperty.call(doc, '_id').should.not.be.true();
+ Object.prototype.hasOwnProperty.call(doc, 'identifier').should.be.true();
+ Object.prototype.hasOwnProperty.call(doc, 'srvModified').should.be.true();
+ Object.prototype.hasOwnProperty.call(doc, 'srvCreated').should.be.true();
+ });
+ });
+
+
+ it('should not exceed the limit of docs count', async () => {
+ const apiApp = self.instance.ctx.apiApp
+ , limitBackup = apiApp.get('API3_MAX_LIMIT');
+ apiApp.set('API3_MAX_LIMIT', 5);
+ let res = await self.instance.get(`${self.urlToken}&limit=10`)
+ .expect(400);
+
+ res.body.status.should.be.equal(400);
+ res.body.message.should.be.equal('Parameter limit out of tolerance');
+ apiApp.set('API3_MAX_LIMIT', limitBackup);
+ });
+
+
+ it('should respect the ceiling (hard) limit of docs', async () => {
+ const apiApp = self.instance.ctx.apiApp
+ , limitBackup = apiApp.get('API3_MAX_LIMIT');
+ apiApp.set('API3_MAX_LIMIT', 5);
+ let res = await self.instance.get(`${self.urlToken}`)
+ .expect(200);
+
+ res.body.length.should.be.equal(5);
+ apiApp.set('API3_MAX_LIMIT', limitBackup);
+ });
+
+});
+
diff --git a/tests/api3.security.test.js b/tests/api3.security.test.js
new file mode 100644
index 00000000000..4cdc8e22b21
--- /dev/null
+++ b/tests/api3.security.test.js
@@ -0,0 +1,189 @@
+/* eslint require-atomic-updates: 0 */
+'use strict';
+
+const request = require('supertest')
+ , apiConst = require('../lib/api3/const.json')
+ , semver = require('semver')
+ , moment = require('moment')
+ ;
+require('should');
+
+describe('Security of REST API3', function() {
+ const self = this
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ ;
+
+ this.timeout(30000);
+
+
+ before(async () => {
+ self.http = await instance.create({ useHttps: false });
+ self.https = await instance.create({ });
+
+ let authResult = await authSubject(self.https.ctx.authorization.storage);
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ });
+
+
+ after(() => {
+ self.http.server.close();
+ self.https.server.close();
+ });
+
+
+ it('should require HTTPS', async () => {
+ if (semver.gte(process.version, '10.0.0')) {
+ let res = await request(self.http.baseUrl) // hangs on 8.x.x (no reason why)
+ .get('/api/v3/test')
+ .expect(403);
+
+ res.body.status.should.equal(403);
+ res.body.message.should.equal(apiConst.MSG.HTTP_403_NOT_USING_HTTPS);
+ }
+ });
+
+
+ it('should require Date header', async () => {
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test')
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_MISSING_DATE);
+ });
+
+
+ it('should validate Date header syntax', async () => {
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test')
+ .set('Date', 'invalid date header')
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_BAD_DATE);
+ });
+
+
+ it('should reject Date header out of tolerance', async () => {
+ const oldDate = new Date((new Date() * 1) - 2 * 3600 * 1000)
+ , futureDate = new Date((new Date() * 1) + 2 * 3600 * 1000);
+
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test')
+ .set('Date', oldDate.toUTCString())
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_DATE_OUT_OF_TOLERANCE);
+
+ res = await request(self.https.baseUrl)
+ .get('/api/v3/test')
+ .set('Date',futureDate.toUTCString())
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_DATE_OUT_OF_TOLERANCE);
+ });
+
+
+ it('should reject invalid now ABC', async () => {
+ let res = await request(self.https.baseUrl)
+ .get(`/api/v3/test?now=ABC`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Bad Date header');
+ });
+
+
+ it('should reject invalid now -1', async () => {
+ let res = await request(self.https.baseUrl)
+ .get(`/api/v3/test?now=-1`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Bad Date header');
+ });
+
+
+ it('should reject invalid now - illegal format', async () => {
+ let res = await request(self.https.baseUrl)
+ .get(`/api/v3/test?now=2019-20-60T50:90:90`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Bad Date header');
+ });
+
+
+ it('should require token', async () => {
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test')
+ .set('Date', new Date().toUTCString())
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_MISSING_OR_BAD_TOKEN);
+ });
+
+
+ it('should require valid token', async () => {
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test?token=invalid_token')
+ .set('Date', new Date().toUTCString())
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal(apiConst.MSG.HTTP_401_MISSING_OR_BAD_TOKEN);
+ });
+
+
+ it('should deny subject denied', async () => {
+ let res = await request(self.https.baseUrl)
+ .get('/api/v3/test?token=' + self.subject.denied.accessToken)
+ .set('Date', new Date().toUTCString())
+ .expect(403);
+
+ res.body.status.should.equal(403);
+ res.body.message.should.equal(apiConst.MSG.HTTP_403_MISSING_PERMISSION.replace('{0}', 'api:entries:read'));
+ });
+
+
+ it('should allow subject with read permission', async () => {
+ await request(self.https.baseUrl)
+ .get('/api/v3/test?token=' + self.token.read)
+ .set('Date', new Date().toUTCString())
+ .expect(200);
+ });
+
+
+ it('should accept valid now - epoch in ms', async () => {
+ await request(self.https.baseUrl)
+ .get(`/api/v3/test?token=${self.token.read}&now=${moment().valueOf()}`)
+ .expect(200);
+ });
+
+
+ it('should accept valid now - epoch in seconds', async () => {
+ await request(self.https.baseUrl)
+ .get(`/api/v3/test?token=${self.token.read}&now=${moment().unix()}`)
+ .expect(200);
+ });
+
+
+ it('should accept valid now - ISO 8601', async () => {
+ await request(self.https.baseUrl)
+ .get(`/api/v3/test?token=${self.token.read}&now=${moment().toISOString()}`)
+ .expect(200);
+ });
+
+
+ it('should accept valid now - RFC 2822', async () => {
+ await request(self.https.baseUrl)
+ .get(`/api/v3/test?token=${self.token.read}&now=${moment().utc().format('ddd, DD MMM YYYY HH:mm:ss [GMT]')}`)
+ .expect(200);
+ });
+
+});
\ No newline at end of file
diff --git a/tests/api3.socket.test.js b/tests/api3.socket.test.js
new file mode 100644
index 00000000000..5c2a5cf6461
--- /dev/null
+++ b/tests/api3.socket.test.js
@@ -0,0 +1,178 @@
+/* eslint require-atomic-updates: 0 */
+/* global should */
+'use strict';
+
+require('should');
+
+describe('Socket.IO in REST API3', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , apiConst = require('../lib/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , utils = require('./fixtures/api3/utils')
+ ;
+
+ self.identifier = utils.randomString('32', 'aA#'); // let's have a brand new identifier for your testing document
+
+ self.docOriginal = {
+ identifier: self.identifier,
+ eventType: 'Correction Bolus',
+ insulin: 1,
+ date: (new Date()).getTime(),
+ app: testConst.TEST_APP
+ };
+
+ this.timeout(30000);
+
+ before(async () => {
+ self.instance = await instance.create({
+ storageSocket: true
+ });
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.colName = 'treatments';
+ self.urlCol = `/api/v3/${self.colName}`;
+ self.urlResource = self.urlCol + '/' + self.identifier;
+ self.urlHistory = self.urlCol + '/history';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.socket = self.instance.clientSocket;
+ });
+
+
+ after(() => {
+ if(self.instance && self.instance.clientSocket && self.instance.clientSocket.connected) {
+ self.instance.clientSocket.disconnect();
+ }
+ self.instance.server.close();
+ });
+
+
+ it('should not subscribe without accessToken', done => {
+ self.socket.emit('subscribe', { }, function (data) {
+ data.success.should.not.equal(true);
+ data.message.should.equal(apiConst.MSG.SOCKET_MISSING_OR_BAD_ACCESS_TOKEN);
+ done();
+ });
+ });
+
+
+ it('should not subscribe by invalid accessToken', done => {
+ self.socket.emit('subscribe', { accessToken: 'INVALID' }, function (data) {
+ data.success.should.not.equal(true);
+ data.message.should.equal(apiConst.MSG.SOCKET_MISSING_OR_BAD_ACCESS_TOKEN);
+ done();
+ });
+ });
+
+
+ it('should not subscribe by subject with no rights', done => {
+ self.socket.emit('subscribe', { accessToken: self.token.denied }, function (data) {
+ data.success.should.not.equal(true);
+ data.message.should.equal(apiConst.MSG.SOCKET_UNAUTHORIZED_TO_ANY);
+ done();
+ });
+ });
+
+
+ it('should subscribe by valid accessToken', done => {
+ const cols = ['entries', 'treatments'];
+
+ self.socket.emit('subscribe', {
+ accessToken: self.token.all,
+ collections: cols
+ }, function (data) {
+ data.success.should.equal(true);
+ should(data.collections.sort()).be.eql(cols);
+ done();
+ });
+ });
+
+
+ it('should emit create event on CREATE', done => {
+
+ self.socket.once('create', (event) => {
+ event.colName.should.equal(self.colName);
+ event.doc.should.containEql(self.docOriginal);
+ delete event.doc.subject;
+ self.docActual = event.doc;
+ done();
+ });
+
+ self.instance.post(`${self.urlCol}?token=${self.token.create}`)
+ .send(self.docOriginal)
+ .expect(201)
+ .end((err) => {
+ should.not.exist(err);
+ });
+ });
+
+
+ it('should emit update event on UPDATE', done => {
+
+ self.docActual.insulin = 0.5;
+
+ self.socket.once('update', (event) => {
+ delete self.docActual.srvModified;
+ event.colName.should.equal(self.colName);
+ event.doc.should.containEql(self.docActual);
+ delete event.doc.subject;
+ self.docActual = event.doc;
+ done();
+ });
+
+ self.instance.put(`${self.urlResource}?token=${self.token.update}`)
+ .send(self.docActual)
+ .expect(204)
+ .end((err) => {
+ should.not.exist(err);
+ self.docActual.subject = self.subject.apiUpdate.name;
+ });
+ });
+
+
+ it('should emit update event on PATCH', done => {
+
+ self.docActual.carbs = 5;
+ self.docActual.insulin = 0.4;
+
+ self.socket.once('update', (event) => {
+ delete self.docActual.srvModified;
+ event.colName.should.equal(self.colName);
+ event.doc.should.containEql(self.docActual);
+ delete event.doc.subject;
+ self.docActual = event.doc;
+ done();
+ });
+
+ self.instance.patch(`${self.urlResource}?token=${self.token.update}`)
+ .send({ 'carbs': self.docActual.carbs, 'insulin': self.docActual.insulin })
+ .expect(204)
+ .end((err) => {
+ should.not.exist(err);
+ });
+ });
+
+
+ it('should emit delete event on DELETE', done => {
+
+ self.socket.once('delete', (event) => {
+ event.colName.should.equal(self.colName);
+ event.identifier.should.equal(self.identifier);
+ done();
+ });
+
+ self.instance.delete(`${self.urlResource}?token=${self.token.delete}`)
+ .expect(204)
+ .end((err) => {
+ should.not.exist(err);
+ });
+ });
+
+});
+
diff --git a/tests/api3.update.test.js b/tests/api3.update.test.js
new file mode 100644
index 00000000000..403aadb022e
--- /dev/null
+++ b/tests/api3.update.test.js
@@ -0,0 +1,289 @@
+/* eslint require-atomic-updates: 0 */
+/* global should */
+'use strict';
+
+require('should');
+
+describe('API3 UPDATE', function() {
+ const self = this
+ , testConst = require('./fixtures/api3/const.json')
+ , instance = require('./fixtures/api3/instance')
+ , authSubject = require('./fixtures/api3/authSubject')
+ , utils = require('./fixtures/api3/utils')
+ ;
+
+ self.validDoc = {
+ identifier: utils.randomString('32', 'aA#'),
+ date: (new Date()).getTime(),
+ utcOffset: -180,
+ app: testConst.TEST_APP,
+ eventType: 'Correction Bolus',
+ insulin: 0.3
+ };
+
+ self.timeout(15000);
+
+
+ /**
+ * Get document detail for futher processing
+ */
+ self.get = async function get (identifier) {
+ let res = await self.instance.get(`${self.url}/${identifier}?token=${self.token.read}`)
+ .expect(200);
+
+ return res.body;
+ };
+
+
+ before(async () => {
+ self.instance = await instance.create({});
+
+ self.app = self.instance.app;
+ self.env = self.instance.env;
+ self.url = '/api/v3/treatments';
+
+ let authResult = await authSubject(self.instance.ctx.authorization.storage);
+
+ self.subject = authResult.subject;
+ self.token = authResult.token;
+ self.urlToken = `${self.url}/${self.validDoc.identifier}?token=${self.token.update}`
+ });
+
+
+ after(() => {
+ self.instance.server.close();
+ });
+
+
+ it('should require authentication', async () => {
+ let res = await self.instance.put(`${self.url}/FAKE_IDENTIFIER`)
+ .expect(401);
+
+ res.body.status.should.equal(401);
+ res.body.message.should.equal('Missing or bad access token or JWT');
+ });
+
+
+ it('should not found not existing collection', async () => {
+ let res = await self.instance.put(`/api/v3/NOT_EXIST?token=${self.url}`)
+ .send(self.validDoc)
+ .expect(404);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should require update permission for upsert', async () => {
+ let res = await self.instance.put(`${self.url}/${self.validDoc.identifier}?token=${self.token.update}`)
+ .send(self.validDoc)
+ .expect(403);
+
+ res.body.status.should.equal(403);
+ res.body.message.should.equal('Missing permission api:treatments:create');
+ });
+
+
+ it('should upsert not existing document', async () => {
+ let res = await self.instance.put(`${self.url}/${self.validDoc.identifier}?token=${self.token.all}`)
+ .send(self.validDoc)
+ .expect(201);
+
+ res.body.should.be.empty();
+
+ const lastModified = new Date(res.headers['last-modified']).getTime(); // Last-Modified has trimmed milliseconds
+
+ let body = await self.get(self.validDoc.identifier);
+ body.should.containEql(self.validDoc);
+ should.not.exist(body.modifiedBy);
+
+ const ms = body.srvModified % 1000;
+ (body.srvModified - ms).should.equal(lastModified);
+ (body.srvCreated - ms).should.equal(lastModified);
+ body.subject.should.equal(self.subject.apiAll.name);
+ });
+
+
+ it('should update the document', async () => {
+ self.validDoc.carbs = 10;
+ delete self.validDoc.insulin;
+
+ let res = await self.instance.put(self.urlToken)
+ .send(self.validDoc)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ const lastModified = new Date(res.headers['last-modified']).getTime(); // Last-Modified has trimmed milliseconds
+
+ let body = await self.get(self.validDoc.identifier);
+ body.should.containEql(self.validDoc);
+ should.not.exist(body.insulin);
+ should.not.exist(body.modifiedBy);
+
+ const ms = body.srvModified % 1000;
+ (body.srvModified - ms).should.equal(lastModified);
+ body.subject.should.equal(self.subject.apiUpdate.name);
+ });
+
+
+ it('should update unmodified document since', async () => {
+ const doc = Object.assign({}, self.validDoc, {
+ carbs: 11
+ });
+ let res = await self.instance.put(self.urlToken)
+ .set('If-Unmodified-Since', new Date(new Date().getTime() + 1000).toUTCString())
+ .send(doc)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ let body = await self.get(self.validDoc.identifier);
+ body.should.containEql(doc);
+ });
+
+
+ it('should not update document modified since', async () => {
+ const doc = Object.assign({}, self.validDoc, {
+ carbs: 12
+ });
+ let body = await self.get(doc.identifier);
+ self.validDoc = body;
+
+ let res = await self.instance.put(self.urlToken)
+ .set('If-Unmodified-Since', new Date(new Date(body.srvModified).getTime() - 1000).toUTCString())
+ .send(doc)
+ .expect(412);
+
+ res.body.should.be.empty();
+
+ body = await self.get(doc.identifier);
+ body.should.eql(self.validDoc);
+ });
+
+
+ it('should reject date alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { date: self.validDoc.date + 10000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field date cannot be modified by the client');
+ });
+
+
+ it('should reject utcOffset alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { utcOffset: self.utcOffset - 120 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field utcOffset cannot be modified by the client');
+ });
+
+
+ it('should reject eventType alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { eventType: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field eventType cannot be modified by the client');
+ });
+
+
+ it('should reject device alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { device: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field device cannot be modified by the client');
+ });
+
+
+ it('should reject app alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { app: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field app cannot be modified by the client');
+ });
+
+
+ it('should reject srvCreated alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { srvCreated: self.validDoc.date - 10000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field srvCreated cannot be modified by the client');
+ });
+
+
+ it('should reject subject alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { subject: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field subject cannot be modified by the client');
+ });
+
+
+ it('should reject srvModified alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { srvModified: self.validDoc.date - 100000 }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field srvModified cannot be modified by the client');
+ });
+
+
+ it('should reject modifiedBy alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { modifiedBy: 'MODIFIED' }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field modifiedBy cannot be modified by the client');
+ });
+
+
+ it('should reject isValid alteration', async () => {
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { isValid: false }))
+ .expect(400);
+
+ res.body.status.should.equal(400);
+ res.body.message.should.equal('Field isValid cannot be modified by the client');
+ });
+
+
+ it('should ignore identifier alteration in body', async () => {
+ self.validDoc = await self.get(self.validDoc.identifier);
+
+ let res = await self.instance.put(self.urlToken)
+ .send(Object.assign({}, self.validDoc, { identifier: 'MODIFIED' }))
+ .expect(204);
+
+ res.body.should.be.empty();
+ });
+
+
+ it('should not update deleted document', async () => {
+ let res = await self.instance.delete(`${self.url}/${self.validDoc.identifier}?token=${self.token.delete}`)
+ .expect(204);
+
+ res.body.should.be.empty();
+
+ res = await self.instance.put(self.urlToken)
+ .send(self.validDoc)
+ .expect(410);
+
+ res.body.should.be.empty();
+ });
+
+});
+
diff --git a/tests/fixtures/api3/authSubject.js b/tests/fixtures/api3/authSubject.js
new file mode 100644
index 00000000000..6036103b0e5
--- /dev/null
+++ b/tests/fixtures/api3/authSubject.js
@@ -0,0 +1,94 @@
+'use strict';
+
+const _ = require('lodash');
+
+function createRole (authStorage, name, permissions) {
+
+ return new Promise((resolve, reject) => {
+
+ let role = _.find(authStorage.roles, { name });
+
+ if (role) {
+ resolve(role);
+ }
+ else {
+ authStorage.createRole({
+ "name": name,
+ "permissions": permissions,
+ "notes": ""
+ }, function afterCreate (err) {
+
+ if (err)
+ reject(err);
+
+ role = _.find(authStorage.roles, { name });
+ resolve(role);
+ });
+ }
+ });
+}
+
+
+function createTestSubject (authStorage, subjectName, roles) {
+
+ return new Promise((resolve, reject) => {
+
+ const subjectDbName = 'test-' + subjectName;
+ let subject = _.find(authStorage.subjects, { name: subjectDbName });
+
+ if (subject) {
+ resolve(subject);
+ }
+ else {
+ authStorage.createSubject({
+ "name": subjectDbName,
+ "roles": roles,
+ "notes": ""
+ }, function afterCreate (err) {
+
+ if (err)
+ reject(err);
+
+ subject = _.find(authStorage.subjects, { name: subjectDbName });
+ resolve(subject);
+ });
+ }
+ });
+}
+
+
+async function authSubject (authStorage) {
+
+ await createRole(authStorage, 'apiAll', 'api:*:*');
+ await createRole(authStorage, 'apiAdmin', 'api:*:admin');
+ await createRole(authStorage, 'apiCreate', 'api:*:create');
+ await createRole(authStorage, 'apiRead', 'api:*:read');
+ await createRole(authStorage, 'apiUpdate', 'api:*:update');
+ await createRole(authStorage, 'apiDelete', 'api:*:delete');
+
+ const subject = {
+ apiAll: await createTestSubject(authStorage, 'apiAll', ['apiAll']),
+ apiAdmin: await createTestSubject(authStorage, 'apiAdmin', ['apiAdmin']),
+ apiCreate: await createTestSubject(authStorage, 'apiCreate', ['apiCreate']),
+ apiRead: await createTestSubject(authStorage, 'apiRead', ['apiRead']),
+ apiUpdate: await createTestSubject(authStorage, 'apiUpdate', ['apiUpdate']),
+ apiDelete: await createTestSubject(authStorage, 'apiDelete', ['apiDelete']),
+ admin: await createTestSubject(authStorage, 'admin', ['admin']),
+ readable: await createTestSubject(authStorage, 'readable', ['readable']),
+ denied: await createTestSubject(authStorage, 'denied', ['denied'])
+ };
+
+ const token = {
+ all: subject.apiAll.accessToken,
+ admin: subject.apiAdmin.accessToken,
+ create: subject.apiCreate.accessToken,
+ read: subject.apiRead.accessToken,
+ update: subject.apiUpdate.accessToken,
+ delete: subject.apiDelete.accessToken,
+ denied: subject.denied.accessToken
+ };
+
+ return {subject, token};
+}
+
+module.exports = authSubject;
\ No newline at end of file
diff --git a/tests/fixtures/api3/const.json b/tests/fixtures/api3/const.json
new file mode 100644
index 00000000000..a0acf37cfee
--- /dev/null
+++ b/tests/fixtures/api3/const.json
@@ -0,0 +1,138 @@
+{
+ "YEAR_2019": 1546304400000,
+ "YEAR_2050": 2524611600000,
+ "TEST_APP": "cgm-remote-monitor.test",
+ "TEST_DEVICE": "Samsung XCover 4-123456735643809",
+
+ "SAMPLE_ENTRIES": [
+ {
+ "date": 1491717830000.0,
+ "device": "dexcom",
+ "direction": "FortyFiveUp",
+ "filtered": 167584,
+ "noise": 2,
+ "rssi": 183,
+ "sgv": 149,
+ "type": "sgv",
+ "unfiltered": 171584,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491718130000.0,
+ "device": "dexcom",
+ "direction": "FortyFiveUp",
+ "filtered": 170656,
+ "noise": 2,
+ "rssi": 181,
+ "sgv": 152,
+ "type": "sgv",
+ "unfiltered": 175776,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491718430000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 173536,
+ "noise": 2,
+ "rssi": 185,
+ "sgv": 155,
+ "type": "sgv",
+ "unfiltered": 180864,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491718730000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 177120,
+ "noise": 2,
+ "rssi": 186,
+ "sgv": 159,
+ "type": "sgv",
+ "unfiltered": 182080,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491719030000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 181088,
+ "noise": 2,
+ "rssi": 165,
+ "sgv": 163,
+ "type": "sgv",
+ "unfiltered": 186912,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491719330000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 184736,
+ "noise": 1,
+ "rssi": 162,
+ "sgv": 170,
+ "type": "sgv",
+ "unfiltered": 188512,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491719630000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 187776,
+ "noise": 1,
+ "rssi": 175,
+ "sgv": 175,
+ "type": "sgv",
+ "unfiltered": 192608,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491719930000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 190816,
+ "noise": 1,
+ "rssi": 181,
+ "sgv": 179,
+ "type": "sgv",
+ "unfiltered": 196640,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491720230000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 194016,
+ "noise": 1,
+ "rssi": 203,
+ "sgv": 181,
+ "type": "sgv",
+ "unfiltered": 199008,
+ "app": "cgm-remote-monitor.test"
+ },
+
+ {
+ "date": 1491720530000.0,
+ "device": "dexcom",
+ "direction": "Flat",
+ "filtered": 197536,
+ "noise": 1,
+ "rssi": 184,
+ "sgv": 186,
+ "type": "sgv",
+ "unfiltered": 203296,
+ "app": "cgm-remote-monitor.test"
+ }
+ ]
+}
\ No newline at end of file
diff --git a/tests/fixtures/api3/instance.js b/tests/fixtures/api3/instance.js
new file mode 100644
index 00000000000..a7693ab3c40
--- /dev/null
+++ b/tests/fixtures/api3/instance.js
@@ -0,0 +1,163 @@
+'use strict';
+
+var fs = require('fs')
+ , language = require('../../../lib/language')()
+ , api = require('../../../lib/api3/')
+ , http = require('http')
+ , https = require('https')
+ , request = require('supertest')
+ , websocket = require('../../../lib/server/websocket')
+ , io = require('socket.io-client')
+ ;
+
+function configure () {
+ const self = { };
+
+ self.prepareEnv = function prepareEnv({ apiSecret, useHttps, authDefaultRoles, enable }) {
+
+ if (useHttps) {
+ process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
+ }
+ else {
+ process.env.INSECURE_USE_HTTP = true;
+ }
+ process.env.API_SECRET = apiSecret;
+
+ process.env.HOSTNAME = 'localhost';
+ const env = require('../../../env')();
+
+ if (useHttps) {
+ env.ssl = {
+ key: fs.readFileSync(__dirname + '/localhost.key'),
+ cert: fs.readFileSync(__dirname + '/localhost.crt')
+ };
+ }
+
+ env.settings.authDefaultRoles = authDefaultRoles;
+ env.settings.enable = enable;
+
+ return env;
+ };
+
+
+ self.addSecuredOperations = function addSecuredOperations (instance) {
+
+ instance.get = (url) => request(instance.baseUrl).get(url).set('Date', new Date().toUTCString());
+
+ instance.post = (url) => request(instance.baseUrl).post(url).set('Date', new Date().toUTCString());
+
+ instance.put = (url) => request(instance.baseUrl).put(url).set('Date', new Date().toUTCString());
+
+ instance.patch = (url) => request(instance.baseUrl).patch(url).set('Date', new Date().toUTCString());
+
+ instance.delete = (url) => request(instance.baseUrl).delete(url).set('Date', new Date().toUTCString());
+ };
+
+
+ self.bindSocket = function bindSocket (storageSocket, instance) {
+
+ return new Promise(function (resolve, reject) {
+ if (!storageSocket) {
+ resolve();
+ }
+ else {
+ let socket = io(`${instance.baseUrl}/storage`, {
+ origins:"*",
+ transports: ['websocket', 'flashsocket', 'polling'],
+ rejectUnauthorized: false
+ });
+
+ socket.on('connect', function () {
+ resolve(socket);
+ });
+ socket.on('connect_error', function (error) {
+ console.error(error);
+ reject(error);
+ });
+ }
+ });
+ };
+
+
+ self.unbindSocket = function unbindSocket (instance) {
+ if (instance.clientSocket.connected) {
+ instance.clientSocket.disconnect();
+ }
+ };
+
+ /*
+ * Create new web server instance for testing purposes
+ */
+ self.create = function createHttpServer ({
+ apiSecret = 'this is my long pass phrase',
+ disableSecurity = false,
+ useHttps = true,
+ authDefaultRoles = '',
+ enable = ['careportal', 'api'],
+ storageSocket = null
+ }) {
+
+ return new Promise(function (resolve, reject) {
+
+ try {
+ let instance = { },
+ hasBooted = false
+ ;
+
+ instance.env = self.prepareEnv({ apiSecret, useHttps, authDefaultRoles, enable });
+
+ self.wares = require('../../../lib/middleware/')(instance.env);
+ instance.app = require('express')();
+ instance.app.enable('api');
+
+ require('../../../lib/server/bootevent')(instance.env, language).boot(function booted (ctx) {
+ instance.ctx = ctx;
+ instance.ctx.ddata = require('../../../lib/data/ddata')();
+ instance.ctx.apiApp = api(instance.env, ctx);
+
+ if (disableSecurity) {
+ instance.ctx.apiApp.set('API3_SECURITY_ENABLE', false);
+ }
+
+ instance.app.use('/api/v3', instance.ctx.apiApp);
+
+ const transport = useHttps ? https : http;
+
+ instance.server = transport.createServer(instance.env.ssl || { }, instance.app).listen(0);
+ instance.env.PORT = instance.server.address().port;
+
+ instance.baseUrl = `${useHttps ? 'https' : 'http'}://${instance.env.HOSTNAME}:${instance.env.PORT}`;
+
+ self.addSecuredOperations(instance);
+
+ websocket(instance.env, instance.ctx, instance.server);
+
+ self.bindSocket(storageSocket, instance)
+ .then((socket) => {
+ instance.clientSocket = socket;
+
+ console.log(`Started ${useHttps ? 'SSL' : 'HTTP'} instance on ${instance.baseUrl}`);
+ hasBooted = true;
+ resolve(instance);
+ })
+ .catch((reason) => {
+ console.error(reason);
+ reject(reason);
+ });
+ });
+
+ setTimeout(function watchDog() {
+ if (!hasBooted)
+ reject('timeout');
+ }, 30000);
+
+ } catch (err) {
+ reject(err);
+ }
+ });
+ };
+
+ return self;
+}
+
+module.exports = configure();
\ No newline at end of file
diff --git a/tests/fixtures/api3/localhost.crt b/tests/fixtures/api3/localhost.crt
new file mode 100644
index 00000000000..21a2a39b0a4
--- /dev/null
+++ b/tests/fixtures/api3/localhost.crt
@@ -0,0 +1,18 @@
+-----BEGIN CERTIFICATE-----
+MIIC+zCCAeOgAwIBAgIJAIx0y57dTqDpMA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
+BAMMCWxvY2FsaG9zdDAeFw0xOTAyMDQxOTM1MDhaFw0yOTAyMDExOTM1MDhaMBQx
+EjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
+ggEBAKCeBaqAJU+nrzNUZMsD1jYQpmcw8+6tG69KQY2XmqMsaPupo2ArwUlYD3pm
+F1HTf9Lkq8u07rlUyMaSSRYrY56vPrMWGSK5Elm4kF8DNS4b/55KwZC+YQM0ZuJK
+wSM6WX4G7JwV936HKJAT+Ec+8Ofq3GQzA9+Z4x2zMwNGC8AghtPjsCk68ORCmr+5
+fdCdC1Rz9hE92Nmofi8e1hUTeZmFROx8hcYRhxYXLIWVxALc/t8yY3MZfsRuZXcP
+/3PageAn0ecxhqlWBY23GDQx7OSEZxSEPgqxnAHQfQXIrPRjMkFNHeMM7HTvITAG
+VCc99zEG3Jy5hatm+RAajdWBH4sCAwEAAaNQME4wHQYDVR0OBBYEFJJVZn5Y91O7
+JUKeHW4La8eseKKwMB8GA1UdIwQYMBaAFJJVZn5Y91O7JUKeHW4La8eseKKwMAwG
+A1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEBAFOU19t9h6C1Hakkik/93kun
+pwG7v8VvDPjKECR5KlNPKNZUOQaiMAVHgNwPWV8q+qvfydzIpDrTd/O5eOaOduLx
+gDVDj078Q05j17RUC+ct5yQ6lPgEHlnkI0Zr/hgFyNC+mtK7oIm6BT8wSSRbv7AG
+3wQzCA5UvW/BQ8rtNZSC42Jyr0BR0ZS9Fo3Gc4v/nZJlgkiBvU2gKVQ7VRKxybCn
+0hDghVwTfBPq7PKmupLX82ktwhYpDJZXCsOVfq9mF6nbQ6b0MieXFD+7cBlEXb1e
+3VgtVzKYyqh/Oex4HfMThzAJZSWa0E4FShr5XdTdIc3nB4Vgbsis5l9Yrcp3Xo4=
+-----END CERTIFICATE-----
diff --git a/tests/fixtures/api3/localhost.key b/tests/fixtures/api3/localhost.key
new file mode 100644
index 00000000000..2486c15fefe
--- /dev/null
+++ b/tests/fixtures/api3/localhost.key
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEAoJ4FqoAlT6evM1RkywPWNhCmZzDz7q0br0pBjZeaoyxo+6mj
+YCvBSVgPemYXUdN/0uSry7TuuVTIxpJJFitjnq8+sxYZIrkSWbiQXwM1Lhv/nkrB
+kL5hAzRm4krBIzpZfgbsnBX3focokBP4Rz7w5+rcZDMD35njHbMzA0YLwCCG0+Ow
+KTrw5EKav7l90J0LVHP2ET3Y2ah+Lx7WFRN5mYVE7HyFxhGHFhcshZXEAtz+3zJj
+cxl+xG5ldw//c9qB4CfR5zGGqVYFjbcYNDHs5IRnFIQ+CrGcAdB9Bcis9GMyQU0d
+4wzsdO8hMAZUJz33MQbcnLmFq2b5EBqN1YEfiwIDAQABAoIBADoh95sGVnrGDjtd
+yD1SXi2jSRcAOMmiDesbzS4aOPXmFPlBJMiiDYsmPDPoz3fmPNVvvl40VlLtxN1a
+BOnpOl0swFzBGsfehC3FBzvcRVsy9wmrtPNWdHZceQBeXhkJ/WoHx4uWx8Ub1iqP
+j8T5mufVsX7yl+xOHk2ZllUQ/R/EEz9x00pkiH8Vsn8DhFI5KNqgi4n4c36T3vrn
+MjTp+1o7bJ/cEnvXLi+IG2CO5y5hVbu3iKb+71YOGc6f/AJVzZ3MegC3KMFho9lh
+DbDzumMuW8fZNyBfslXXoOr6oDqNq92n/jC/2hR8Xlth/aafisJiIVGydeVdDXhM
+gDjdroECgYEAy3hXuo/Q1acncInGhIJvHjS/sVShP2epHz9zp8XuWl4NCuGP5V2c
+jLT0hDW+ZKTUFweK9sQJNta81gs4pYc+2HGI8RP65XW4vgesNoKbBcE9xhEq0HMX
+KN3/MJiwkNkM95T3nWqulhzNszhgNbZDMAU3Ule+o4n8udwOlFCTeXMCgYEAyhV4
+PoL3wp05BY0ssyKEqld3EqHNlPdQeJe1Dg9LSBy+3Z9sNngRD1/FuTo7RX6UY0FH
+MaSI1JwhHSQ+2GNkqdMvVAilTXIDRw8vU9B77bYiHjny8+vMU06I9V3cJ57bNfmR
+NUJtPmGO9xQ5UYxhP9rFOcI4MIecSzu1tvqiG4kCgYB01NoS7sdsFrnnvcS2i6rA
+PmufqEeaf6w1nBqNyHJPg1eb2t7kRfdBOBp6291CLv71Zkhd3zynN3BguzrAmUL1
+x2Npgh57qTf2LbOt7RqUmFwfIfZikONIfQgt4E7qLSdr9iakRgCPg2R9ty5PSSOV
+LDmS131IrE/obLoWYZn8jwKBgQDIaAxMahONA+CFueCHcgcA6yah6qZ3QeCjB0g9
+vjsZM7CxFqX5So8YoRDzxWT8YTCFUjppZ9NujbtlLAnLDJ7KsC2yd7R/Hj9T3CJC
+S3JrZoFlWnCvJ7wFLdAzDTcEb8zTNUGlANBX2eYu7/Z8Aex7p9iJlCunLQV5sqhd
+4yaaiQKBgQCERrz1XcJpM8S93nXdAv3Nn1bwA1V/ylx42DRxNEBl2JZQ1sQeqN36
+JvXPXhVZ3vTQDhVUqcVgqJIAb2xMviIVBnssOq3+pi/hOs13rakJf4AOulZ/3Si7
+HSLdymfQAMEKczU2261kw4pjPwiurkjAFWbQG2C8RGE/rR2y38PkDg==
+-----END RSA PRIVATE KEY-----
diff --git a/tests/fixtures/api3/utils.js b/tests/fixtures/api3/utils.js
new file mode 100644
index 00000000000..942f948c10e
--- /dev/null
+++ b/tests/fixtures/api3/utils.js
@@ -0,0 +1,21 @@
+'use strict';
+
+function randomString (length, chars) {
+ let mask = '';
+ if (chars.indexOf('a') > -1) mask += 'abcdefghijklmnopqrstuvwxyz';
+ if (chars.indexOf('A') > -1) mask += 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
+ if (chars.indexOf('#') > -1) mask += '0123456789';
+ if (chars.indexOf('!') > -1) mask += '~`!@#$%^&*()_+-={}[]:";\'<>?,./|\\';
+
+ let result = '';
+
+ for (let i = length; i > 0; --i)
+ result += mask[Math.floor(Math.random() * mask.length)];
+
+ return result;
+}
+
+
+module.exports = {
+ randomString
+};
\ No newline at end of file