[entityanalytics_okta] Initial Release for the Entity Analytics Okta (#…

…6911)

* Initial Release for the Entity Analytics Okta

* Update the changelog entry

* Resolve the comments

* Resolve the comments

.github/CODEOWNERS

@@ -81,6 +81,7 @@
 /packages/elastic_package_registry @elastic/ecosystem
 /packages/elasticsearch @elastic/infra-monitoring-ui
 /packages/enterprisesearch @elastic/infra-monitoring-ui
+/packages/entityanalytics_okta @elastic/security-external-integrations
 /packages/etcd @elastic/obs-infraobs-integrations
 /packages/f5 @elastic/security-external-integrations
 /packages/f5_bigip @elastic/security-external-integrations

packages/entityanalytics_okta/_dev/build/build.yml

@@ -0,0 +1,4 @@
+dependencies:
+  ecs:
+    reference: git@v8.8.0
+    import_mappings: true

packages/entityanalytics_okta/_dev/build/docs/README.md

@@ -0,0 +1,138 @@
+# Okta Entity Analytics
+
+This [Okta Entity Analytics](https://www.okta.com/) integration allows users to securely stream User Entities data to Elastic Security via the REST API. When integrated with Elastic Security, this valuable data can be leveraged within Elastic for risk-scoring scenarios (e.g., context enrichments) and detecting advanced analytics (UBA) use cases.
+
+## Compatibility
+
+This module has been tested against the Core Okta API version **v1**.
+
+## Data streams
+
+The Okta Entity Analytics integration collects one type of data: user.
+
+**User** is used to retrieve all user logs available in an organization. See more details in the API documentation [here](https://developer.okta.com/docs/reference/api/users/#list-users).
+
+## Requirements
+
+- Elastic Agent must be installed.
+- You can install only one Elastic Agent per host.
+- Elastic Agent is required to stream data using Entity Analytics Input and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.
+
+### Installing and managing an Elastic Agent:
+
+You have a few options for installing and managing an Elastic Agent:
+
+### Install a Fleet-managed Elastic Agent (recommended):
+
+With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.
+
+### Install Elastic Agent in standalone mode (advanced users):
+
+With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.
+
+### Install Elastic Agent in a containerized environment:
+
+You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.
+
+There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+The minimum **kibana.version** required is **8.9.0**.
+
+## Setup
+
+### To collect data from Okta, follow the below steps:
+
+- Required URL namespace, which should be preceded by an organization's subdomain (tenant) or configured custom domain.
+- Create an Okta API Token for Authentication. Follow this [guide](https://developer.okta.com/docs/guides/create-an-api-token/main/).
+
+### Enabling the integration in Elastic:
+
+1. In Kibana, go to Management > Integrations.
+2. In the "Search for integrations" search bar, type Okta Entity Analytics.
+3. Click on the "Okta Entity Analytics" integration from the search results.
+4. Click on the Add Okta Entity Analytics Integration button to add the integration.
+5. While adding the integration, add the URL and API Token that we got earlier.
+6. Save the integration by adding other necessary parameters.
+
+## Usage
+
+The Okta provider periodically contacts the Okta API, retrieving updates for users, updates its internal cache of user metadata, and ships updated user metadata to Elasticsearch.
+
+Fetching and shipping updates occurs in one of two processes: **full synchronizations** and **incremental updates**. Full synchronizations will send the entire list of users in state, along with write markers to indicate the start and end of the synchronization event. Incremental updates will only send data for changed users during that event. Changes on a user can come in many forms, whether it be a change to the user’s metadata, or a user was added or deleted. By default, full synchronizations occur every 24 hours and incremental updates occur every 15 minutes. These intervals may be customized to suit your use case.
+
+## Sample Events
+
+A user document:
+
+```json
+{
+  "@timestamp": "2023-07-04T09:57:19.786056-05:00",
+  "event": {
+    "action": "user-discovered"
+  },
+  "okta": {
+    "id": "userid",
+    "status": "RECOVERY",
+    "created": "2023-06-02T09:33:00.189752+09:30",
+    "activated": "0001-01-01T00:00:00Z",
+    "statusChanged": "2023-06-02T09:33:00.189752+09:30",
+    "lastLogin": "2023-06-02T09:33:00.189752+09:30",
+    "lastUpdated": "2023-06-02T09:33:00.189753+09:30",
+    "passwordChanged": "2023-06-02T09:33:00.189753+09:30",
+    "type": {
+      "id": "typeid"
+    },
+    "profile": {
+      "login": "name.surname@example.com",
+      "email": "name.surname@example.com",
+      "firstName": "name",
+      "lastName": "surname"
+    },
+    "credentials": {
+      "password": {},
+      "provider": {
+        "type": "OKTA",
+        "name": "OKTA"
+      }
+    },
+    "_links": {
+      "self": {
+        "href": "https://localhost/api/v1/users/userid"
+      }
+    }
+  },
+  "user": {
+    "id": "userid"
+  },
+  "labels": {
+    "identity_source": "okta-1"
+  }
+}
+```
+
+Full synchronizations will be bounded on either side by "write marker" documents.
+
+```json
+{
+    "@timestamp": "2022-11-04T09:57:19.786056-05:00",
+    "event": {
+        "action": "started",
+        "start": "2022-11-04T09:57:19.786056-05:00"
+    },
+    "labels": {
+        "identity_source": "okta-1"
+    }
+}
+```
+
+## Logs reference
+
+### User
+
+This is the `User` dataset.
+
+#### Example
+
+{{event "user"}}
+
+{{fields "user"}}

packages/entityanalytics_okta/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,17 @@
+version: '2.3'
+services:
+  entityanalytics_okta:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    hostname: trial-xxxxxxx-admin.okta.com
+    ports:
+      - 443
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: "443"
+    command:
+      - http-server
+      - --addr=:443
+      - --config=/files/config.yml
+      - --tls-cert=/files/certificate.crt
+      - --tls-key=/files/private.key

packages/entityanalytics_okta/_dev/deploy/docker/files/certificate.crt

@@ -0,0 +1,20 @@
+-----BEGIN CERTIFICATE-----
+MIIDTjCCAjYCCQDXiI0YQTvKWTANBgkqhkiG9w0BAQsFADBpMQswCQYDVQQGEwJY
+WDEVMBMGA1UEBwwMRGVmYXVsdCBDaXR5MRwwGgYDVQQKDBNEZWZhdWx0IENvbXBh
+bnkgTHRkMSUwIwYDVQQDDBx0cmlhbC14eHh4eHh4LWFkbWluLm9rdGEuY29tMB4X
+DTIzMDYyNzEyNDA1NVoXDTI0MDYyNjEyNDA1NVowaTELMAkGA1UEBhMCWFgxFTAT
+BgNVBAcMDERlZmF1bHQgQ2l0eTEcMBoGA1UECgwTRGVmYXVsdCBDb21wYW55IEx0
+ZDElMCMGA1UEAwwcdHJpYWwteHh4eHh4eC1hZG1pbi5va3RhLmNvbTCCASIwDQYJ
+KoZIhvcNAQEBBQADggEPADCCAQoCggEBANfafZ9Sz4mYMsJBvc94VtQHGRO8g7ne
+LXa1E2rY4e4zVbQxm7EtZw25+MQjJp2zxHntYS6Loa3bfm1g9au97tQk4YlL7Ie8
+9ng1qMhAFqc07Xr7jTpYD1XUKXzJ6m/ExgjHz3ohTVT4+PporR18CrGI1PiLPOFI
+hdgwjYYsz7ghKglWjC3wxXtSOBwFNo0QM0/FHmfuUluEfoEBagPqTESvPsCtrUSz
+0JuWGm4jbacY0BWfTZtZYF81468b9JDDXvFXZLre6WvFbWmw86E/Jv4bdy0RuX3L
+UtMbSdqOBw+QwaLpQn72OuxhDPZVtNjI9ihzLeixTY20C36QTXSoJXcCAwEAATAN
+BgkqhkiG9w0BAQsFAAOCAQEAPAkhHfOVE8SonSRQK9yp8MdVWC7KyqtayHZq7xkj
+UMwX2z0Jk1FQFimXNw3dRLV6yuOW0+ra0opuODV+VwsIj/2FqHKMnPQBUj5jrLui
+TYGHv4RYE5R8YqvphCeCw+Tr3bSMdpuQuRLhWjqZUpSR+yav72ebBXJvxqvOVBTT
+lsJVUm1XPfmbH8yI7Lc4DzIoQmUXdhqLu7mx4kVvLvYEwq1xgK/PZDzPde0uqvZR
+gqF5pq77W705pLc+GYWWTjlDNXIrcXnMuljcGR+UIs5GuGwAaflhBiqLXrKdi8oh
+3pQrTZO28+/Yc7PYFkVZOwvWl/OJ8bKSDFuRuz2eGAWgmQ==
+-----END CERTIFICATE-----

packages/entityanalytics_okta/_dev/deploy/docker/files/config.yml

@@ -0,0 +1,7 @@
+rules:
+  - path: /api/v1/users
+    methods: ["GET"]
+    responses:
+      - status_code: 200
+        body: |
+          [{"id":"00ub0oNGTSWTBKOLGLNR","status":"ACTIVE","created":"2013-06-24T16:39:18.000Z","activated":"2013-06-24T16:39:19.000Z","statusChanged":"2013-06-24T16:39:19.000Z","lastLogin":"2013-06-24T17:39:19.000Z","lastUpdated":"2013-07-02T21:36:25.344Z","passwordChanged":"2013-07-02T21:36:25.344Z","profile":{"firstName":"Isaac","lastName":"Brock","email":"isaac.brock@example.com","login":"isaac.brock@example.com","mobilePhone":"555-415-1337"},"credentials":{"password":{"value":"tlpWENT2m"},"recovery_question":{"question":"Who's a major player in the cowboy scene?","answer":"Annie Oakley"},"provider":{"type":"OKTA","name":"OKTA"}}}]

packages/entityanalytics_okta/_dev/deploy/docker/files/private.key

@@ -0,0 +1,28 @@
+-----BEGIN PRIVATE KEY-----
+MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDX2n2fUs+JmDLC
+Qb3PeFbUBxkTvIO53i12tRNq2OHuM1W0MZuxLWcNufjEIyads8R57WEui6Gt235t
+YPWrve7UJOGJS+yHvPZ4NajIQBanNO16+406WA9V1Cl8yepvxMYIx896IU1U+Pj6
+aK0dfAqxiNT4izzhSIXYMI2GLM+4ISoJVowt8MV7UjgcBTaNEDNPxR5n7lJbhH6B
+AWoD6kxErz7Ara1Es9CblhpuI22nGNAVn02bWWBfNeOvG/SQw17xV2S63ulrxW1p
+sPOhPyb+G3ctEbl9y1LTG0najgcPkMGi6UJ+9jrsYQz2VbTYyPYocy3osU2NtAt+
+kE10qCV3AgMBAAECggEAb8Z7wNIwGE9REZjbgqENTXRzfu3ixJARuOPhrzWqISBC
+JDKALgqVz9zavi55o7ZgascKyQCTc6Uy3c8UyJ9tGjwS8nb3f/7RHsSipzfQwzs8
+MrAqnOjW4BoidbVVihU6e3eEr/iow/S9cWZdJuSB4Rc7OY1YtExKyv2xe6ae1fia
+OhJZxJ5zxCfw0hd0B1w9cgd0nzfKn1xGQktLn/8XbJUUEm9C1PQ6ocpjXV0b+KSg
+5gcFEOW3ndyFvnz1ayBs0VSAlhKo3nffYUUUJoMod+dNRx2UAW9En2NYmJORDu11
+VNUKZjmQWnNODu8QwzOlrlPzmyKdHnyncDm7AZe58QKBgQD++Ar1UuJ9CfQA2lK/
+NSC5L4UhkXt53vZiYy0z+mmox97DYotqSraRiqLk/KUr9Qx8Fyjgyxj7+MScKRiA
+Hu3NEuYeE6VVf+bCf1L1EntaC9rpSuMxLB+qKqW7Ni3oyBypFxWCblPcezWl41Ay
+eSCTAZyfIkJupvVa5EB5VScGiQKBgQDYufQcHCGF9GBUAwRdJY8FbX64mkeIX+YH
+6TmG5mmsYjhbYy2HiWlHQhhWDiaVtzQMKTBgXhV/ISumOwyuXqUiVY7HYHdQN+6G
+ZZmdOIqk+x/7ArW/MIXX56aBoV1K0bCTqs4XSYTsNmECeU8W++SylXMZP5vCbozb
+2ZWiJUrL/wKBgDI8y2P1Fd1PRb2ZRu6X2D/ExjWMBclIjkFqizCPqZvw7cqPsyqZ
+WAmrSFdYFG9p1DDrG0nXmq+jH5IpYsNz1KUwkRjiBBSpo9002zKglp9ycIrBD6zT
+pdWQm6EclsUkEMmcW8ecV3e86yApftbDGRD+I08ToAVPbk6fZLjDd3IJAoGAaLQ9
++S6mC39q4muqug2hMTAbHrh6gqpCCeRlRIUg3IV/ULdzkZ2TXTfTZg/kGESwQofL
+tWL+47J00hSJJdzXzuSi3DQAsKK0+/0t38UnRlmEs9I1WC7rGsnxqYL5/WGc+1pz
+wtBkBXKoH3fX+/SqIpUNTE9swtyB9N0D65g0sKUCgYEA2WAL8t8TICDRY2Iyq2fN
+SdqqOEP3noR+6lSHPWhjFeC5xQXPFKNUDnr9e3KofT3+wdgTqSh77zEdF7+yfR/y
+JEsDVB5ThYVDS2sHvUoPGnLg42i3CauirMiBgmQZgL5h1Un+tzN0jJS79p9iWtJj
+1igypjZrzf2/vRj23rEcPOM=
+-----END PRIVATE KEY-----

packages/entityanalytics_okta/changelog.yml

@@ -0,0 +1,6 @@
+# newer versions go on top
+- version: "0.1.0"
+  changes:
+    - description: Initial Release.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/6911

packages/entityanalytics_okta/data_stream/user/_dev/test/pipeline/test-common-config.yml

@@ -0,0 +1,4 @@
+fields:
+  tags:
+    - preserve_original_event
+    - preserve_duplicate_custom_fields

packages/entityanalytics_okta/data_stream/user/_dev/test/pipeline/test-user.json

@@ -0,0 +1,60 @@
+{
+    "events": [
+        {
+            "@timestamp": "2023-06-13T07:12:17.341Z",
+            "okta": {
+                "id": "00ub0oNGTSWTBKOLGLNR",
+                "status": "ACTIVE",
+                "created": "2013-06-24T16:39:18.000Z",
+                "activated": "2013-06-24T16:39:19.000Z",
+                "statusChanged": "2013-06-24T16:39:19.000Z",
+                "lastLogin": "2013-06-24T17:39:19.000Z",
+                "lastUpdated": "2013-06-27T16:35:28.000Z",
+                "passwordChanged": "2013-06-24T16:39:19.000Z",
+                "profile": {
+                    "login": "isaac.brock@example.com",
+                    "firstName": "Isaac",
+                    "lastName": "Brock",
+                    "nickName": "issac",
+                    "displayName": "Isaac Brock",
+                    "email": "isaac.brock@example.com",
+                    "secondEmail": "isaac@example.org",
+                    "profileUrl": "http://www.example.com/profile",
+                    "preferredLanguage": "en-US",
+                    "userType": "Employee",
+                    "organization": "Okta",
+                    "title": "Director",
+                    "division": "R\u0026D",
+                    "department": "Engineering",
+                    "costCenter": "10",
+                    "employeeNumber": "187",
+                    "mobilePhone": "+1-555-415-1337",
+                    "primaryPhone": "+1-555-514-1337",
+                    "streetAddress": "301 Brannan St.",
+                    "city": "San Francisco",
+                    "state": "CA",
+                    "zipCode": "94107",
+                    "countryCode": "US"
+                },
+                "credentials": {
+                    "provider": {
+                        "type": "OKTA",
+                        "name": "OKTA"
+                    }
+                }
+            },
+            "user": {
+                "id": "00u5tvodynDjUCNKn697"
+            },
+            "event": {
+                "action": "user-modified"
+            },
+            "input": {
+                "type": "entity-analytics"
+            },
+            "ecs": {
+                "version": "8.0.0"
+            }
+        }
+    ]
+}