-
Notifications
You must be signed in to change notification settings - Fork 242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OPSEXP-1369 Add elasticsearch connector #693
Merged
Merged
Changes from all commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
1e997aa
first stub
gionn d9e915d
pod running
gionn 7765bc6
elastic
gionn 065d56a
enable elasticsearch in repo
gionn be0bbc3
fixme
gionn 8fc0f5b
disable sync
gionn 2840423
fix license and solr shared secret
gionn 7c76fbe
fix securecomm
gionn a505b1f
cleanup
gionn 95f24f3
fix pre-commit checks
gionn 0c38fc6
fixup
gionn 2cff507
globals for database secret
gionn 2dd30c1
less hardcoded url
gionn b281777
remove unnecessary ingress and service
gionn 61e49b3
revert
gionn 957f26e
cleanup
gionn 8d7ae33
simplify
gionn 9ea1009
cleanup values
gionn f9b1bb2
fix helm-docs
gionn f86109e
cleanup
gionn 32381d5
invert verifies suites and fail fast
gionn 024301f
really exit when pod not ready
gionn ca801ce
print events of failed pods
gionn 637be07
limit pod logs to 1024
gionn d71ad82
fix deployment secret name
gionn c9ec98c
retry newman run before giving up
gionn aea2309
faster namespace deletion during tests
gionn 07088f5
test persistence failover only for latest suite
gionn 86504b1
make sure newman return error when retries expire
gionn 9dfbe3f
use upstream checkov hook and test elasticsearch connector
gionn ff050ba
fixup graceperiod
gionn 390adf6
reuse content-services helper and avoid globals
gionn 6fcff23
enforce secret for solr by default
gionn 12a2f02
variable for elasticsearch host
gionn b18fcb3
reuse template helper for content and filestore
gionn 4380f23
inject postgresql details from parent
gionn 50b3584
minor move in chart dependencies
gionn aac4a19
refactor in a common library chart
gionn 7639e9d
continue building the matrix to avoid rerun every suite
gionn f3be84f
early exit for persistence failover
gionn cf41b9f
expose support for full postgres url
gionn d7c510a
rename subchart to alfresco-content-common
gionn cd481e2
unbox image values for each service
gionn 312f297
Merge branch 'master' into OPSEXP-1369-elasticsearch-connector
gionn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,8 +14,10 @@ | |
*.swp | ||
*.bak | ||
*.tmp | ||
*.orig | ||
*~ | ||
# Various IDEs | ||
.project | ||
.idea/ | ||
*.tmproj | ||
.vscode/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,6 +14,8 @@ home: https://www.alfresco.com | |
sources: | ||
- https://github.com/Alfresco/acs-deployment | ||
dependencies: | ||
- name: alfresco-content-common | ||
version: 0.1.0 | ||
- name: postgresql | ||
version: 10.16.2 | ||
repository: https://charts.bitnami.com/bitnami | ||
|
@@ -34,6 +36,14 @@ dependencies: | |
- name: alfresco-search | ||
version: 1.0.4 | ||
condition: alfresco-search.enabled | ||
- name: alfresco-elasticsearch-connector | ||
version: 0.1.0 | ||
condition: alfresco-elasticsearch-connector.enabled | ||
- name: alfresco-sync-service | ||
version: 3.0.9 | ||
- name: elasticsearch | ||
alias: elasticsearch | ||
repository: https://helm.elastic.co | ||
version: 7.10.1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jesty check, is there a newer tested version? |
||
condition: elasticsearch.enabled | ||
icon: https://avatars0.githubusercontent.com/u/391127?s=200&v=4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 changes: 5 additions & 0 deletions
5
helm/alfresco-content-services/charts/alfresco-content-common/Chart.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
apiVersion: v2 | ||
name: alfresco-content-common | ||
description: A helper subchart to avoid duplication in alfresco-content-services chart and subcharts | ||
type: library | ||
version: 0.1.0 |
9 changes: 9 additions & 0 deletions
9
helm/alfresco-content-services/charts/alfresco-content-common/templates/_helpers.tpl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{{- define "content-services.shortname" -}} | ||
{{- $name := (.Values.NameOverride | default (printf "%s" "alfresco-cs")) -}} | ||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} | ||
|
||
{{- define "alfresco.shortname" -}} | ||
{{- $name := (.Values.NameOverride | default (printf "%s" "alfresco-")) -}} | ||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} |
10 changes: 10 additions & 0 deletions
10
helm/alfresco-content-services/charts/alfresco-elasticsearch-connector/Chart.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
apiVersion: v2 | ||
name: alfresco-elasticsearch-connector | ||
description: A Helm chart for deploying Alfresco Elasticsearch connector | ||
type: application | ||
version: 0.1.0 | ||
appVersion: "3.0.0" | ||
dependencies: | ||
- name: alfresco-content-common | ||
version: 0.1.0 | ||
repository: "file://../alfresco-content-common" |
62 changes: 62 additions & 0 deletions
62
helm/alfresco-content-services/charts/alfresco-elasticsearch-connector/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
# alfresco-elasticsearch-connector | ||
|
||
![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.0.0](https://img.shields.io/badge/AppVersion-3.0.0-informational?style=flat-square) | ||
|
||
A Helm chart for deploying Alfresco Elasticsearch connector | ||
|
||
## Requirements | ||
|
||
| Repository | Name | Version | | ||
|------------|------|---------| | ||
| file://../alfresco-content-common | alfresco-content-common | 0.1.0 | | ||
|
||
## Values | ||
|
||
| Key | Type | Default | Description | | ||
|-----|------|---------|-------------| | ||
| affinity | object | `{}` | | | ||
| contentMediaTypeCache.enabled | bool | `true` | | | ||
| contentMediaTypeCache.refreshTime | string | `"0 0 * * * *"` | | | ||
| elasticsearch.host | string | `"elasticsearch-master"` | | | ||
| elasticsearch.port | int | `9200` | | | ||
| fullnameOverride | string | `""` | | | ||
| global.alfrescoRegistryPullSecrets | string | `"quay-registry-secret"` | | | ||
| imagePullSecrets | list | `[]` | | | ||
| indexName | string | `"alfresco"` | | | ||
| liveIndexing.content.image.pullPolicy | string | `"IfNotPresent"` | | | ||
| liveIndexing.content.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-live-indexing-content"` | | | ||
| liveIndexing.content.image.tag | string | `"3.0.0"` | | | ||
| liveIndexing.content.replicaCount | int | `1` | | | ||
| liveIndexing.mediation.image.pullPolicy | string | `"IfNotPresent"` | | | ||
| liveIndexing.mediation.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-live-indexing-mediation"` | | | ||
| liveIndexing.mediation.image.tag | string | `"3.0.0"` | | | ||
| liveIndexing.metadata.image.pullPolicy | string | `"IfNotPresent"` | | | ||
| liveIndexing.metadata.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-live-indexing-metadata"` | | | ||
| liveIndexing.metadata.image.tag | string | `"3.0.0"` | | | ||
| liveIndexing.metadata.replicaCount | int | `1` | | | ||
| liveIndexing.path.image.pullPolicy | string | `"IfNotPresent"` | | | ||
| liveIndexing.path.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-live-indexing-path"` | | | ||
| liveIndexing.path.image.tag | string | `"3.0.0"` | | | ||
| liveIndexing.path.replicaCount | int | `1` | | | ||
| nameOverride | string | `""` | | | ||
| nodeSelector | object | `{}` | | | ||
| parentNameOverride | string | `""` | | | ||
| pathIndexingComponent.enabled | bool | `true` | | | ||
| podAnnotations | object | `{}` | | | ||
| podSecurityContext | object | `{}` | | | ||
| reindexing.enabled | bool | `true` | | | ||
| reindexing.image.pullPolicy | string | `"IfNotPresent"` | | | ||
| reindexing.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-reindexing"` | | | ||
| reindexing.image.tag | string | `"3.0.0"` | | | ||
| reindexing.pathIndexingEnabled | bool | `true` | | | ||
| reindexing.postgresql.database | string | `"alfresco"` | | | ||
| reindexing.postgresql.hostname | string | `"postgresql-acs"` | | | ||
| reindexing.postgresql.port | int | `5432` | | | ||
| reindexing.postgresql.url | string | `nil` | | | ||
| resources.limits.memory | string | `"2048Mi"` | | | ||
| resources.requests.memory | string | `"256Mi"` | | | ||
| securityContext | object | `{}` | | | ||
| tolerations | list | `[]` | | | ||
|
||
---------------------------------------------- | ||
Autogenerated from chart metadata using [helm-docs v1.8.1](https://github.com/norwoodj/helm-docs/releases/v1.8.1) |
62 changes: 62 additions & 0 deletions
62
.../alfresco-content-services/charts/alfresco-elasticsearch-connector/templates/_helpers.tpl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
{{/* | ||
Expand the name of the chart. | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.name" -}} | ||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} | ||
{{- end }} | ||
|
||
{{/* | ||
Create a default fully qualified app name. | ||
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). | ||
If release name contains chart name it will be used as a full name. | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.fullName" -}} | ||
{{- if .Values.fullnameOverride }} | ||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} | ||
{{- else }} | ||
{{- $name := default .Chart.Name .Values.nameOverride }} | ||
{{- if contains $name .Release.Name }} | ||
{{- .Release.Name | trunc 63 | trimSuffix "-" }} | ||
{{- else }} | ||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} | ||
{{- end }} | ||
{{- end }} | ||
{{- end }} | ||
|
||
{{/* | ||
Create chart name and version as used by the chart label. | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.chart" -}} | ||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} | ||
{{- end }} | ||
|
||
{{/* | ||
Common labels | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.labels" -}} | ||
helm.sh/chart: {{ include "alfresco-elasticsearch-connector.chart" . }} | ||
{{ include "alfresco-elasticsearch-connector.selectorLabels" . }} | ||
{{- if .Chart.AppVersion }} | ||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | ||
{{- end }} | ||
app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
{{- end }} | ||
|
||
{{/* | ||
Selector labels | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.selectorLabels" -}} | ||
app.kubernetes.io/name: {{ include "alfresco-elasticsearch-connector.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
{{- end }} | ||
|
||
{{/* | ||
Create the name of the service account to use | ||
*/}} | ||
{{- define "alfresco-elasticsearch-connector.serviceAccountName" -}} | ||
{{- if .Values.serviceAccount.create }} | ||
{{- default (include "alfresco-elasticsearch-connector.fullname" .) .Values.serviceAccount.name }} | ||
{{- else }} | ||
{{- default "default" .Values.serviceAccount.name }} | ||
{{- end }} | ||
{{- end }} |
23 changes: 23 additions & 0 deletions
23
...ntent-services/charts/alfresco-elasticsearch-connector/templates/liveindexing-config.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: {{ template "alfresco-elasticsearch-connector.fullName" . }}-configmap | ||
labels: | ||
heritage: {{ .Release.Service }} | ||
release: {{ .Release.Name }} | ||
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} | ||
component: search | ||
data: | ||
ELASTICSEARCH_INDEXNAME: "{{ .Values.indexName }}" | ||
SPRING_ELASTICSEARCH_REST_URIS: http://{{ .Values.elasticsearch.host }}:{{ .Values.elasticsearch.port }} | ||
SPRING_ACTIVEMQ_BROKERURL: nio://{{ .Release.Name }}-activemq-broker:61616 | ||
ALFRESCO_SHAREDFILESTORE_BASEURL: http://{{ template "alfresco.shortname" . }}-filestore:80/alfresco/api/-default-/private/sfs/versions/1/file/ | ||
ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_BASEURL: http://{{ template "content-services.shortname" . }}-transform-misc/transform/config | ||
ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_REFRESHTIME: "{{ .Values.contentMediaTypeCache.refreshTime }}" | ||
ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_ENABLED: "{{ .Values.contentMediaTypeCache.enabled }}" | ||
ALFRESCO_PATHINDEXINGCOMPONENT.ENABLED: "{{ .Values.pathIndexingComponent.enabled }}" | ||
{{- if .Values.liveIndexing.environment }} | ||
{{- range $key, $val := .Values.liveIndexing.environment }} | ||
{{ $key }}: {{ $val | quote }} | ||
{{- end }} | ||
{{- end }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the reason for changing the order here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since they are executed sequentially and now
latest
is the only one executing the persistence failover test (that I've disabled on the remaining jobs since it's slowing all the builds without really any added value), it's better to build latest first that is more comprehensive