-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: in-line versioning + deprecation for multi-semaphore #13645
base: main
Are you sure you want to change the base?
Changes from all commits
4177329
9bb34ac
67f1607
f995950
aa2183d
e1ce321
1619b36
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -24,10 +24,6 @@ data: | |
template: "2" # Two instances of template can run at a given time in particular namespace | ||
``` | ||
|
||
!!! Warning | ||
Each synchronization block may only refer to either a semaphore or a mutex. | ||
If you specify both only the semaphore will be locked. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No longer relevant in 3.6 |
||
|
||
## Workflow-level Synchronization | ||
|
||
You can limit parallel execution of workflows by using the same synchronization reference. | ||
|
@@ -36,44 +32,14 @@ In this example the synchronization key `workflow` is configured as limit `"1"`, | |
|
||
Using a semaphore configured by a `ConfigMap`: | ||
|
||
```yaml | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: synchronization-wf-level- | ||
spec: | ||
entrypoint: hello-world | ||
synchronization: | ||
semaphores: | ||
- configMapKeyRef: | ||
name: my-config | ||
key: workflow | ||
templates: | ||
- name: hello-world | ||
container: | ||
image: busybox | ||
command: [echo] | ||
args: ["hello world"] | ||
```yaml title="examples/synchronization-wf-level.yaml" | ||
--8<-- "examples/synchronization-wf-level.yaml:12" | ||
``` | ||
|
||
Using a mutex is equivalent to a limit `"1"` semaphore: | ||
|
||
```yaml | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: synchronization-wf-level- | ||
spec: | ||
entrypoint: hello-world | ||
synchronization: | ||
mutexes: | ||
- name: workflow | ||
templates: | ||
- name: hello-world | ||
container: | ||
image: busybox | ||
command: [echo] | ||
args: ["hello world"] | ||
```yaml title="examples/synchronization-mutex-wf-level.yaml" | ||
--8<-- "examples/synchronization-mutex-wf-level.yaml:3" | ||
``` | ||
|
||
## Template-level Synchronization | ||
|
@@ -85,77 +51,16 @@ This applies even when multiple steps or tasks within a workflow or different wo | |
|
||
Using a semaphore configured by a `ConfigMap`: | ||
|
||
```yaml | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: synchronization-tmpl-level- | ||
spec: | ||
entrypoint: synchronization-tmpl-level-example | ||
templates: | ||
- name: synchronization-tmpl-level-example | ||
steps: | ||
- - name: synchronization-acquire-lock | ||
template: acquire-lock | ||
arguments: | ||
parameters: | ||
- name: seconds | ||
value: "{{item}}" | ||
withParam: '["1","2","3","4","5"]' | ||
|
||
- name: acquire-lock | ||
synchronization: | ||
semaphores: | ||
- configMapKeyRef: | ||
name: my-config | ||
key: template | ||
container: | ||
image: alpine:latest | ||
command: [sh, -c] | ||
args: ["sleep 10; echo acquired lock"] | ||
```yaml title="examples/synchronization-tmpl-level.yaml" | ||
--8<-- "examples/synchronization-tmpl-level.yaml:11" | ||
``` | ||
|
||
Using a mutex will limit to a single concurrent execution of the template: | ||
|
||
```yaml | ||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
generateName: synchronization-tmpl-level- | ||
spec: | ||
entrypoint: synchronization-tmpl-level-example | ||
templates: | ||
- name: synchronization-tmpl-level-example | ||
steps: | ||
- - name: synchronization-acquire-lock | ||
template: acquire-lock | ||
arguments: | ||
parameters: | ||
- name: seconds | ||
value: "{{item}}" | ||
withParam: '["1","2","3","4","5"]' | ||
|
||
- name: acquire-lock | ||
synchronization: | ||
mutexes: | ||
- name: template | ||
container: | ||
image: alpine:latest | ||
command: [sh, -c] | ||
args: ["sleep 10; echo acquired lock"] | ||
```yaml title="examples/synchronization-mutex-tmpl-level.yaml" | ||
--8<-- "examples/synchronization-mutex-tmpl-level.yaml:3" | ||
``` | ||
|
||
Examples: | ||
|
||
1. [Workflow level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-wf-level.yaml) | ||
1. [Workflow level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level.yaml) | ||
1. [Step level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-tmpl-level.yaml) | ||
1. [Step level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level.yaml) | ||
1. [Legacy workflow level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-wf-level-legacy.yaml) | ||
1. [Legacy workflow level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-wf-level-legacy.yaml) | ||
1. [Legacy step level semaphore](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-tmpl-level-legacy.yaml) | ||
1. [Legacy step level mutex](https://github.com/argoproj/argo-workflows/blob/main/examples/synchronization-mutex-tmpl-level-legacy.yaml) | ||
|
||
## Queuing | ||
|
||
When a workflow cannot acquire a lock it will be placed into a ordered queue. | ||
|
@@ -204,30 +109,6 @@ Examples: | |
!!! Warning | ||
If a Workflow is at the front of the queue and it needs to acquire multiple locks, all other Workflows that also need those same locks will wait. This applies even if the other Workflows only wish to acquire a subset of those locks. | ||
|
||
## Legacy | ||
|
||
In workflows prior to 3.6 you can only specify one lock in any one workflow or template using either a mutex: | ||
|
||
```yaml | ||
synchronization: | ||
mutex: | ||
... | ||
``` | ||
|
||
or a semaphore: | ||
|
||
```yaml | ||
synchronizaion: | ||
semamphore: | ||
... | ||
``` | ||
|
||
Specifying both would not work in <3.6, only the semaphore would be used. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We just need to fix this bug per #13358 (comment) and #11859 -- i.e. give a validation error instead of silently ignoring it in 3.5 and prior versions There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed with a validation error in #13669 |
||
|
||
The single `mutex` and `semaphore` syntax still works in version 3.6 but is considered deprecated. | ||
Both the `mutex` and the `semaphore` will be taken in version 3.6 with this syntax. | ||
This syntax can be mixed with `mutexes` and `semaphores`, all locks will be required. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if we want to describe this, this would be better described with another example |
||
|
||
## Other Parallelism support | ||
|
||
You can also [restrict parallelism at the Controller-level](parallelism.md). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,5 @@ | ||
# This example demonstrates the use of a Synchronization Mutex lock on template execution. Mutex lock limits | ||
# only one of the template execution across the workflows in the namespace which has same Mutex lock. | ||
|
||
apiVersion: argoproj.io/v1alpha1 | ||
kind: Workflow | ||
metadata: | ||
|
@@ -28,18 +27,22 @@ spec: | |
|
||
- name: acquire-lock | ||
synchronization: | ||
mutex: | ||
mutex: # deprecated: v3.5 and before | ||
name: welcome | ||
# mutexes: # v3.6 and after | ||
# - name: welcome | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm inclined to just move these to internal tests and otherwise remove the legacy examples entirely from the user-facing and flipped syntax + in-line comments here is a stopgap, especially as the in-line deprecation is hard to understand without the future syntax right next to it (and users are likely to miss |
||
container: | ||
image: alpine:latest | ||
command: [sh, -c] | ||
args: ["sleep 20; echo acquired lock"] | ||
|
||
- name: acquire-lock-1 | ||
synchronization: | ||
mutex: | ||
mutex: # deprecated: v3.5 and before | ||
name: test | ||
# mutexes: # v3.6 and after | ||
# - name: test | ||
container: | ||
image: alpine:latest | ||
command: [sh, -c] | ||
args: ["sleep 50; echo acquired lock"] | ||
args: ["sleep 50; echo acquired lock"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is suboptimal as the field checker automation is plain text search and so does not recognize that it's in a comment, but nbd in context as it does show it and say it's deprecated