Add tolerations to the helm chart

[manuelbuil]

Description

Flannel might deploy in clusters whose nodes contain taints and thus we need the tolerations field to be configurable

Todos

• Tests
• Documentation
• Release note

Release Note

None required

[fungaren]

This commit introduced a bug that not all taints are tolerated, at least the default taint for master node node-role.kubernetes.io/master:NoSchedule is not tolerated.

Please be careful if you are changing the behavior.

[brandond]

@fungaren the node-role.kubernetes.io/master label has been deprecated for quite some time, in favor of control-plane. In what environment do you still see this taint being set?

[fungaren]

@brandond I have an old Kubernetes cluster which is using 1.24. While upgrading flannel version the problem appeared. Now I have decide to add --set flannel.tolerations[0].effect=NoSchedule to address it.

Maybe it should not be considered as a bug, but it is indeed a breaking change.