map create: operation not permitted

[MarahAbu]

Describe the bug
Couldn't load probes - error loading BPF objects from go-side. field HandleSockSetState: program handle_sock_set_state: map sock_infos: map create: operation not permitted (MEMLOCK may be too low, consider rlimit.RemoveMemlock)

To Reproduce
Steps to reproduce the behavior:

1. Go to one of carettas failing pods
2. Click on logs
3. Scroll down to the last line
4. See error

Expected behavior
healthy pods but instead pods are restarting over and over again and failing
Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: linux
• Kubernetes cluster information - k8s v1.23

Additional context
same configuration works on aws clusters only failing in azure and gcp

[udiz]

Hey @MarahAbu, are you using the default helm chart to deploy caretta?

Can you share the Linux kernel version of the nodes?

[MarahAbu]

Hi, yes using the default chart, I only disabled victoria metrics and grafana and added a servicemonitor
Azure kernel version: 5.4.0
GCP kernel version: 5.10.162+
and sorry the k8s version is v1.24 not v.123

[MarahAbu]

Hi @udiz, any update?

[udiz]

Do you encounter this log on both the Azure instances and the GCP ones?
i'm guessing it's either a security issue or a memory availability issue