Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README.md for kops/weave/flannel/cnigenie #323

Merged
merged 2 commits into from
Jan 14, 2020
Merged

README.md for kops/weave/flannel/cnigenie #323

merged 2 commits into from
Jan 14, 2020

Conversation

nonsense
Copy link
Member

@nonsense nonsense commented Jan 7, 2020

Here is a README which goes through a setup of Kubernetes cluster with kops and Flannel and Weave for networks.

It takes about 10-15min to go over it and have a working cluster.

There are a few known issues that would be good to have a second pair of eyes on, in case I am doing something stupid. I have tried to dynamically attach/detach network interfaces with weave to pods, but they are not accessible. I believe this must be possible because Kubernetes / CNI-Genie / Weave CNI must be setting up those interfaces somehow correctly when the annotations are present.


Here is a demo of this setup in case you find this useful:

  1. Download https://drive.google.com/open?id=1p8RqDv7tyqJH0V6uOpLEtBvaa_fU2zjg
  2. asciinema play -s 3 ./demo2.cast

@nonsense
Copy link
Member Author

nonsense commented Jan 7, 2020

@lanzafame adding you as well in case you have some insight into the known issues part.

@nonsense
Copy link
Member Author

nonsense commented Jan 9, 2020

Known issues have been addressed by disabling the weave-npc as the weave network policy controller is picking up the PodIP when adding rules to ipset (used by iptables), and this is not OK when the PodIP is set to an IP within the flannel network.

@nonsense
Copy link
Member Author

Merging this as this is pretty much the setup we used with @Stebalien to try sidecar with k8s.

The current known issues are:

  1. When using the host network, for some reason there is a missing ip route to the Kubernetes Services CIDR. I opened an issue/question for this one at ClusterIP services not accessible when using flannel CNI from host machines in Kubernetes flannel-io/flannel#1243 - not sure if this is a bug within kops or flannel, or we are misconfiguring something on our side. For now we add these routes manually (or via an init container after Flannel is installed).

  2. We can't allocate IPs because the Weave CNI doesn't support that. We have to submit a PR or fork it, or use the weave cmdline utility. Technically it is possible to allocate a specific IP with Weave, but not through the current version of the Weave CNI.


Will follow up with additional PR, where we improve docs, and add code to support sidecar k8s runner.

@nonsense nonsense merged commit cf7de96 into master Jan 14, 2020
@nonsense nonsense deleted the kops-weave branch January 14, 2020 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant