Skip to content
This repository has been archived by the owner on Sep 12, 2022. It is now read-only.

Make another attempt to fix adding fixed ips #604

Merged
merged 4 commits into from
Apr 17, 2018

Conversation

cdosborn
Copy link
Contributor

@cdosborn cdosborn commented Apr 13, 2018

Description

Try different approach to working around openstack port availability zone issue

Problem

After upgrading Marana to Pike we found that ports which were attached post boot, would never successfully attach. This meant that every time an instance would boot (start/unshelve) minus initial launch, it would not be reachable. A workaround was discovered to enable allow-hotplug for the given interface in cloud-init, but it was deemed more complicated than simply preserving the ports.

We were deleting/recreating the ports in the first place, because of a bug which affected j7m https://bugs.launchpad.net/nova/+bug/1759924.

Solution

Rather than add/delete ports, add/remove fixed ips, but explicitly check if we're dealing with the availability zone issue and then try deleting/recreating the port. It is unknown why j7m is able to attach ports post boot, and we cannot.

I tested the following scenarios. Start/stop, shelve/unshelve on marana and indiana. (and in particular the shelve scenario where the AZ was incorrect)

Checklist before merging Pull Requests

  • Add an entry in the changelog
  • Reviewed and approved by at least one other contributor.

We are no longer removing ports. After upgrading Marana to Pike we found that
ports which were attached post boot, would never successfully attach. A
workaround was discovered to enable allow-hotplug for the given interface
 in cloud-init, but it was deemed more complicated than simply preserving the
 ports.

We were deleting/recreating the ports in the first place, because of a bug
which affected j7m https://bugs.launchpad.net/nova/+bug/1759924.

This reverts commit af002eb.
@cdosborn cdosborn self-assigned this Apr 13, 2018
@coveralls
Copy link

coveralls commented Apr 13, 2018

Coverage Status

Coverage decreased (-0.02%) to 37.322% when pulling 1277702 on cdosborn:fix-broken-fixed-ip-add into b06865b on cyverse:v32.

…zone issue

Problem:

    After upgrading Marana to Pike we found that ports which were attached
    post boot, would never successfully attach. A workaround was discovered to
    enable allow-hotplug for the given interface in cloud-init, but it was
    deemed more complicated than simply preserving the ports.

    We were deleting/recreating the ports in the first place, because of a bug
    which affected j7m https://bugs.launchpad.net/nova/+bug/1759924.

Solution

    Rather than add/delete ports, add/remove fixed ips, but explicitly check
    if we're dealing with the availability zone issue and then try
    deleting/recreating the port. For some reason j7m is able to attach ports
    post boot. If its discovered that this is because of some race condition,
    we will need another workaround.
Copy link
Contributor

@c-mart c-mart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this lets us recover currently stopped/shelved instances with no fixed IP without having to reboot them again, +1.

Going forward with modern, load-balanced control plane, I suspect that we're causing unnecessary trouble for ourselves by deleting fixed IPs, ports, and private networks to begin with. I suspect that if we stopped doing that then there would be no need for these sorts of hackarounds.

instance.id, network_id=network_id)

elif not instance._node.private_ips:
# Only add fixed ip, if the instance doesn't already have on
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo, "doesn't already have one"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants