[Podification] Make it work with Zones

[Fryguy]

Requires reworking of the podified orchestrator

[carbonin]

ManageIQ/manageiq#19648 further decouples the server from workers by reducing our use of Drb and simplifies the server.

[carbonin]

The problem here is that, currently, the pods architecture uses only a single server and the "orchestrator" is just the server process (worker manager) for that single server. This is a problem because if we want multiple zones (and specific roles within those zones) we need multiple servers to put in zones.

We don't have a way to create a new "server" in pods so the plan here is to create a stub server model for each zone to serve as a link between zones, roles, and workers that will allow an external worker monitor to dictate worker assignments for the whole region.

The path forward on this is to simplify the server logic enough so it's possible to move it out of the actual MiqServer model. At that point we can generalize the logic so it can work against multiple servers, effectively de-coupling the worker monitor logic from a specific runtime. Then, in pods, we would set up the "orchestrator" to monitor workers for every server in the region.

The biggest challenge here is to not break the existing appliance-based server architecture during this process. This means that we want to reduce the differences between the pods and appliance cases as much as possible. Some places I've picked out as candidates for this kind of refactoring are:

• Simplifying the responsibilities of the worker monitor
  • Remove sync_blacklisted_event_names server queue message
  • Simplify MiqServer#monitor_workers
  • Schedule the check for timed out queue messages once for the region
  • Remove waiting_for_stop_before_restart monitor status
  • Move the important parts of monitoring workers into #monitor_workers
• Removing the server-to-worker communication over Drb
  • Use TERM signals instead of an exit message over drb to stop workers
  • Use file based heartbeating for all worker deployment cases
  • Requires the removal of the VimBrokerWorker as mentioned in [Podification] Removal of the MiqVimBrokerWorker manageiq-providers-vmware#484 (comment) (the broker is the last user of the worker to server drb message passing)