-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Targeted refresh #153
Targeted refresh #153
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
class ManageIQ::Providers::AnsibleTower::AutomationManager::Refresher < ManageIQ::Providers::BaseManager::Refresher | ||
class ManageIQ::Providers::AnsibleTower::AutomationManager::Refresher < ManageIQ::Providers::BaseManager::ManagerRefresher | ||
include ManageIQ::Providers::AnsibleTower::Shared::AutomationManager::Refresher | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Collector < ManageIQ::Providers::Inventory::Collector | ||
require_nested :AutomationManager | ||
require_nested :ConfigurationScriptSource | ||
require_nested :TargetCollection | ||
end |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Collector::TargetCollection < ManageIQ::Providers::Inventory::Collector | ||
include ManageIQ::Providers::AnsibleTower::Shared::Inventory::Collector::TargetCollection | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Parser < ManageIQ::Providers::Inventory::Parser | ||
require_nested :AutomationManager | ||
require_nested :ConfigurationScriptSource | ||
end |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Persister < ManageIQ::Providers::Inventory::Persister | ||
require_nested :AutomationManager | ||
require_nested :ConfigurationScriptSource | ||
require_nested :TargetCollection | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,7 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Persister::AutomationManager < ManageIQ::Providers::AnsibleTower::Inventory::Persister | ||
include ManageIQ::Providers::AnsibleTower::Inventory::Persister::Definitions::Collections | ||
include ManageIQ::Providers::AnsibleTower::Inventory::Persister::Definitions::AutomationCollections | ||
|
||
def initialize_inventory_collections | ||
%i(credentials | ||
configuration_script_sources | ||
configured_systems).each do |name| | ||
|
||
add_collection(automation, name) | ||
end | ||
|
||
add_collection(automation, :configuration_scripts) do |builder| | ||
builder.add_properties(:model_class => ManageIQ::Providers::AutomationManager::ConfigurationScript) | ||
end | ||
|
||
add_configuration_script_payloads | ||
|
||
add_inventory_root_groups | ||
|
||
add_vms | ||
initialize_automation_inventory_collections | ||
end | ||
end |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
class ManageIQ::Providers::AnsibleTower::Inventory::Persister::TargetCollection < ManageIQ::Providers::AnsibleTower::Inventory::Persister | ||
include ManageIQ::Providers::AnsibleTower::Inventory::Persister::Definitions::AutomationCollections | ||
|
||
def targeted? | ||
true | ||
end | ||
|
||
def strategy | ||
:local_db_find_missing_references | ||
end | ||
|
||
def initialize_inventory_collections | ||
initialize_automation_inventory_collections | ||
end | ||
end |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
module ManageIQ::Providers::AnsibleTower::Shared::Inventory::Collector::TargetCollection | ||
MAX_FILTER_SIZE = 200 | ||
|
||
def connection | ||
@connection ||= manager.connect | ||
end | ||
|
||
def initialize(_manager, _target) | ||
super | ||
|
||
infer_related_manager_refs! | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this needed if all it does is There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it's not, I'll remove it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it's needed now :) it adds dependent targets |
||
|
||
def inventories | ||
find_records(:inventory_root_groups, connection.api.inventories) | ||
end | ||
|
||
def hosts | ||
return @hosts if @hosts.present? | ||
@hosts = find_records(:configured_systems, connection.api.hosts) | ||
end | ||
|
||
def job_templates | ||
return @job_templates if @job_templates.present? | ||
@job_templates = find_records(:configuration_scripts, connection.api.job_templates) | ||
end | ||
|
||
def configuration_workflows | ||
find_records(:configuration_scripts, connection.api.workflow_job_templates) | ||
end | ||
|
||
def projects | ||
return @projects if @projects.present? | ||
@projects = find_records(:configuration_script_sources, connection.api.projects) | ||
end | ||
|
||
def credentials | ||
find_records(:credentials, connection.api.credentials) | ||
end | ||
|
||
protected | ||
|
||
def infer_related_manager_refs! | ||
if references(:configured_systems).present? | ||
hosts.each do |host| | ||
add_simple_target!(:inventory_root_groups, host.inventory_id.to_s) | ||
end | ||
end | ||
|
||
if references(:configuration_scripts).present? | ||
job_templates.each do |job_template| | ||
add_simple_target!(:inventory_root_groups, job_template.inventory_id.to_s) | ||
add_simple_target!(:configuration_script_sources, job_template.try(:project_id)) if job_template.try(:project_id).present? | ||
%w(credential_id cloud_credential_id network_credential_id).each do |credential_attr| | ||
next unless job_template.respond_to?(credential_attr) | ||
credential_id = job_template.public_send(credential_attr).to_s | ||
next if credential_id.blank? | ||
add_simple_target!(:credentials, credential_id) | ||
end | ||
end | ||
end | ||
|
||
# target was added in previous block, compute references again | ||
target.manager_refs_by_association_reset | ||
if references(:configuration_script_sources) | ||
projects.each do |project| | ||
add_simple_target!(:credentials, project.credential_id.to_s) | ||
end | ||
end | ||
|
||
target.manager_refs_by_association_reset | ||
end | ||
|
||
# Calls API and finds existing records. | ||
# @param inventory_collection_name [Symbol] IC name (as identified in persister's definitions) | ||
# @param endpoint - endpoint for AnsibleTowerClient api call | ||
def find_records(inventory_collection_name, endpoint) | ||
refs = references(inventory_collection_name) | ||
return [] if refs.blank? | ||
|
||
if refs.size == 1 | ||
begin | ||
[endpoint.find(refs[0])] | ||
rescue AnsibleTowerClient::ResourceNotFoundError | ||
[] | ||
end | ||
else | ||
multi_query(refs) do |refs_batch| | ||
# returns Enumeration | ||
endpoint.all(:id__in => refs_batch.join(',')) | ||
end | ||
end | ||
end | ||
|
||
def multi_query(refs) | ||
refs.each_slice(MAX_FILTER_SIZE).map { |refs_batch| yield(refs_batch) }.lazy.flat_map(&:lazy) | ||
end | ||
|
||
# @param collection [Symbol] inventory collection name (as identified in persister's definitions) | ||
def references(collection) | ||
manager_ref = manager_ref_by_collection(collection) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hm I would probably do this explicitly, rather than than building tmp persister? Most of the manage_ref are the same, aren't they? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I want to have definition of manager_ref on 1 place rather than repeating the same on two places and ansible tower inventories doesn't have |
||
target.manager_refs_by_association.try(:[], collection).try(:[], manager_ref.to_sym).try(:to_a).try(:compact) || [] | ||
end | ||
|
||
# Using first manager_ref from persister's IC definitions | ||
# All calls are identified by one unique ID | ||
def manager_ref_by_collection(inventory_collection_name) | ||
@tmp_persister ||= self.class.to_s.gsub('::Collector', '::Persister').constantize.new(@manager, @target) | ||
@tmp_persister.collections[inventory_collection_name].manager_ref[0] | ||
end | ||
|
||
def add_simple_target!(association, ems_ref) | ||
return if ems_ref.blank? | ||
|
||
target.add_target(:association => association, :manager_ref => { manager_ref_by_collection(association) => ems_ref }) | ||
end | ||
end |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
describe ManageIQ::Providers::AnsibleTower::AutomationManager::Refresher do | ||
it_behaves_like 'refresh targeted', | ||
:provider_ansible_tower, | ||
described_class.parent, | ||
:ansible, | ||
described_class.name.underscore + '_targeted' | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@slemrmartin is this required? I'm trying to remove ManagerRefresher in another PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agrare it raises error when I'm refreshing
ConfigurationScriptSource
with Refresher.I think there is error in
collector_inventory_for_targets()
. InManagerRefresher
, it derives Collector class inInventory
, there it tries to do by itself (https://github.com/ManageIQ/manageiq/blob/master/app/models/manageiq/providers/base_manager/refresher.rb#L141). This construction doesn't work for TargetCollection.Then it calls
parse_targeted_inventory
withcollector == nil
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, for
InventoryRefresh::Target
it doesn't work...it contantizesManageIQ::Providers::AnsibleTower::Inventory::Collector::Target
instead ofManageIQ::Providers::AnsibleTower::Inventory::Collector::TargetCollection