Adds file splitting support to dump/backup commands #50
Conversation
NickLaMuro
force-pushed
the
file_split_for_db_backup_and_db_dump
branch
4 times, most recently
from
afdeb8c
to
284fef1
Compare
NickLaMuro
force-pushed
the
file_split_for_db_backup_and_db_dump
branch
4 times, most recently
from
a04492a
to
1d51ff0
Compare
NickLaMuro
changed the title
|
Might change how we are handling file splitting, so setting this as See ManageIQ/manageiq#17798 (comment) for some further details. |
NickLaMuro
force-pushed
the
file_split_for_db_backup_and_db_dump
branch
from
1d51ff0
to
68db4a9
Compare
| @@ -147,6 +148,12 @@ def ask_for_tables_to_exclude_in_dump | |||
| end || true | |||
| end | |||
|
|
|||
| def ask_to_split_up_output | |||
| if action != :restore && should_split_output? | |||
There was a problem hiding this comment.
I think we should only support this for the :dump action on the first pass since restore from split backups won't work and it wasn't really part of the original RFE.
There was a problem hiding this comment.
since restore from split backups won't work
Neither restoring from :dump or :backup is supported with splits currently, since there is no way to fetch the splits (via a glob) and combine them into a single stream.
Adding support for restoring from a split would be a separate RFE, but it would effectively be the same work to do it for both as it would be for one. They basically would need some form of a pipeline to first take the files in order and read their data in order as if they were a single stream.
There was a problem hiding this comment.
I guess what I'm saying is that we shouldn't give the user the option to backup using splits if we can't restore with splits. I'm fine with the option to dump using splits because the intention was never to restore it.
There was a problem hiding this comment.
I guess what I'm saying is that we shouldn't give the user the option to backup using splits if we can't restore with splits.
I guess that is fair.
I figured that someone would eventually request the splitting functionality for backups as well anyway, specifically for s3, since there limits to file size.
Please note that I am also partially arguing this point simply because I don't want to delete the appliance console integration tests that I have written that already support running against and validating split based backups...
(deep down... I am really a simple human being really...)
There was a problem hiding this comment.
Definitely not arguing against creating split backups at some point, but I think we need split restore at that point as well. If we leave this as is, some user will be rather unhappy when we have to tell them that they can't restore a backup that we let them create.
There was a problem hiding this comment.
Welp, I can do that. Despite not wanting to comment out/remove some of my personal tests, I think I agree with that line of thinking.
Will try and take care of that code change shortly.
NickLaMuro
changed the title
Adds a set of questions to be queried for `:dump` and `:restore` actions in DatabaseAdmin. It asks: - If the wishes to split up the output of the dump/restore - If yes, how big should the split files be (default 500M)
NickLaMuro
force-pushed
the
file_split_for_db_backup_and_db_dump
branch
from
68db4a9
to
b7ab9cb
Compare
|
Checked commits NickLaMuro/manageiq-appliance_console@2f084da~...b7ab9cb with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0 spec/database_admin_spec.rb
|
NOTE: Currently built off of #51Update: This has been rebased and this branch updated accordingly.This exposes the
:byte_countflag that is added to theevm:db:backupandevm:db:dumptasks here:The user experience will look something like the following:
Links