Card Config: Migrations

Overview

The Aperta project consists of many custom Task models that are responsible for managing the collection of specific subsets of information from the user. There is an ongoing effort to convert these legacy Tasks into CustomCardTasks which can manage the generic collection of data for any subset of questions within the system. Having a data-driven CustomCardTask allows Aperta administrators to easily create their own set of questions and modify them on the fly.

This document concerns itself with migrating a specific custom Task model into a generic CustomCardTask. For matter of efficiency, it is written in a sequential manner, expecting the developer to complete each section before moving to the next.

Components of a Migration

The following items will be part of every legacy task migration:

1. Generation of XML to represent a new CustomCardTask
2. Migration and re-parenting existing answer data to the Answer table
3. Converting hardcoded handlebars template questions to CardContent
4. Retirement of existing legacy task code

Analysis

The first and most important step in creating a migration is to conduct proper analysis of the existing legacy task. Each task migration is a unique snowflake and needs proper analysis to determine the level of complexity.

The legacy tasks in Aperta have been developed over a multi-year history with varying degrees of business rule complexity which can result in wide inconsistencies in how each legacy task works and where it stores answer data.

During the analysis phase, the following questions should be answered to determine the complexity of the existing Task and how much prior work can be leveraged to ease the burden of conversion:

1. Where are answers currently stored?
   • on the legacy Task model?
   • in the Answer table?
   • as attributes of a related model?
2. What types of questions are being asked?
   • do questions show / hide based on other answers?
   • do blocks of questions "repeat" (like they do for Funders or Authors)?
   • are any questions not currently supported by the existing types of card-content?
3. Does the card interact with models other than Answer?
4. Are there any highly custom, non-standard display elements on the card?
   • Examples: dynamic footer that changes based on current answers, select boxes that populate with external data such as liquid templates
   • Does the card show different states of the same data (example: when a block of author questions are marked complete, it shows a summary of this content instead of disabled form fields)?
5. Are the question answers being exported to external systems? (billing log / salesforce / APEX / etc.)
   • If so, be sure to look closely to see if there is custom logic around how that data is being used.
6. Are there unique permissions being applied to the questions being asked?
   • Examples: UploadManuscriptTask and PaperReviewerTask
   • Typically, you will find this in the corresponding controller
7. Does the card require custom version snapshot views?

The more times that yes is answered to the above questions, the higher the complexity. If you encounter a high level of complexity, it is important to raise this as a concern to a product owner. This might be an opportunity to simplify how the card works or streamline the business process.

Custom Card Content Creation

Questions 2 and 4 in the analysis phase will determine if a new custom card component will need to be created. Creation of a new component is out of scope for this document, but an example can be found in the commit to add financial-disclosure-summary card content.

Future steps in this document assume that all custom card content types already exist to build the new card.

XML Creation

Once the analysis phase is complete and the developer has a solid understanding as to how the legacy card works, the new XML can be created using the administrative interface.

• The preview can be manually compared to the legacy task by using the admin card preview tab.
• Existing question text and idents can be obtained from the legacy task's handlebar templates.
• The existing data migrator (covered later in this document) expects that the new XML idents match the existing ones. In other words, the developer should not add or subtract idents as part of the initial migration. This should be a feature to feature migration. Any changes are expected to be introduced as a new CardVersion.

Once the XML has been created, it should be saved in this specific folder for loading by the card migrator: lib/custom_card/configurations/xml_content

Card Configuration

A CustomCard::Configurations class is responsible for determining the name of the custom card being created, whether it should be given a state of published (immediate available for use) or whether it should be loaded in just a development environment. It also determines the role name permissions that should be granted to the card.

The configuration class looks for an associated XML file in lib/custom_card/configurations/xml_content.
An example configuration file looks like this: lib/custom_card/configurations/early_version.rb

A service class exists to generate the permission role names. To use it, load a recent production data export into your development environment and then from the Rails console, Listing permissions shows the commands to use.

> CustomCard::PermissionInquiry.new(legacy_class_name: "MyLegacyTask").legacy_permissions
> { view: ["Academic Editor", "Billing Staff", "Collaborator", "Cover Editor"], edit: ["Collaborator", "Cover Editor", "Creator"]}

The output can be copied directly into the configuration class.

Reusing the Legacy Task Subclass via CardTaskType

There may be times where it's impractical to convert the legacy Task subclass into a simple CustomCardTask. The Task may be tied to specific code elsewhere in the system, or the existing Roles and Permissions scheme may depend on that Task in a way that makes removing it prohibitive. A few extra steps are necessary to integrate the existing Task subclass.

1. The task_class method on a CustomCard::Configurations class must be defined as something other than "CustomCardTask" as it is in CustomCard::Configurations::Base. This example from the PR for migrating the Upload Manuscript Task shows the full changes.

1. A corresponding CardTaskType record will need to be added to the database if you're not going to use the CustomCard::Migrator service (see Section *Simple Data Migration for details). There's an example of this for TahiStandardTasks::UploadManuscriptTask in the production database.

1. The existing handlebars template for the task subclass will have to be changed to match the contents of the custom-card-task.hbs file. For example, the upload-manuscript-task.hbs file has the same contents of the custom-card-task.hbs file. This is necessary based on how the system renders templates for a given Task type.

Data Migration

The data migration is the process of creating the correct tree of card content and moving existing answers to appropriate Answer records. It executes within the context of an ActiveRecord::Migration so that it is wrapped within a transaction and can be rolled back if something fails.
There are three steps to the data migration:

1. Loading the newly created Card using the CardConfiguration class created earlier.
2. Calling the migrator to migrate existing data to the new Card.
3. Updating the existing data.yml for seed data.

Based on the analysis conducted earlier, the work to complete the migration will be either simple or custom.

Simple Data Migration

A simple data migration can be used if all current answers are already stored in the Answer table. In this case, re-parent the existing Answers to the newly created CardContent.

A CustomCard::Migrator service class already exists for this purpose, so the full data migration would look like Listing migrator .

class MigratePublishingRelatedQuestionsTaskToCustomCard < ActiveRecord::Migration
  def up
    # load custom card into the system
    CustomCard::Loader.all

    # migrate legacy task to custom card
    migrator = CustomCard::Migrator
                 .new(legacy_task_klass_name: "TahiStandardTasks::PublishingRelatedQuestionsTask",
                      configuration_class: CustomCard::Configurations::PublishingRelatedQuestions)
    migrator.migrate
  end
end

Custom Data Migration

A custom migration is necessary if there are repeated blocks of content (Funders, Authors, etc.) or if answers are stored in multiple places within the application.

In this case, a custom migrator will need to be created.

The following tasks must be performed by the custom migrator:

• Convert Legacy Task to a CustomCardTask
• Re-parent all answers to the appropriate newly created CardContent
• Update the existing TaskTemplate to point to the new Card
• Destroy the legacy JournalTaskType record
• Create=Repetitions= for any repeated content blocks

An example of a custom migrator meeting all these requirements can be found with the FinancialDisclosureMigrator.

Models Involved

Figure cardmigrationerd shows the models involved in custom cards. Here is a brief overview of these models:

• CardContent - each record represents a distinct piece of content that is displayed on a Card. Often, this manifests itself as a question, but there are other types of CardContent that are not answerable. CardContent is created by converting XML nodes found on the card admin screen.
• CardVersion - when any changes are made to a card, a new CardVersion will be created. This allows versioning of a tree of CardContent.
• Card - manages a tree of CardContent.
• Answer - holds the answer to a specific piece of CardContent. In the case of a repeated block of questions (such as Funder), the Answer record will also be tied to a Repetition.
• Repetition - represents a tree of CardContent that can be repeated on a Card.

CardConfigMigrations

Repetitions

Conceptionally, a Repetition is a block of repeated content with its answers. For instance, three Funders would correspond to three separate Repetition records.

Concretely, a Repetition is a tree model where the root of the tree corresponds to a CardContent of type repeat. This repeat CardContent is a tree that contains all the content that might be repeated. For instance, the tree might have a child content that asks for the funder's name. It is useful to think of this CardContent tree as the canonical representation of content that might be repeated on a card. If a user had added three different funders on a paper, there would still only be one repeat CardContent.

Although the root of the Repetition content corresponds to a single CardContent of type repeat, it itself is also a tree and will contain children that each correspond to the canonical piece of CardContent. For example, if a user has added three funders, the root of the Repetition will have a relationship to the repeat CardContent. It will also have a child that corresponds a different CardContent that is responsible for asking for a funder name. In other words, each node of the Repetition tree corresponds to the a node in the canonical CardContent repeat tree. They mirror each other.

The children of a Repetition each correspond to a piece of CardContent so that it can obtain question text, an Answer record so that it knows the user's answer, and an owner (typically the CustomCardTask). This is how the answers for two different funders are kept separately from each other.

Finally, a Repetition uses acts_as_list to maintain positional order. This is necessary for being able to reorder a list of content in a sensible, performant manner.

Appendix

ERD Diagram

In Listing carderd , the rails-erd gem is used to generate the diagram in Figure cardmigrationerd

rake erd only="Card,CardVersion,CardContent,CardTaskType,Answer,Task,Repetition,EntityAttribute" filename="cardmigrationerd" filetype="pdf" title=false inheritancee=true indirect=true

Attachments:

quadrant-grid.png (image/png)

cardmigrationerd.pdf (application/pdf)