manual.Rmd

---
title: "Project Manual"
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(echo = TRUE,
                      error = TRUE)
```

# Stages {#stages}

There are several stages of the project:

1. identify as many measures as possible that assess externalizing problems and related constructs (and obtain the freely available ones)
    - fill out the [`Measures`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0) tab of the `Existing Measures and Items` spreadsheet
    - add the instrument (as documents/PDFs) to the [`Instruments`](https://drive.google.com/drive/u/0/folders/1UL9_9tbKPQeKNLftlBTdWI9kVcqEn51f) folder
    - add the source reference(s) (as documents/PDFs) to the [`Source References`](https://drive.google.com/drive/u/0/folders/1yWhRvr4pHbRIpYhiVvvghk4g3wGq0-GV) folder
    - add the scoring manual(s) (as documents/PDFs) to the [`Scoring`](https://drive.google.com/drive/u/0/folders/1ptIl7G2ksPdj3YZVxYiPGjgjayw_borY) folder
1. obtain as many measures as possible that are not freely available
    - i.e., add additional instruments to the [`Instruments`](https://drive.google.com/drive/u/0/folders/1UL9_9tbKPQeKNLftlBTdWI9kVcqEn51f) folder
1. identify the scales and subscales for each measure
    - i.e., fill out the [`Scales and Subscales`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=618359966) tab of the `Existing Measures and Items` spreadsheet
1. identify the items for each scale and subscale for each measure
    - i.e., fill out the [`Items`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=1930981316) tab of the `Existing Measures and Items` spreadsheet
1. identify the facet assessed by each item
    - i.e., fill in the `facet` column of the [`Items`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=1930981316) tab of the `Existing Measures and Items` spreadsheet
1. select and edit the items to ensure each facet is assessed comprehensively at each age
1. have the focus groups of experts review, refine, and rate the construct relevance of the items at each age
1. select the final item pool for each age
1. recruit participants to test the item pools
1. select items for the final measure at a given age for a given rater type based on testing
1. recruit a nationally representative sample of participants whose informants who will rate them on the measure
1. generate age- and sex-based norms
1. develop a computerized adaptive test
1. develop a computerized scoring system
1. develop translated versions

# Instructions for all Spreadsheets

- Use Roboto font size 10
- Use an en dash (`–`; i.e., not a hyphen) to indicate a range:
    - e.g., 1–18 (not 1-18)
	- an en dash is technically correct; in addition, spreadsheets often read 3-7 as March 7th, but they correctly read 3–7

# Stage 1: Identify Measures {#identifyMeasures}

This stage involves 1) identifying as many measures as possible that assess externalizing problems and related constructs and 2) obtaining the freely available ones.
To achieve this, conduct a thorough literature search using the suggested search terms ([below](#searchTerms)).
As you identify measures, fill out the `Measures` tab of the `Existing Measures and Items` spreadsheet and (if the instrument is freely available) add the instrument to the `Instruments` folder.

## Search Terms {#searchTerms}

The following are search terms to use for identifting measures that assess externalizing problems and related constructs:

- externalizing
- antisocial
- disruptive
- conduct problems
- callous-unemotional
- aggression
- oppositionality
- inattention
- hyperactivity
- ADHD
- conduct disorder
- oppositional defiant disorder
- antisocial personality disorder

## Fill out the `Measures` tab {#measuresTab}

Fill out the `Measures` tab of the `Existing Measures and Items` spreadsheet, located [here](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0).
If there are multiple editions of the same measure, select (and specify) the most recent edition (e.g., `Behavior Rating Inventory of Executive Function–2` for the 2nd Edition of the BRIEF).

### Data Dictionary {#dataDictionary-measures}

- `measureName`: the full official measure name
- `abbreviation`: the (capitalized) abbreviation of the measure name; duplicates are allowed
- `measureClass`: the broader assessment family/system; if multiple measures are from the same assessment family/system (e.g., both the Child Behavior Checklist and Teacher's Report Form are part of the Achenbach System of Empirically Based Assessment) provide a short label for the measure class; if the measure is not part of a broader measure class, use the lowercase version of the abbreviation
    - should be lowercase with no punctuation
    - only measures that are part of the same measure class should have the same label for `measureClass`; if measures from a different measure class have the same label for `measureClass`, this may have to include a disambuating element (such as the author name or year of publication) so measures from different measure classes are not given the same label
- `instrument`: the lowercase version of the abbreviation
    - should be lowercase with no punctuation
    - this may have to include a disambuating element (such as the author name or year of publication) so that there are no duplicates in the `measure` column
- `measure`: a calculated field that combines the `measureClass` and the `instrument` into one field, separated by an underscore; there should be no duplicates (duplicates are highlighted in red); this field is the unique identifier of the measure
    - do **not** enter this manually; drag the formula from other cells in the column to calculate
- `type`: the type of measure; one of:
    - `questionnaire`
    - `interview`
    - `observation`
    - `other`
- rater: the rater for whom the measure was designed to be completed; one of:
    - `parent`
    - `teacher`
    - `caregiver`: caregivers who are not parents or teachers (e.g., other family members or care providers)
    - `self`
    - `peer`
    - `nurse`
    - `clinician`
    - `nurse`
    - `sibling`
    - `other`: any informant who knows the child well
- `ageLowerYears`: the lower limit of the ratee's age range for which the measure was designed to be completed (in years)
- `ageUpperYears`: the upper limit of the ratee's age range for which the measure was designed to be completed (in years); if no upper limit is specified in the documentation for the instrument, use the oldest age in the original sample. If the oldest age is not available and the scale was designed to be used without an upper age limit, specify `99`.
- `ageLowerMonths`: the lower limit of the ratee's age range for which the measure was designed to be completed (in months)—especially relevant for infants/toddlers
    - this is calculated automatically using a formula based on `ageLowerYears`, so you should not enter this unless the measure specifies ratee's age range in months rather than years
- `ageUpperMonths`: the upper limit of the ratee's age range for which the measure was designed to be completed (in months)—especially relevant for infants/toddlers
    - this is calculated automatically using a formula based on `ageUpperYears`, so you should not enter this unless the measure specifies ratee's age range in months rather than years
- `responseOptions`: the response options for each item
    - Use a semicolon (`;`) to separate response options
    - For example: `0=Not True (as far as you know); 1=Somewhat or Sometimes True; 2=Very True or Often True`
- `proprietary`: whether the measure is proprietary; one of:
    - `yes`
    - `no`
- `notes`: any notes on the measure
- `url`: the URL for the measure (i.e., where to access or purchase it); you can include multiple URLs if they would be useful (separate them with a semicolon)
- `keyReferences`: any key references for the measure, including seminal references, references that include the items, etc.; you can include multiple key references if they would be useful (separate them with a semicolon)
- `instrumentPDF`: whether or not we have the PDF of the instrument (indicate yes with an `x`); if we are unable to obtain it, specify why not (e.g., `need to order` or `cannot find`)
- `sourceRefPDF`: whether or not we have the PDF of the source reference (indicate yes with an `x`); if we are unable to obtain it, specify why not (e.g., `need to request` or `cannot find`)

- `availableToOrder`: whether the instrument is available to order from publisher or instrument creator (indicate yes with an `x` if instrument is available to order); if we are unable to obtain it, leave blank

### How Rows are Sorted

The rows in the `Measures` tab of the `Existing Measures and Items` spreadsheet are sorted by the `measure` column, which sorts first by `measureClass` and then by `instrument`.
When adding rows for new measures, place the new row in the appropriate location accordingly.

### How to Indicate Measures from the Same "Measure Class" {#measureClass}

As described in the [Data Dictionary](#dataDictionary-measures), the "measure class" is the broader assessment family or system.
Multiple instruments can belong to the same measure class (e.g., both the Child Behavior Checklist and Teacher's Report Form are part of the Achenbach System of Empirically Based Assessment).
If there are multiple measures within the same measure class, use borders to indicate that measures have the same measure class.
For instance, the first assessment in a given measure class should have a top border, and the last assessment in a given measure class should have a bottom border.

## Add instruments to the `Instruments` folder {#addInstruments}

If the instrument is freely available—i.e., publicly available or provided in a publication—add the instrument to the `Instruments` folder, located [here](https://drive.google.com/drive/u/0/folders/1UL9_9tbKPQeKNLftlBTdWI9kVcqEn51f).
Save the PDF using the capitalized abbreviation of the measure name.

## Add source reference(s) to the `Source References` folder {#addSourceRefs}

If the instrument is freely available—i.e., publicly available or provided in a publication—add the source reference(s) to the `Source References` folder, located [here](https://drive.google.com/drive/u/0/folders/1yWhRvr4pHbRIpYhiVvvghk4g3wGq0-GV).
Save the source reference(s) using the following format, depending on the number of authors:

- one author: "Author, Year, Abbreviation.pdf"
    - e.g.: "Achenbach, 2000, CBCL.pdf"
- two authors: "Author1 & Author2, Year, Abbreviation.pdf"
    - e.g.: "Achenbach & Rescorla, 2000, CBCL.pdf"
- three or more authors: "Author1 et al, Year, Abbreviation.pdf"
    - e.g.: "Achenbach et al, 2000, CBCL.pdf"

## Add scoring manual(s) to the `Scoring` folder {#scoring}

If the instrument is freely available—i.e., publicly available or provided in a publication—add the scoring manual(s) (as documents/PDFs) to the `Scoring` folder, located [here](https://drive.google.com/drive/u/0/folders/1ptIl7G2ksPdj3YZVxYiPGjgjayw_borY).
Save the PDF using the capitalized abbreviation of the measure name.

# Stage 2: Obtain Measures {#obtainMeasures}

This stage involves obtaining as many measures as possible that are not freely available—e.g., measures that are proprietary (e.g., by purchasing them) or are not provided in publications (e.g., by contacting the authors).
After obtaining measures, complete steps of Stage 1 with the newly obtained measures, including:

- add the instrument (as documents/PDFs) to the [`Instruments`](https://drive.google.com/drive/u/0/folders/1UL9_9tbKPQeKNLftlBTdWI9kVcqEn51f) folder (described [here](#addInstruments))
- add the source reference(s) (as documents/PDFs) to the [`Source References`](https://drive.google.com/drive/u/0/folders/1yWhRvr4pHbRIpYhiVvvghk4g3wGq0-GV) folder (described [here](#addSourceRefs))
- add the scoring manual(s) (as documents/PDFs) to the [`Scoring`](https://drive.google.com/drive/u/0/folders/1ptIl7G2ksPdj3YZVxYiPGjgjayw_borY) folder (described [here](#scoring))

# Stage 3: Identify the Scales and Subscales for Each Measure {#scalesSubscales}

This stage involves identifying the scales and subscales for each measure.
For a measure, identify the names of the scales and subscales as specified by the authors.
Fill out the [`Scales and Subscales`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=618359966) tab of the `Existing Measures and Items` spreadsheet.
Follow the Camel case naming convention when adding the names of the scales and subscales to the Measures table (e.g., "Rule Breaking Behavior" = `ruleBreakingBehavior`).
If a scale or subscale does not have a clear name, use the naming convention to label with 1–3 words that best describe the scale or subscale.

## Data Dictionary {#dataDictionary-subscales}

- `measure`: the unique identifier of the measure; combines the `measureClass` and the `instrument` into one field; should be the same value as the `measure` in the [`Measures`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0) tab of the `Existing Measures and Items` spreadsheet; see the [Data Dictionary for the Measures tab](#dataDictionary-measures) for a detailed description of the column
- `higherOrderScale`: the higher-order scale(s) (if any) for the measure; a higher-order scale is a scale that subsumes multiple subscales; should be "n/a" if the measure does not have a higher-order scale
- `subscale`: the subscale(s) of the measure;

# Stage 4: Identify the Items {#identifyItems}

This stage involves identifying the items for each scale and subscale for each measure.
To do this, fill out the [`Items`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=1930981316) tab of the `Existing Measures and Items` spreadsheet by transcribing the items from the PDF of each measure that is located in the [`Instruments`](https://drive.google.com/drive/u/0/folders/1UL9_9tbKPQeKNLftlBTdWI9kVcqEn51f) folder.

## Data Dictionary {#dataDictionary-items}

- `itemWording`: the wording of the item
- `facet`: the facet that is assessed by the item
- `reverseScored`: whether the item is reverse scored; one of:
    - `yes`
    - `no`
- `measure`: the unique identifier of the measure that the item is from; combines the `measureClass` and the `instrument` into one field; should be the same value as the `measure` in the [`Measures`](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0) tab of the `Existing Measures and Items` spreadsheet; see the [Data Dictionary for the Measures tab](#dataDictionary-measures) for a detailed description of the column
- `itemNumber`: the number of the item on `measure1`
- `higherOrderScale`: the higher-order scale of the item on `measure`; see the [Data Dictionary for the Scales and Subscales tab](#dataDictionary-subscales) for a detailed description of the column
- `itemSortOrder`: the sort order (ascending) for an item in a given measure; the sort order should reflect the order that we want the items presented—either based on subscale groupings or based on the order in which they are present in the instrument
- `subscale`: the subscale of the item on `measure`; see the [Data Dictionary for the Scales and Subscales tab](#dataDictionary-subscales) for a detailed description of the column

<!--
The following columns are filled out only if the same item appears on multiple measures within the same [measure class](#measureClass):

- `measure2`
- `measure2itemNumber`
- `measure2HigherOrderScale`
- `measure2Subscale`
- `measure3`
- `measure3itemNumber`
- `measure3HigherOrderScale`
- `measure3Subscale`
- `measure4`
- `measure4itemNumber`
- `measure4HigherOrderScale`
- `measure4Subscale`
- `measure5`
- `measure5itemNumber`
- `measure5HigherOrderScale`
- `measure5Subscale`
-->

# Stage 5: Identify the Facet Assessed by Each item {#identifyItemFacets}

To prevent multiple people from coding the same scale, write your name in the itemFacet column next to each measure you are working on in ['Measures'](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0).
Using the list of ['Facets'](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=639387683), identify the facet of each item's for the scale you have selected to work on here: ['Items'](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=1930981316)
You can leave comments in ['Items'] for yourself to address.
If questions come up for the group to discuss or if you have ideas for revising the facets, you can use Slack.
Mark an x by any scales you've completed (i.e., which have all the item facets identified) in the itemFacet column in ['Measures'](https://docs.google.com/spreadsheets/d/1luduZayyVmONdxBCcvrMGVv1vshhHzQ7eEOLg1aaMxE/edit#gid=0).