improve display of correctness in all question components

[oliverfoster]

Glossary

Widget – The widget is the component section between the instruction text and the question submit buttons. It holds the question items selection user interface.

Introduction

We have been discussing how to display correctness for core question component widgets. The widget correctness is distinct from the overall question correctness indicator and label that are located between the submit and feedback buttons.

Research summary

• Storyline - has no default marking or correctness display, it must be made up at build
• Elucidat and articulate – marking is only for the correctness of the question in its entirety, there is no way to see the completed question and check item correctness
• Web quizzes – correct and incorrect selections are listed in the feedback only

Styles

There are two distinct proposed styles for widget marking and correctness display.

Distinct styles:

1. Marking: Show user selection correctness
2. Correctness: Show item correctness

With a pure marking style, only the user’s selections are explicitly indicated to be correct or incorrect.

With a pure correctness style, item correct and incorrect indicators are shown next to each visible item, alongside a user selection indicator. The user must infer what they did wrong, their item selections are not directly marked for correctness.

Our current MCQ user interface initially shows only the user’s selections, occluded by a correctness indicator.

For each question there is currently a toggle button switching the ticks and crosses between showing only the correctness of the user’s selections and between showing only the correctness of the correct items. It is agreed that there is a lot of cognitive effort required to understand the function of the toggle button.

Consensus

With the MCQ, there is consensus on:

• Showing the correctness ticks and crosses to the right of the user’s selection
• Removing the state toggle button
• Showing the correct items and their correctness always
• For incorrect items, showing the correctness of at least the incorrect selected items
• Showing selection indicators independently of the correctness indicators

Discussed issues

There were two subject important issues, specifically with multi-select MCQs, when:

1. The user has selected fewer correct items than are required
2. The user can select fewer items than there are correct items

Work to do

Mock-up pure correctness style for each core question component, where the toggle button is removed:

• Text input will show a comma separated list of correct options. An item correctness icon should sit to the right of the text input as it does currently.
• Matching will show a comma separated list of correct options below each dropdown. A correctness icon should site to the right of the dropdown as it does currently.
• Slider will show ticks and crosses across the appropriate ranges, below the numbers .
• MCQ will have the ticks and crosses to the right of the item text. Item selection will be represented using the usual item selection colour and item selection icons.
• GMCQ will have ticks and crosses centred and below the item text. Item selection will be represented using the usual item selection colour and item selection icons.