Testing of the editor designs at the Connexions Conference

The OERPUB UI team, Adrian Garcia and Max Starkenburg, were hard at work during the sprints after the Connexions Conference testing  the usability of new features that have been designed for the editor. The testing, as usual, was highly informative.

They were testing this mockup:

 

New Features

It has several new features or combinations of features since the last set of testing:

• Pedagogy templates in the toolbar. This was implemented for instances in which an organization embeds the editor but doesn't have screen real estate for the toolbox. Some organizations may even want both.
• An inline menu. This will enable users to make key terms, code font, or foreign text, and to remove formatting. This inline menu appears when authors hover over highlighted text, over styled text (which they could convert to a more semantically rich choice), or over key terms, etc.
• Pedagogy options menu. This menu provides a list of all possible pedagogy templates and allows users to select which ones are visible.
• An icon for inserting videos into documents. This will enable users to search for videos hosted on various sites and embed them in their document.
• Quotation template. These enable authors to create quotations.

Questions and Answers

1. Can users discover important features of the editor on their own? The pedagogy toolbox is much more discoverable than the pedagogy menu. In the menu, "Add a new" sounded like a toolbar configuration rather than something to include in the author's content. We were trying to avoid "insert" which other research has shown to be confusing to authors, but we will need to work on the wording of this menu and make it more inviting to try.
2. How discoverable is the inline context menu? The inline context menu was the least discoverable of the features. Perhaps changing the color of the menu's icon so that is is brighter will increase discoverability. It may also help if it appears more quickly. This mockup shows a possible fix.
3. Will it be a surprise to users that the inline menu does not provide any styling options? Might the inline menu distract from styling? This doesn't appear to be a problem for authors. They seemed fine accessing styling exclusively from the top toolbar. Keyboard shortcuts for bold and italics should be supported (and they are in the implemented editor!)
4. Are participants able to use and understand the pedagogy? Can they successfully customize it? No change from prior tests. Authors are able to use the tools easily. Sometimes they bypass the templates and just type things in. In this test, authors struggled with the widget for altering the labels (exercise->question for instance).
5. Is it confusing to have two pedagogy menus in the editor? Basically, yes. The confusion abates after experimenting with both. Most authors concluded they were the same, but we don't know how high their confidence was. We aren't sure whether we will try to fix this problem.
6. Are participants able to properly insert videos? Yes, the workflow worked well. The icon was hard for authors to find, but they did find it. It would be good to create a better icon, but we can probably live with this one for a while.
7. Are participants able to properly insert quotations? Will they have issues with their appearance? The workflow was fine. Participants wanted them to be styled differently. They will be more likely to use the semantic template if it is visually pleasing. This is an easy fix, but will only affect the content in the editor and when saved from the editor. Repos may style quotes differently.

The Full Report

You can download the full report in PDF format. We are working on a more remixable format for the report.