Saturday, October 20, 2012

Open Ed 2012 : Preliminary Results of Testing the OERPUB Editor designs


Chinese food in Vancouver
Open Ed 2012 just closed in Vancouver and I wrote this on my way home while looking forward to the community of family. I am extraordinarily lucky to work with a group of smart, passionate, and creative colleagues, whose hard work paid off for us handsomely in Vancouver. We were bringing designs for what we hope is a sustainable approach to authoring education resources that can be remixed, adapted, improved, and extended.

Since Open Ed attracts a diverse community of educators, technologists, and visionaries, we brought with us two things we wanted to vet. We wanted to learn from this group and attract partners to our effort, and we have succeeded in both. First, we brought high-fidelity (which here means it looks great and parts of it work :-)) mockups to test with authors at the conference. Secondly, we brought a technical architecture and working prototype for delivering the features in the mockup as extensible, open-source software. In addition to me, two of our user interface designers, Max Starkenburg and Adrian Garcia, came and also acted as test monitors and test analysts. We recruited Megan Beckett from Siyavula to our cause since she was already coming to talk about Siyavula's textbooks, community authoring, and badges. She helped beta-test and refine the tests, and then even acted as a test monitor for one of the days. To convey the technical architecture, two of the leading developers on the project, Marvin Reimer and Phil Schatz, joined and talked with anyone who cared about the technology, which turned out to be a substantial number of
people and projects at the conference. 

Testing the interface at OpenEd.
We had close to 30 people go through the tests of the editor! People spent between 15 and 45 minutes helping test. For the curious, here are the tasks they were given.

What we learned. Well, Adrian and Max and Megan have lots of data we haven't yet gone through, yet, but we spent Thursday evening after the conference working through the highlights. Our overall System Usability Score was a solid B, which seems like exactly where we should be right now. Our ambitions are sky high and the best design principals have to be tested.  

Analyzing the results (qualitative)

Things that tested well : the approach to math editing, the math cheat sheet, the drag and dropping of semantic features, the highlighting box and options gear-icon.

Things that need some rethinking (luckily with a whole lot of user data to inform them): the exact way we presented image sourcing information was confusing and needs a rework. Some people loved it, but some people didn't understand it. One of the semantic features we tested was "exercis
es", the Connexions inspired structured question/answer format. The naming of the parts wasn't intuitive. The other interesting problem was that in the editor the solution to the question shows directly below the question. Authors didn't like that one bit, because they don't want readers to see the solution right there. Earlier designs hid the solution, but they were jumpy, so we need to figure out how to cleanly show that the solution will be hidden from readers. We had a major problem with table editing -- no one could find how to add/delete rows. This one is so easy to fix, though, because we know everywhere people looked for it and we will just make those work. That is the beauty of this kind of testing.

Engaging new partners: Since we have designed the editor so that it can be embedded in a wide-variety of projects, built
it on Aloha, an HTML5 editor with active contributors, and since the underlying format of the content is HTML5, we were successful in enticing the serious interest of other projects at the conference. There is still a lot of hard work to successfully engage participation and move forward. I am looking forward to that process. It has been my dream for this project since the beginning.

Wednesday, October 17, 2012

Open Ed 2012, Vancouver, Invitation to talk and User Interface Testing

We are proud to present our newest OERPUB importers at Open Education for Education 2012 in Vancouver! If you want to see them in action, create an account at the Open Repository cnx.org and start importing and transforming your Google Docs, Word or MS Office files or even webpages here: remix.oerpub.org.

Our motto for authoring educational content is:

Everything in, Everything out, Everywhere.

Find out what that means at Kathi and Marvin's talk at 
We are currently creating a WYSIWYG authoring tool for easily remixable OER. It works with Connexions out of the box, and the new HTML5 format means you can customize it to work in other projects and content repositories.

We are testing the editor here at Open Ed. Come help us, and have a cookie, in room C215 at 
  • 2:00 pm Tuesday
  • 1:30 pm Wednesday
  • 10:30am on Thursday
Source: http://www.flickr.com/photos/shardsofblue/5875237526/in/photostream/
Image by Shades of Blue, Roxanne Ready. Used by Creative Commons license.

Usability goals for the editor:
  1. The editor must support the incremental development of educational content.
  2. The editor must be self-explanatory enough so that novice (first time) users can edit without training, yet powerful enough so that experienced users are efficient. Experienced users those who have published 5 modules, or 1 textbook, or that have received training.   
  3. The editor must support the activity of infrequent users by leading the user by hand through tasks when necessary, so they need not remember details of proper procedure between uses.
  4. The editor must be designed free of cultural assumptions to be used by persons from diverse cultures and geographical locations.
  5. Using the editor should not require knowledge of the underlying technology.
  6. The editor should be fun to use by all groups of users.
  7. Authors that create educational content should think they would like to use the editor to create content for learners.
  8. Authors' content should evolve toward more semantically rich "markup". For example, over time using the editor should result in increased use of headers, exercises, examples, and notes. Authors' images should be more likely to contain descriptions that aid visually impaired learners. Author's tables and images should be more likely to include titles, captions, and source information.
 See some videos of our work with the editor: