Home / OpenIDEO Usability Assessment Final Report – Spring 2014

OpenIDEO Usability Assessment Final Report – Spring 2014


Ivan Buchanan, Pierce Gordon, Meredith Hitchcock, Ronnie Jakobsen

Individual Contributions

Ivan Buchanan: elaborated usability test guide and protocol, performed heuristic evaluation of the OpenIDEO website, observer and note-taker in a usability test, compiled and edited video examples for the project presentation.

Pierce Gordon: communicated with OpenIDEO, performed in-depth research upon OpenIDEO’s past iterations and future directions, identified and contacted potential one-on-one interviewees, performed one-on-one interviews.

Meredith Hitchcock: project management, developed focus group guide and protocol, wrote consent forms, moderated some usability tests, performed heuristic evaluation of the OpenIDEO website, identified and contacted potential usability testers, prepared the web version of the report, compiled and edited the video for the final written report.

Ronnie Jakobsen: moderated some usability tests, note-taker for some usability tests, performed heuristic evaluation of the OpenIDEO website, note-taker for one-on-one interviews, prepared software setup for usability tests.

Group Activities

  • Analyzing the data, notes and observations on heuristic evaluation, usability tests and one-on-one interviews
  • Highlighting and commenting each team member’s experience and insights
  • Identifying and voting on the main usability pain points
  • Writing the final report
  • Elaborating conclusions and recommendations

Executive Summary

OpenIDEO is an open collaboration platform focused on social good. By using the framework of Human-Centered Design and by harnessing the capabilities of 60,000+ members (at the time of this writing), they aim to address problems in many disparate fields by collecting rich and disparate research, brainstorming ideas, and inciting community feedback. However, the community that actually contributes to the challenges is drastically smaller – approximately 4,000 members. Moreover, research shows that a large percentage of that 4,000 only participate in one challenge, and then barely contribute after said challenge is complete. Therefore, for this usability study, we decided to focus upon understanding user retention and user collaboration issues in OpenIDEO. We used three research methods for the experience: heuristic evaluation, usability tests with novice users, and expert user interviews. Through these methods, we found three main points of tension: Issue 1: The main function of the website, the contribution process, was overly complex. To remedy this, we suggest:

  • Reduce the number of clicks from first page to the contribution form.
  • From the “Challenges” page, clicking on a challenge should lead to the current stage of the challenge.
  • Offer suggestions about the recommended length of text for ALL boxes in the contribution form.

Issue 2: Language inconsistency on the OpenIDEO website confused users. To remedy this, we suggest:

  • Make language around challenge involvement consistent.
  • Rename the “Forum” tab to “Help”.

Issue 3: Users have difficulty understanding important facts of the website, such as the roles of sponsors, staff, and other designers, and the role of the website altogether. To remedy this, we suggest:

  • Explicitly state the different actors’ involvement at each stage. Who picks the “winners”? What’s the sponsor’s role?
  • Explain the site concept with concrete examples using the challenge timeline as the main thread.

Regardless of the usability issues of the website, our members did like the website. In the spirit of the website’s tenet that it is ‘always in beta,’ we offer this way to improve the website, to keep users involved in the process of designing better, together.

Background About the Project

OpenIDEO, a subsidiary program of the design firm IDEO, is an online collaborative design community that aims to address interdisciplinary challenges in varied fields. IDEO is well known as an innovative consulting company which uses user-centered design to address issues in multinational enterprise, international development, community engagement, and technological need through product, service, and experience-based interventions; and OpenIDEO aims to use the same framework with many more designers able to contribute. Organizations such as Amnesty International Steelcase Inc, Case Western Reserve University’s Power Center for Sustainable Value have all been sponsors of challenges with varied and ingenious solutions. As of this writing, the program has 22 completed challenges and two challenge in progress, ranging from such fields as affordable learning tools for the developing world, increasing bone marrow donors, and celebrating companies who innovate for world benefit. OpenIDEO differs from a standard “Question and Answer” type forum in several key ways: the process of developing solutions is split into several key phases, and the process of submitting a solution encourages iteration and building off of prior submissions. The platform goes through five main phases on the Website: “Research,” “Ideas,” “Evaluation,” “Applause,” and announcing the winner (see figure A). During the “Research” phase, the designers are encouraged to “share existing stories, tools, case studies, and examples.” During the “Ideas” phase, OpenIDEO asks designers questions such as “How would you solve this problem?” and suggests the designers consider “…areas of opportunity where new ideas might flourish – to spark our creative efforts.” After the new ideas are developed, the designers have a chance to evaluate the ideas on a collection of qualitative metrics, including their level of innovation sustainability, and its capacity to address the problem at hand. After this time, IDEO, the OpenIDEO staff, and the challenge sponsors come together and decide which of the designs are good enough to back; usually nine to ten are chosen [1].