Driving Data in Research: A Roadmap to Enhanced System Usage and Capabilities

An institute within a large federal agency focused on medical research turned to Censeo to conduct a multi-faceted system assessment and provide insightful recommendations to drive improved usage.

Situation

An institute within a large federal agency hired Censeo to provide a diagnostic assessment for one of its many tools aimed to support the goal of an interoperable health information technology infrastructure. The specific system, which provides researchers with access to data elements and forms, was troubled by low usage and an unclear mission objective for the future. Censeo was tasked with assessing the system from three lenses: 1. Identifying primary users and analyzing their needs. 2. Conducting usability testing. 3. Reviewing the technical infrastructure with consideration for long-term sustainability, functional alignment with other systems, and interoperability and capacity for connection with other systems. Based on data collected during these phases, Censeo laid out a strategic roadmap for the client, containing phased recommendations and detailed key activities to drive improved usage of not only the system, but data elements more broadly within the federal agency.

Approach

Censeo followed a four-step process as represented in Figure 1 and described in more detail below.

Perform Environmental Scan

Before launching into data collection, an environmental scan was performed to build an understanding of the organization’s previous work. This consisted of interviewing the system team, conducting a document review, and performing additional research. This work informed the methodology for the remaining process steps such as identifying key stakeholder groups to be interviewed, system use cases to be evaluated in usability testing, and initial hypotheses regarding system pain points to be further explored.

Collect Data

• Stakeholder interviews. Several activities were performed to collect data for the assessment, beginning with stakeholder interviews. Informed by the environmental scan, Censeo conducted 47 stakeholder interviews, with a total of 67 interviewees, including the agency’s leadership, frontline staff, and intramural and extramural researchers.
• Usability testing. The more conceptual nature of the stakeholder interviews was supplemented by usability testing to help identify and rectify potential usability deficiencies in the system. Eleven sessions were held with participants. A set of overarching use cases was identified, which then translated into specific tasks that users would commonly face when using the system. During the usability testing, the participants were asked to perform these tasks. As moderators guided testers through these tasks, the users’ onscreen movements were tracked with usability testing, allowing for calculation of various metrics. Users’ in-session commentary and post-session survey responses were also noted to add qualitative insights to the findings.
• Technical infrastructure review. For the technical infrastructure review, Censeo identified relevant technical documents as well as system owners to be interviewed. A number of documents were analyzed and 16 staff members were interviewed, representing 13 systems (the system being assessed and 12 other relevant systems).

Recommend Actions

Analysis of the stakeholder interviews, usability testing, and technical infrastructure review identified key challenges limiting system usage. Informed by insights from the interviews and testing, general best practices, and Censeo’s deep experience, we developed a set of over 20 recommendations to address these challenges. We outlined detailed activities needed to implement each of these actions within a broader phased approach, sharing them with the client for input.

Implement Prioritized Actions

After refining these recommended actions and their related implementation activities with the client’s input, we then prioritized select implementation activities as ones that could be executed using the data collected from the stakeholder interviews, usability testing, and technical infrastructure review. These prioritized activities provided the client with a head start in implementing the recommended actions.

As part of the engagement, Censeo identified six stakeholder segments, each with specific goals and pain points. Here is an example of a typical “journey” of one of the stakeholder groups, which outlines the stakeholder’s activities, emotional state, and possible engagement tactics as it relates to the lifecycle of a research project.