Research Design

We performed an uncontrolled interventional study design. Field observations and interviews were performed from September 2010 until October 2010. Initially field observations with clinicians, nurses, and other staff were conducted during the first two weeks of September 2010. During this time a total of 10 hours of observations took place across two separate sessions. In the following weeks the researcher made visits to the clinic to interview the clinic staff and obtain more specific information. We created Unified Modeling Language (UML) activity Diagrams of the clinical and research workflows based on the information retrieved.

We then developed workflow analyses from November 2010 to May 2011, using a Lean methodology to identify workflow dissonance that led to the proposition of a new dissonance classification system; subsequently we proposed and implemented workflow modifications in order to address the identified dissonances. This new workflow was implemented in June 2011. Finally we observed the number of remaining dissonances, screened and enrolled patients from September 2010 until November 2011.

Intervention

In the context of this case study our intervention refers to the restructuring of workflow based on the identification of dissonances between research tasks and clinical practice. Intervention was carried out in four phases: (1) research and clinical task observations and workflow modeling, (2) dissonance identification through workflow analysis, (3) the development of a workflow modification plan, and (4) the implementation of those changes as outlined in this case study.

Observation of Research and Clinical Tasks

Workflows were initially assessed through field observations followed by interviews with doctors, nurses and the research team to confirm the collected information. Observations and interviews were conducted with the goal of understanding the major tasks and their corresponding characteristics. Although a formal time and motion study was not conducted, we were able to estimate the duration of the tasks based on observations and reports from the staff. Finally, we modeled the research and clinical workflow using a series of UML Activity Diagrams.

All activity diagrams were created using Astah Professional 6.6.3 (model version 36) evaluation license [18] and were modeled according to UML version 2.0 [19]. We chose UML Activity Diagrams because they are used in sequences that model tasks with both conditional and parallel behavior [20]. In these diagrams, the starting point is indicated by a darkened circle, called initial node. The ending point is indicated by a darkened circle with a border, referred to as final node. Rounded rectangles represent tasks. These tasks can either be a real-world process, such as a patient assessment or the execution of a software routine, such as prescribing a medication using a prescription writer [19] (Figure 1A). Transition arrows connect the different tasks. Branches and merges are used to represent conditional behavior and are represented by diamonds in the diagram. Branches have one incoming transition and several outgoing transitions (Figure 1B). Different decisions lead to different outgoing paths. For example, when preparing a treatment plan for a patient a physician will evaluate the need for prescribing a medication. If he believes the patient should receive pharmaceutical treatment he will use the prescription writer for that task, otherwise he will take no action. Merges indicate the end of a conditional behavior that was started by a branch. Forks and joints indicate the existence of simultaneous tasks, both being represented through black bars (Figure 1C).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Schematic representation of the UML Activity Diagram elements.

A. Initial and Final nodes, tasks and transition arrows; B. Branches and merges; C. Forks and joints.

https://doi.org/10.1371/journal.pone.0075167.g001

Workflow Analysis for Identification of Dissonances

We defined workflow dissonances as discrepancies between research and clinical workflow. We used the Lean method approach for identifying workflow dissonances that generates research inefficiencies [21]. The Lean method is based on the Toyota model, and seeks to eliminate non-value added elements from the process, known as waste [21]. Waste includes any activity that absorbs money, time, and people but does not create value [22]. Value expresses the end user needs (customer) [22]. Therefore, for the evaluation of value it is necessary to determine who is the end user and what is that person’s point of view [21]. In this project we assumed the researcher was the end user, and that the value was the high quality data in a short period of time.

There are eight categories of activities identified as waste in the literature [22,23]. They are described as follows:

1. 1. Waiting is the time spent waiting for the next process to be initiated;
2. 2. Overproduction, present when the first of two sequential step processes is faster than the following step;
3. 3. Inventory, which indicates products waiting for processing or to be delivered to the end user;
4. 4. Defects, that demands rework for fixing problems with the product or the discard of the problematic products;
5. 5. Transportation is related to the unnecessary movement of products;
6. 6. Overprocessing, related to presence of processing characteristics not valued by en users;
7. 7. Wasted motion, related to unnecessary movement of employees;
8. 8. Underutilized human skills and time, present whenever a worker or equipment is idle [22,23].

The value stream defines the steps required to complete a process and the examination of the value stream reveals possible wastes within a process [21,22]. One should map the value stream and analyze it for non-value added waste and redesign the flow to remove as much of non-value added waste as possible and standardize the ongoing process. Batches and queues within a process must be eliminated so processes can flow uninterrupted and processes must be completed in order to allow the next step in the process to start [21].

We analyzed the UML Activity Diagrams to identify wastes in research and clinical workflow, and then analyzed the tasks associated with wastes using the conceptual framework, “Workflow Elements Model” proposed by Unertl and collaborators [24]. The model has both a specific and a pervasive level. The components of the specific level are the people performing actions (actors), the physical and virtual tools used (artifacts), details of actions performed (actions), characteristics describing the actions (characteristics) and the end products of the actions (outcomes). The pervasive levels (context, temporality and aggregation) have components that apply over the specific elements of workflow: the context includes physical and virtual workspace and organizational factors. The temporal factors are related to scheduling, temporal rhythms and coordination of events. Aggregation refers to the relationship and interaction among different tasks and actors [24].

We used components of the specific (actors, artifacts) and pervasive level (duration, location, period of the day) from this framework to further describe the tasks associated with wastes. Information relating to identified wastes (type and description) and task description were taken into account. We chose to describe the tasks in a broad way since this was an initial assessment of the workflow. In subsequent phases of workflow evaluation a more refined granularity could be used to reveal issues related to the tasks that a more general analysis was not able to show. We also registered the data variables generated at each task as this indicates that the task is of particular importance for the end user (researcher).

We investigated which workflow component was associated to the dissonance and named it accordingly. The designation specifically alluded to the aspect of the activity that represented the source of inefficiency. Most importantly, this inefficiency indicates a flawed aspect of the integration of the research and clinical workflow. This procedure was carried out for all of the tasks. Two additional authors reviewed the process (RP and HB).

Development of Workflow Modification Plan

Workflow modifications were focused on the elimination of the most frequent dissonances and then the remaining dissonances by decreasing frequency. Strategies for improving the overall clinical workflow while allowing more efficient data collection was discussed with the clinic coordinator. As a result, we created a new workflow addressing the dissonances that were identified based on these analyses and group discussions with the clinic coordinator.

Implementation of Changes

As a case study, we implemented the proposed workflow based on the dissonance analysis. After the clinic director approved the proposed workflow it was presented by the clinical coordinator to professionals from the outpatient clinic during a staff meeting. At that time the clinic staff and researchers were trained to use the new tools so that they would be able to perform the new tasks. One week later, the clinic implemented the new workflow. All workflow modifications were implemented simultaneously in the first week of June 2011.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. UML representation of the identified for the clinical workflow.

https://doi.org/10.1371/journal.pone.0075167.g002

We considered the following measures as indicators of efficiency in research workflow: (1) Number of patients approached for enrollment, (2) number of patients enrolled in the study. This information was derived from a spreadsheet that was used to register information about the patients who were screened, contained patient names, identification numbers, screening assessments dates, screening assessment results, and information on whether the patients had been included in the research protocol. We obtained data corresponding to the period between September 2010 and November 2011. December 2010, January and July 2011 were not included since the outpatient clinic was closed. We also recorded the number of dissonances before and after the workflow modification.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. UML representation of the identified for the research workflow.

https://doi.org/10.1371/journal.pone.0075167.g003

Empirical Setting

We conducted our observations at the Autistic Spectrum Disorder outpatient clinic at the Institute of Psychiatry at the Hospital das Clínicas, an urban teaching hospital at the University of Sao Paulo Medical School. Its mission is to integrate health care, education, and research in order to better serve the community. The clinic operates one morning a week with an average 20 patients per week. We selected this clinic for our case study as it initially seemed like a location where clinical and research workflows were misaligned. One important research objective in this clinic was the development of diagnosis assessment protocols and assessment tools. However, the research process was hindered by the limited number of patients that were screened (evaluated with autism screening instruments) and enrolled into research protocols.

Tasks Category

The observer did not use a predefined list of tasks since the main objectives of the observation were to describe and document the overall clinic workflow. Tasks of interest included all those necessary to complete the clinical workflow (the outpatient medical visit for one patient) or the research workflow (research data collection for one patient).

Observer

One of the authors (LC) conducted the observations and interviews. This researcher did not participate in specific training prior to data collection, but had a background in health informatics and prior experience in workflow analysis for information systems implementation.

Subject

The clinic had a team composed of four residents, two medical assistants, and a nurse involved solely in patient care. Additionally there was a multidisciplinary team involved in research and patient care. This team was comprised of one occupational therapist, three psychologists, one neuropsychologist, and two speech therapists.

A physician and a representative of each multidisciplinary team member category selected by convenience were accompanied during one morning (from 8 am to 11 am) so the observer could understand the routine pattern of their tasks. The remainder of the observation period was used to observe the general approach and interaction of researchers towards the patients, and the interaction among the clinic staff.

Data Recording and Analysis

Observational data was recorded through notes collected by the author (LC) during fieldwork and a qualitative approach was used to analyze this data.

Clinical and Research UML Workflow Modeling Results

Two distinct activity diagram workflows present the main tasks identified during the observation period. The Extensible Markup Language (XML) Metadata Interchange (XMI) models from all of the UML models created in this research are available at figshare.com [25]. Models for the clinical and research workflow are presented in Figures 2 and 3.

Observation and Dissonance Classification Results

We analyzed the UML Activity Diagrams to identify existing wastes and opportunities to integrate process steps more efficiently, and wastes were indicated as notes in the UML Activity Diagrams (Figures 2 and 3). Waste, its description and the tasks associated with the waste are presented in the first three columns of Table 1. Additionally, details of the tasks are presented in presented in the 4th through 10th columns and workflow dissonances designations are presented in the last column.

We observed three tasks related to the clinical workflow that when not accomplished could delay the research workflow and were associated with wastes (Figure 2).

The task, “screening tools application” was related to the waste type waiting since researchers would have to wait for physicians to screen patients before they could perform their evaluations. The review of the detail of these tasks indicated that the number of actors (physicians) limited the clinic’s ability to successfully conduct screening evaluations. Also, residents were assigned to the clinic for only a semester for each period, and therefore new physicians needed to be trained to use the screening tools. This inefficiency was then designated as “actor dissonance,” since the specific cause of the disruption of research workflow was directly linked to the actor of this activity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Integrated clinical and research workflow.

https://doi.org/10.1371/journal.pone.0075167.g004

The use of paper-based forms for performing screening is associated with the waste type defect since it demands rework for the researchers as it is necessary to reenter the data in an electronic format. This strategy hindered real-time sharing of the results with researchers so it was classified as “artifact dissonance.”

The task, “referring patients to research protocols” is related to the waste type waiting since researchers would have to wait for physicians to refer patients to the research protocol before they could perform their evaluations. Some physicians did not know the inclusion and exclusion criteria, thus hindering the referral of potential subjects for research protocols. This was classified as “information dissonance”. Even when patients were screened and met research inclusion criteria communication problems existed between the physicians and researchers and undermined the referral of patient and this was designated as “communication dissonance.”

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. The number of medical visits pre- and post intervention at the Autism Outpatient Clinic.

The number of medical visits per month and corresponding number of patients not yet screened are presented before and after intervention. Data refer to the period between October 2010 and November 2011 at the Autism outpatient clinic.

https://doi.org/10.1371/journal.pone.0075167.g005

The problems with the referral process motivated researchers to look for subjects by reviewing patient charts. The task, “register patient information in the medical record” was associated with the waste type overprocessing since researchers had to manually review the patient chart.

The analysis of the UML Activity Diagrams indicated the existence of waste from some tasks of the research workflow (Figure 3). We observed two types of wastes associated with the tasks of performing research evaluation: defect and transportation. Defects were associated with the use of inadequate tools for scheduling and registering research evaluations. The use of paper-based forms to collect research data led to rework, since it was required to input collect data on a spreadsheet, and also implicated fragmented information. These were designated as “artifact dissonance” and “information dissonance.” Research evaluations were also associated with transportation waste since research evaluations had to be carried out at a different time and locations than where clinical tasks took place. These were designated as “time dissonance” and “space dissonance”.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Pre and Post intervention screening and enrollment at the Autism Outpatient Clinic.

Percentage of patients approached and enrolled in relation to total number of patients available for screening for the month. Data refer to the period between October 2010 and November 2011 at the Autism outpatient clinic.

https://doi.org/10.1371/journal.pone.0075167.g006

Observed factors regarding tasks that disrupted workflow and corresponding classification are presented in Table 2. This list was then summarized to eliminate duplicate terms and improve the overall ease of use of the classification and presented under Table 3.

Case Study Proposing New Workflow

Table 4 presents the frequency of the workflow dissonance observed in the initial workflow analysis. A total of 22 episodes of dissonance were observed across 6 dissonance categories. The most frequent dissonance belonged to the information and artifact dissonance categories, followed by the time and space dissonances. The least frequent dissonances were the actor and communication dissonances. We proposed five modifications to address the identified workflow dissonance points by aiming to eliminate the most frequent dissonances first:

1. 1. We replaced the paper-based data collection system with an electronic data capture [26]. This addressed the “artifact dissonance” and “information dissonance” by eliminating rework for researchers and allowing information exchange among the clinic’s multidisciplinary team.
2. 2. We granted the research team consultation access to the patient scheduling information in the hospital management information system. This information enabled scheduling of research evaluations more efficiently and addressed one source of “information dissonance.”
3. 3. We reorganized the flow of activities so the time patients spent in the waiting room could be used for the application of screening tests. Patients who had a screening score compatible with the inclusion criteria for the research protocol would be informed about the study and asked to participate. As a result researchers could quickly schedule visits and decrease the number of patients who declined to enroll in the study since patients are more likely to accept when they are approached in person rather than over the phone. We also proposed scheduling research assessments on the same day they had their medical appointment. This measure was also able to address “time dissonance.”
4. 4. All the research assessments could not be developed at the clinic at the same period as medical consultations due to limitation of physical space. We divided research tasks into two modules. The first was performed while patients were waiting before or after the medical visit. The second module, with time consuming evaluations was performed in the afternoon. This measure partially addressed the “time and space dissonance.”
5. 5. We transferred the responsibility of performing the screening evaluations to two researchers and this addressed the “actor dissonance.” This also addressed the “communication dissonance” since physicians no longer had to refer patients for participation in the research protocol, and the “information dissonance” since researchers no longer had to review patient charts to identify potentially eligible subjects.

We designed a new workflow combining research and clinical tasks by implementing all the above mentioned modifications (Figure 4). We were not able to address the source of dissonance related to the difficulty of retrieving information from the paper-based medical record. However, there is an ongoing project of implementing an electronic medical record that will be able to address it.

Pre and Post-Intervention Patient Enrollment

The number of medical visits per month and corresponding number of patients not yet screened are presented in Figure 5. The number of appointments per month varied each month and was influenced by the occurrence of holidays during the period. However, it did not change after the implementation of the new workflow (indicated by a dashed line). During the months of the study patients that were previously screened returned for follow-up visits, thus contributing to a decrease in the number of patients that were still available for screening.

Figure 6 presents the percentage of the patients screened and enrolled in relation to the total number of patients available for screening during the months of the study. A dashed line indicates the moment of the workflow modification. We observed a marked increase in the number of patients screened and enrolled after the implementation of the new workflow.

At the end of the observation period there was a small decrease in the percentage of patient screened. However, the percentage of patient enrollment remained steadily increasing (Figure 6). Dataset and scripts that were used to generate plots are available at Figshare [25].

Table 4 presents the frequency of the workflow dissonance remaining after the new workflow implementation. The number of dissonances decreased from 22 to 4. The “actor” and “communication dissonances” were addressed. One episode of information and one of artifact dissonance related to the use of a paper-based patient medical record remained unresolved. One time and space dissonance was not addressed, as the more time-consuming evaluations had to be carried out after the medical appointment.