Three modes.

CACTI employs three modes that satisfy different functions: parsing continuous behavioral data from a WAV audio file into codeable utterances, sequentially coding previously parsed utterances, and assigning Likert-type global ratings. (See Figures S1, S2, and S3 for sample illustrations.) Those modes create output files with file extensions of.parse,.casaa, and.global, respectively; within these files, utterances are sequentially enumerated automatically, and codes are uniquely named and numbered by the user within the XML file (see operation manual, described below). All CACTI output files can be opened as tab-delimited text or spreadsheet files and then imported into statistical software for analysis.

Downloadable software package.

The CACTI software package is a free 2 MB ZIP file (downloadable from http://code.google.com/p/ctcsu-player/downloads/list) that includes six components: the CACTI.jar archive file, UserConfiguration.xml file, images folder, operation manual [33], user-configuration guide [34], and the GetCASAA script [35] to convert CACTI output files to Sequential Data Interchange Standard (SDIS) [36] format. CACTI output files should be saved locally during parsing and coding, but sizeable WAV audio files may be stored on the local machine or on a shared local area network, as needed. A test dataset also is available from that software archive.

Customizable-coding-system feature.

Although CACTI was designed originally for use with the MISC 2.5, it was intentionally built to be user-modifiable to accommodate other coding systems. Moreover, CACTI is free and open-source under the GNU General Public License, which means that anyone may use and modify it. These features facilitate CACTI’s utility for psychotherapy research–or audio-based content analysis more generally-rather than only within a motivational interviewing framework.

By default, the UserConfiguration.xml file is arranged for MISC 2.5 coding, but most elements of the three modes–including buttons, sub-menus, and keyboard shortcuts–are user-modifiable using XML. CACTI can be configured to accommodate any number of behavior codes or sliders, with limitations imposed only by the user’s screen resolution (e.g., 16 global ratings for 1280×800 pixels) and desired number of characters displayed in button names. For complex coding systems, behavior-code buttons may be expanded to become menus of sub-codes, which allows for the streamlined presentation of code–sub-code combinations. Detailed information about configuring the XML file is included within the software package in the operation manual and user configuration guide.

Automatic back up.

To prevent data loss, each mode produces a real-time backup file that saves upon the placement of a new parse, behavior code, or global rating. This feature ensures that data are not lost if a CACTI session is interrupted.

Visual time line.

The parsing and sequential-coding modes feature a visual timeline to help coders anticipate the ends and beginnings of utterances and ensure that codes are placed as intended. The time line appears as a series of colored rectangular boxes that move as a coder progresses through the audio file; start and end times, utterance enumeration, and assigned code names and numbers are displayed for the previous, current, and next utterances.

Playback and function buttons.

In the parsing and sequential-coding modes, two panels of buttons control the playback and parsing or coding functions. Playback controls allow users to pause or resume audio, rewind, remove parses or codes, or extend the previous utterance. Function buttons allow users to designate beginnings and ends of parses or assign codes to pre-parsed utterances. These controls may be triggered by mouse or through keyboard shortcuts, which are intended to decrease coder reaction time, accommodate different user preferences, and reduce possible strain from repetitive motions.

Sliders.

The global-coding mode includes multiple sliders that submit Likert-type global ratings of clients and clinicians, along with a text box for notes. The default (MISC 2.5) format uses seven five-point scales, but any number of scales and values may be specified.

Precise and comprehensive timing information.

In CACTI, all parse and sequential-coding files are denoted to the hour, minute, second, and bit (see Example). Thus, utterances can be assigned as precisely as coder reaction time will allow, which can be beneficial in capturing events of brief duration or finely parsing audio information.

Example.

Note that the first column indicates the utterance number, the second and third the starting and ending times, the fourth and fifth the starting and ending byte numbers, and the sixth and seventh the numeric values and names of the codes assigned.

Ease of data analysis.

Using the GetCASAA.m MATLAB script, users can summarize a set of.casaa files into a tab-delimited text file containing counts for each coding variable by session, an additional tab-delimited text file containing the cumulative duration of each coding variable by session (e.g., “talk time” for each speaker, or total time spent on task, rounded to the nearest second), as well as a Sequential Data Interchange Standard [36] file for the GSEQ application [37], organized by session. Count files can be imported into standard statistical packages, including R, SAS, or SPSS, for subsequent analysis. Lag-sequential analysis can be performed in GSEQ, estimating conditional probabilities and odds ratios for each possible transition, either pooled across a sample of sessions, or computed individually and exported for subsequent analysis.

Advantages over non-computerized coding methods.

First, CACTI is more efficient than coding from transcripts, because it eliminates the lengthy and costly steps of transcribing sessions and manually entering data. Producing a clean transcript has taken our lab approximately 480–600 minutes per hour of audio, and other researchers have reported transcription times of up to 900 minutes per hour of audio [15], exclusive of parsing and coding time. In our experience, CACTI has required only 90 minutes per hour to parse and 75 minutes per hour to code; we estimate that the parent study, Project ELICIT, has saved at least 7500 work hours and upward of $60,000 by using CACTI instead of transcript-based methods. Second, the “talk time” for behaviors of theoretical interest can be calculated easily by adding the lengths of utterances, whereas previous methods required the manual use of a stopwatch, which was more difficult and error-prone. Other time-based calculations may be performed easily as well, by importing the output files into spreadsheets. Third, CACTI supports coding “on the fly” in real time, which, relative to using transcripts alongside audio recordings, might be more naturalistic, capture a greater scope of information, and avoid dividing coder attention. Fourth, the representation of interactions through transcripts has been criticized as potentially inaccurate and inherently “theory laden” [38], which CACTI avoids by referring coders only to the original audio source.

Advantages over other software.

First, CACTI allows for the parsing, sequential coding, and global coding of audio files; these three features are not available together in any other software package that we could identify. CACTI’s ability to support the parsing of data prior to coding is a particular strength. As described in previous studies (e.g., [6], [10]), pre-parsing data can improve both interrater reliability and validity, because it ensures that coders are coding the same portion of the interaction. Second, CACTI is available to all users free of charge and intended for cross-platform use. These features support sharing and innovation, and remove some barriers to conducting research. Third, the software was designed to be easily modifiable by the user to match almost any audio-based coding system. Most aspects of the interface (e.g., rows, columns, titles, and values) can be changed in a simple XML file, and codes can be arranged hierarchically if needed. Fourth, the visual time line in the parsing and sequential-coding modes allows for greater precision than audio markers alone, which potentially improves interrater reliability.

Interrater reliability.

Interrater reliability measures the degree of agreement among raters, and often is computed using intraclass correlations (ICCs) [39] in coding research. Per Cicchetti [40], ICCs above 0.60 represent “good” to “excellent” reliability, and have been considered acceptable in coding studies similar to Project ELICIT (e.g., [41]).

Project PREMIR [6], which performed sequential coding with concurrent transcripts and audio recordings, reported that ICCs for 10 of 14 summary ratings were above 0.75, which is within the “excellent” range described by Cicchetti [40]. The present study (ELICIT [29]), which used the CACTI program and a similar coding system, also achieved estimates of reliability in the “excellent” range on 10 of 14 of these same summary ratings. Both studies computed reliability using the ICC [3], [1] model [39]. To compare inter-rater reliability estimates between these two studies, ICCs were transformed to z scores using Fisher’s transformation and the effect size q [42] was computed. Results ranged from q = −0.40 (favoring PREMIR) to q = 0.69 (favoring ELICIT), with a mean of 0.11 (SD = .33). (See Table S1 for an overview.) These results suggest that coding with CACTI can produce comparable interrater reliabilities to those acquired via previous methods. We also have reported estimates of reliability for ELICIT using Krippendorff’s alpha [43], a measure of reliability than has been suggested for standard use in coding studies [44].

User acceptability and training needs.

Although reliability is one measure of success, users’ subjective experiences of the software also are important in determining their satisfaction and their willingness to persist through the grueling and detail-intensive task of coding for a large, multi-year research project. Within our own lab, coders have reported that CACTI is useful, convenient, dependable, efficient, and easy to operate. In addition, we have found that training requirements are minimal, and offer substantial improvement over previous methods: Training four proficient MISC 2.5 coders to use CACTI reliably took approximately three hours per person in a combination of group and individual settings.

In summary, we have found that CACTI shows several advantages over both previous (i.e., transcript-based) sequential-coding methods and existing software. In addition, trained coders appear able to code standard psychotherapy-length verbal interactions (i.e., 45–60 minutes) and achieve similar interrater reliabilities as previous methods, while reporting less subjective difficulty than using transcripts, and substantially reducing expenditures of time and money. Training requirements were minimal, and users’ anecdotal reports indicated satisfaction with the product.