Figures
Abstract
Purpose
To assess the impact of mental health visits (MHV) on the cost of care for Veterans with diabetes and comorbid mental health conditions.
Methods
A national cohort of 120,852 Veterans with diabetes and at least one mental health diagnosis (i.e., substance abuse, depression or psychoses) in 2002 was followed through 2006. Outcomes were pharmacy, inpatient and outpatient costs in 2012 dollars.
Results
Least-square covariate adjusted estimates from the joint model of total VA costs of the number of MHV using December 31, 2012 value dollars indicate that relative to those with fewer MHV, those with 3+ MHV had the lowest mean inpatient cost ($21,406), but the highest mean outpatient and pharmacy cost ($9,727 and $2,015, respectively). If all Veterans who received zero MHV actually received 3+ MHV, we estimate through simulated scenarios that between $32,272,329 and $181,460,247 in inpatient costs would be saved. However, these savings would be offset by additional expenditures of between $1,166,017,547 and $1,166,224,787 in outpatient costs and between $151,604,683 and $161,439,632 in pharmacy costs.
Citation: Egede LE, Gebregziabher M, Zhao Y, Dismuke CE, Walker RJ, Hunt KJ, et al. (2014) Impact of Mental Health Visits on Healthcare Cost in Patients with Diabetes and Comorbid Mental Health Disorders. PLoS ONE 9(8): e103804. https://doi.org/10.1371/journal.pone.0103804
Editor: Robert Stewart, Institute of Psychiatry, United Kingdom
Received: March 27, 2014; Accepted: June 25, 2014; Published: August 1, 2014
Copyright: © 2014 Egede et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that, for approved reasons, some access restrictions apply to the data underlying the findings. Department of Veterans Affairs policy and U.S. federal law prohibit disclosure of Veteran information outside the U.S. Veterans Health Administration. Investigators interested in obtaining data to replicate our analysis and who have appropriate affiliation with the U.S. Veterans Health Administration are welcome to contact Mr. Charlie Stroup, the Deputy Director for Health Information Access at the VA National Data Systems office. Mr. Stroup's telephone is 202-465-1581, and his email address is Charlie.Stroup@va.gov.
Funding: This study was supported by grant #IIR-06-219 funded by the VHA Health Services Research and Development (HSR&D) program. The funding agency did not participate in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Individuals with diabetes are more likely than those without diabetes to have mental health disorders. For example, individuals with diabetes have 60% higher odds of major depressive disorder and 123% higher odds of generalized anxiety disorder [1]. Individuals with diabetes are also more likely to have schizophrenia and bipolar disorder than those without diabetes [2], [3]. Our group has previously examined multiple aspects of the relationship between diabetes and depression [3], [4]. Comorbid depression has been associated with poor glycemic control, decreased medication and diet adherence, increased risk of complications, increased health care costs and decreased health-related quality of life [4]. This may be due to the effect of depression on behavioral pathways or physiological pathways [5]. Less is known about the relationship between diabetes and other mental health disorders, but existing reports document similar impacts on outcomes including glycemic control, diabetic complications, quality of life, disability, and medication adherence [3], [6], [7].
Comorbid diabetes and mental health disorders are also associated with increased healthcare utilization and costs. Overall, the cost of diabetes in the U.S. was estimated at $245 billion in 2012, with the largest component of expenditures attributed to inpatient costs [8]. Healthcare expenditures for individuals with diabetes are, on average, 2.3 times higher than for individuals without diabetes [8]. When considering patients with comorbid depression, costs are significantly higher even after adjustment for the presence of variables such as age, sex, race, insurance, and other chronic medical illnesses [3]–[5]. A recent review of healthcare costs for patients with diabetes and comorbid mental health disorders identified 31 studies, 27 of which focused on depression [9]. Overall, comorbid mental health disorders were associated with increased healthcare costs, however the authors suggested the need for further inquiry into allocation of healthcare resources and costs in comorbid conditions beyond depression [9].
We hypothesized that increased frequency of mental health visits would result in favorable net effects on healthcare costs over time. We also hypothesized that while outpatient and pharmacy costs might increase with mental health visits; there would be a corresponding decrease in inpatient costs associated with overall savings. Therefore, the purpose of this study was to determine whether mental health visits, as defined by visit with a VA clinic stop for a mental health or substance abuse service, have an impact on the cost of care for Veterans with diabetes and comorbid mental health conditions.
Methods
Data Source and Sample
A national cohort of U.S. Veterans with type 2 diabetes and at least one mental health diagnosis, was created by linking multiple patient and administrative files from the Veterans Health Administration (VHA) National Patient Care and Pharmacy Benefits Management (PBM) databases. Veterans were first selected into a diabetes cohort who had: 1) type 2 diabetes defined by two or more International Classification of Diseases, Ninth Revision (ICD-9) codes for diabetes (250, 357.2, 362.0, and 366.41) in the previous 24 months (2000 and 2001) and 2) ICD-9 codes for type 2 diabetes from inpatient stays and/or outpatient visits on separate days (excluding codes from lab tests and other non-clinician visits) in 2002, and 3) prescriptions for insulin or oral hypoglycemic agents in 2002 based on a previously validated algorithm [10], [11]. Of the 740,195 Veterans included in the diabetes cohort, 120,852 had at least one ICD-9 diagnosis of substance abuse (305.00, 303.9, 305.1–305.9, 304.0–304.9), depression (262.2, 296.3) or psychosis (295.1–295.3, 295.6, 295.4, 295.4) and comprise the study population for the current analysis. Subjects were followed from 2002 until death, loss to follow-up, or through December 2006. Linking the datasets with scrambled social security numbers, we calculated annual patient-specific VA health care costs through VA Decision Support System (DSS) data, which extracts costs from the VA payroll and general ledger. Costs were then classified as inpatient, outpatient (including emergency department, primary and non mental health specialty care), and pharmacy. The study was approved by Medical University of South Carolina Institutional Review Board and Ralph H. Johnson Veteran's Affairs Medical Center Research and Development committee. A HIPAA waiver of consent was obtained in order to access medical records due to the need to merge longitudinal data with identifiable information. Patient records were anonymized and de-identified prior to analysis. Department of Veterans Affairs policy and U.S. federal law prohibit disclosure of Veteran information outside the U.S. Veterans Health Administration. However, investigators interested in obtaining data to replicate our analysis can learn more at the VA Information Resource Center (VIReC) at (http://www.virec.research.va.gov/Index.asp).
Study Variables
Outcome variables.
The primary outcome variables were three cost types (pharmacy, inpatient, and outpatient including all emergency department, primary care and non-mental health specialty care) measured in 2006 US dollars from the perspective of the federal payer. We adjusted final costs in to reflect 2012 dollar values using http://www.bls.gov/data/inflation_calculator.htm. DSS cost data was applied to VA encounter codes (diagnosis or procedure). Prescription drug costs were identified from the PBM system and summed from the price per dispensed unit. Costs for each study year were represented in person-years to account for censoring.
Covariates.
The primary covariate was the number of mental health visits (MHV), as defined by VA clinic stop code for a visit in a mental health or substance abuse service, categorized as 0, 1, 2 and 3+. Clinic stop codes are used by VHA to track clinic workload and productivity, and they represent discrete healthcare encounters. Other covariates included in the full model were age, gender, marital status, service connected disability level, race/ethnicity, residence, region, and comorbidities. Marital status was defined as single or married. Service connected disability level was categorized as <50 or ≥50%. Veterans with high levels of service-connected disability (i.e., > 50%) are exempted from co-payments. Race/ethnicity was classified as non-Hispanic white (NHW), non-Hispanic black (NHB), Hispanic, and other/unknown/missing. Location of residence was defined as urban and rural/highly rural [12], and hospital region was defined by the five geographic regions of the country based on VHA Veteran's Integrated Service Networks (VISNs): Northeast (VISNs 1, 2, & 3), Mid-Atlantic (VISNs 4, 5, 6, 9, & 10), South (VISNs 7, 8, 16, & 17), Midwest (VISNs 11, 12, 15, 19, & 23), and West (VISNs 18, 20, 21, & 22). Comorbidity variables based on ICD-9 codes included, anemia, cancer, cerebrovascular disease, congestive heart failure, cardiovascular disease, hypertension, hypothyroidism, liver disease, lung disease, fluid and electrolyte disorders, obesity, psychoses, and other (AIDS, rheumatoid arthritis, renal failure, peptic ulcer disease and bleeding, weight loss).
Statistical Analyses
Descriptive measures including mean and median costs were computed for each cost type for each MHV group. Preliminary analysis included plotting the unadjusted mean costs in each cost type (inpatient, outpatient, pharmacy) over time by MHV, with time on the x-axis and unadjusted mean cost on the y-axis. The plots helped to examine trends over time in each source of cost by MHV before adjusting for any other covariates. To model the relationship between the three cost categories and covariates in a manner that accounted for differences in variance among the three outcomes, a joint model based on a multivariate generalized linear mixed model (mGLMM) approach with shared random intercept and slope was used [13].
Since the response variable is a vector of three correlated cost outcomes, mGLMM with a random intercept and slope was used to account for the correlation among the cost outcomes and estimate the joint effect of MHV on the three outcomes. This is implemented in SAS Proc GLIMMIX (SAS Institute Inc, Cary, NC) using SAS code provided in [13]. To account for the skewness in the observed cost data, a log-normal distribution with an identity link was used. Hence the exponent of the parameter estimates can be interpreted as the percent change in each type of cost as a function of unit change in the covariates. Comparison with analysis results based on fitting separate models for each outcome was also performed (see online Table 1). In the joint models, the random intercept shared by the three cost outcomes captures the association in the natural heterogeneity among the individual subjects' inpatient, outpatient and pharmacy costs, while the random slope captures the correlation in the trajectory over time of cost outcomes. Model goodness of fit was assessed using pseudo-AIC-type statistics (based on pseudo-likelihood as implemented in Proc GLIMMIX) and using residual plots.
Potential savings were estimated by examining the adjusted mean cost differential between those with no mental health visits and those with one or more MHV in every cost category in every year and multiplying by the number of Veterans in that year. Finally, projected annual potential total and incremental savings (losses) to the VA at each MHV value was estimated using the 5 year least-square mean estimates (means that are adjusted for covariates as computed by the LSMEANS statement of PROC GLIMMIX) from the joint model. Costs and potential savings were calculated, from the VHA perspective, of providing 3 MHV to individuals with mental health co-morbidities receiving zero MHV were made based on number of Veterans. For the number of Veterans in a MHV category using any service and having any type of cost the high value was used. For those Veterans who had non-negative cost only the low number was used. The U.S. Department of Labor Statistics estimate of the medical component of the Consumer Price Index (CPI) was used to convert all 2002–2005 costs to 2006 dollars prior to model estimation. The CPI was also used to convert potential savings (losses) to the VA from increased mental health visits from 2006 dollars to December 31, 2012 dollar values. All statistical analyses were performed using SAS 9.2 (SAS Institute, Carey, NC).
Results
Population Characteristics and Mental Health Visit (MHV) Status
The final study sample consisted of 120,852 Veterans with a diagnosis of type 2 diabetes and either depression, substance abuse, or psychosis during 2002 and followed until death, loss to follow-up or through 2006 (Table 1). The proportion of the cohort with diagnoses of depression, psychoses and substance abuse were 73.3%%, 25.6%% and 22.6%, respectively. Moreover, 81.7% had only one mental health co-morbidity, 15.2% had two mental health co-morbidities and 3.1% had all three mental health co-morbidities. Of the 120,785 Veterans with diagnosed mental health co-morbidity, 26.3% did not have a mental health visit from 2002 through 2006. Among those who had at least one MHV, the mean number of mental health visits declined from 2002 (10.8 visits) to 2003 (7.4 visits) and then reached its peak in 2004 (11.9 visits). Those who had at least one MHV were younger, more likely to be a minority (i.e., NHB, Hispanic or other), more likely to have greater than 50% service connected disability, less likely to be married and have a rural residence, and more likely to have multiple mental health co-morbidities than those without MHV.
Longitudinal Costs Associations with MHV
Table 2 highlights findings of significant (P<0.001) cost associations with mental health visit (MHV) after adjustment for demographics and the type of medical and mental health co-morbidities and accounting for the correlation of cost categories over time. Relative to the zero MHV group, one MHV was associated with a 1.4% higher inpatient cost, 30.9% higher outpatient cost, and 11.9% higher pharmacy cost between 2002 and 2006. Having two MHV was associated with 5.4% higher inpatient cost, 70.3% higher outpatient cost and 30.8% higher pharmacy cost. Having three mental health co-morbidities was associated with a 4% lower inpatient, 164% higher outpatient, and 71.4% higher pharmacy cost.
Estimates of Mean and Total VA Costs for Number of MHV
Least-square estimates from the joint model of total VA costs of the number of MHV based on December 31, 2012 value dollars are shown in Table 3. Over the 5 year period, mean inpatient cost was highest for those with two MHV ($23,537) and lowest for those with three or more MHV ($21,406). Mean outpatient cost was highest for those with three or more MHV ($9,727) and lowest for those with zero MHV ($3,631). Mean pharmacy cost was highest for those with three or more MHV ($2,015) and lowest for those with zero MHV ($1,171). Least-square estimates from the joint model of total VA costs of the number of MHV based on December 31, 2012 value dollars by year for 2002 through 2005 are found online (Table S1) which consists of three panels for inpatient (panel a), outpatient (panel b) and pharmacy (panel c) costs.
Potential savings to the VA under different possible scenarios of MHV are computed and the results for are shown in Table 4 where total costs were calculated by multiplying the mean least-square estimate from the joint model by the number of Veterans in the low and high categories. The mean least-square estimate was multiplied by the consumer price index multiplier to approximate December 2012 dollar values. We considered two scenarios based on number of Veterans. For the number of Veterans in a MHV category using any service and having any type of estimated cost the high value was used. For those Veterans who had estimated cost only, the low number was used. If all Veterans who received zero MHV, actually received three or more MHV, we estimated that between $32,272,329 and $181,460,247 in inpatient costs would be saved. However, these savings would be offset by additional expenditures of between $1,166,017,547 and $1,166,224,787 in outpatient costs and between $151,604,683 and $161,439,632 in pharmacy costs.
Discussion
Our research team has previously examined the effects of medication non-adherence on costs and the potential cost savings from improving adherence [13]. We found that improving diabetic medication adherence would result in annual estimated cost savings ranging from ∼$661 million to ∼$1.16 billion. Using similar methods, we hypothesized that increased frequency of mental health visits would result in favorable net effects on healthcare costs over time. As anticipated, inpatient costs were lowest in Veterans with diabetes with mental health comorbidities who had three or more MHV, even after adjusting for demographic factors, psychiatric comorbidities, and medical comorbidities. Outpatient and pharmacy costs were highest in this group as well. However, contrary to our hypothesis, the dollars saved on inpatient costs were not sufficient to offset the increased outpatient and pharmacy cost in Veterans with three or more MHV relative to those with zero, one, or two MHV. Potential savings of providing 3 MHV to Veterans with mental health comorbidities currently receiving zero MHV indicate the VA would save between $32,373,329 and $181,460,247 on inpatient cost. However, this savings would be offset by additional expenditures of between $1,166,017,547 and $1,166,224,787 in outpatient costs and between $151,604,683 and $161,439,632 in pharmacy costs.
On August 31, 2012, President Barack Obama signed an executive order designed to improve access to mental health services for Veterans, service members, and military families [14]. This order contained provisions to improve suicide prevention programs, enhance partnerships between the Department of Veterans Affairs and community providers, improve mental health research and development, and expand VA mental health services staffing. Specifically, the VA is currently in the process of hiring over 1,600 additional mental health providers and 800 new peer counselors. This analysis is relevant as one approach to answer questions regarding how to accomplish these goals, especially as it pertains to high-risk patients with comorbid medical illnesses like diabetes. In particular, how should VA mental health providers allocate their time and resources given a contemporary environment of belt-tightening and constricting federal budgets?
Multiple studies have documented that, among patients with comorbid diabetes and psychiatric disorders, poor disease control in one domain can adversely impact disease control and outcomes in the other [15]. Several studies also suggest that increased mental health participation can lead to improvements in diabetes control. For example, Dixon et al. analyzed diabetes control in patients with comorbid schizophrenia and severe mood disorders [16]. They found lower hemoglobin A1C values among patients actively enrolled in mental health treatment compared to individuals with diabetes without mental health disorders and concluded that there may be previously unrecognized benefits for diabetes management among persons with severe mental illnesses who receive regular mental health care. Similarly, Kreyenbuhl et al. found better adherence in patients with schizophrenia and diabetes than patients with only diabetes, and they suggested enhanced awareness to overall medical well being as an outcome of mental health treatment [17]. They also found a greater number of visits increased the odds of receiving ACE-inhibitors to reduce cardiovascular risk [18]. While these studies indicate that high-quality mental health care may contribute to improved diabetes control, our analysis raises concerns that increasing clinic visits alone may not be the best approach for reducing utilization and cost.
Because inpatient cost reduction tends to drive cost savings, it is likely that a more nuanced approach to the provision of mental health services is required, one that tailors service provision to individual patient needs, severity of illness, and local resources. For example, patients with diabetes and mild depression may not benefit from multiple mental health visits as much as patients with more severe depression, substance abuse, or schizophrenia. There may also be other care models besides mental health clinic visits that better suit the needs of some Veterans. The VHA has led innovation in developing models for co-locating primary care and mental health services [19]–[22]. The VHA has also developed leading telemental health programs, which increase access to care for mental health services for many Veterans, especially in rural areas [21], [23]–[26]. Finally, Veterans with the highest severity of mental illness will likely benefit from intensive case management strategies, the cost outlays for which are not likely captured by current workload models [27].
This study should be interpreted in light of certain limitations. First, only 3.6% of our sample was female which may limit the generalizability of our findings for women Veterans. However, our cohort did contain over 4,000 women. Second, our database did not contain additional information regarding potential cost predictors such as self-care behaviors, disease knowledge, and health beliefs. Unfortunately, collecting such information was not feasible for such a large cohort. Our study lacked cost data from other payers such as Medicare. Thus, our cost estimates may be low if subjects tended to use large amounts of non-VHA care. However, our inclusion criteria tended to select patients who utilize VHA for most of their healthcare. In addition, since our focus was on multimorbidity, we chose to analyze the total number of mental health visits as a count rather than examining number of visits by individual mental health comorbidity. Future estimates of cost savings should explore the impact of discrete mental health disorders on cost and outcomes in this population. Finally, future studies are needed that incorporate adjustment for severity of illness within mental health disease categories and that stratify costs by clinical outcomes. In summary, patients with diabetes and comorbid mental health disorders represent a high-risk patient population that requires extensive resource allocation to improve health outcomes. We expect that these results will be of interest to healthcare providers and policy makers who strive to achieve optimal outcomes for patients while managing costs in a national healthcare system. First, providers must develop and select healthcare treatments and programs among the menu of available alternatives. Our results suggest that a simplistic approach that simply increases provider capacity and visit numbers may not lead to improvement in inpatient healthcare utilization substantial enough to offset outpatient and pharmacy investments. Instead, other evidence-based strategies mentioned above e.g., co-located primary care, telemental health, intensive case management) may be preferred. In turn, healthcare executives can use modeling techniques as described here in order to better plan for the financial impact of programs selected for implementation at the local and regional level. Finally, national healthcare payors such as Department of Veterans Affairs or Medicare can utilize this type of analysis in long-range budget planning.
Supporting Information
Table S1.
Panel A. Inpatient Total Cost by number of mental health visits (MHV) by year from Joint Model Least-square Estimates of Inpatient, Outpatient and Pharmacy Cost In December 31, 2012 Value Panel B. Outpatient Total Cost by number of mental health visits (MHV) by year from Joint Model Least-square Estimates of Inpatient, Outpatient and Pharmacy Cost In December 31, 2012 Value. Panel C. Pharmacy Total Cost by number of mental health visits (MHV) by year from Joint Model Least-square Estimates of Inpatient, Outpatient and Pharmacy Cost In December 31, 2012 Value.
https://doi.org/10.1371/journal.pone.0103804.s001
(DOCX)
Acknowledgments
1. The manuscript represents the views of the authors and not those of the VA or HSR&D.
2. Drs. Kelly J Hunt, Mulugeta Gebregziabher and Leonard E Egede are the guarantor's of this work and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Author Contributions
Conceived and designed the experiments: LEE MG CED. Performed the experiments: LEE. Analyzed the data: LEE MG CED KJH YZ. Contributed reagents/materials/analysis tools: LEE. Contributed to the writing of the manuscript: LEE MG YZ CED RJW KJH RNA.
References
- 1. Fisher L, Skaff MM, Mullan JT, Arean P, Glasgow R, et al. (2008) A longitudinal study of affective and anxiety disorders, depressive affect and diabetes distress in adults with Type 2 diabetes. Diabet Med 25(9): 1096–101
- 2. Dickerson F, Brown CH, Fang L, Goldberg RW, Kreyenbuhl J, et al. (2008) Quality of life in individuals with serious mental illness and type 2 diabetes. Psychosomatics 49(2): 109–14
- 3. Egede LE, Ellis C (2010) Diabetes and depression: global perspectives. Diabetes Res Clin Pract 87(3): 302–12.
- 4. Egede LE, Zheng D, Simpson K (2002) Comorbid depression is associated with increased health care use and expenditures in individuals with diabetes. Diabetes Care 25(3): 464–70.
- 5. Kalsekar ID, Madhavan SM, Amonkar MM, Scott V, Douglas SM, et al. (2006) The effect of depression on health care utilization and costs in patients with type 2 diabetes. Manag Care Interface 19(3): 39–46.
- 6. Egede LE, Dismuke CE (2012) Serious psychological distress and diabetes: a review of the literature. Current psychiatry reports 14(1): 15–22
- 7. Hultsjo SM, Hjelm K (2012) Organizing care for persons with psychotic disorders and risk of or existing diabetes mellitus type 2. Journal of psychiatric and mental health nursing 19(10): 891–902
- 8. American Diabetes Association (2008) Economic costs of diabetes in the U.S. In 2007. Diabetes Care 31(3): 596–615
- 9. Hutter N, Schnurr A, Baumeister H (2010) Healthcare costs in patients with diabetes mellitus and comorbid mental disorders—a systematic review. Diabetologia 53(12): 2470–9
- 10. Egede LE, Gebregziabher M, Hunt KJ, Axon RN, Echols C, et al. (2011) Regional, geographic, and racial/ethnic variation in glycemic control in a national sample of veterans with diabetes. Diabetes Care 34(4): 938–43
- 11. Miller DR, Safford MM, Pogach LM (2004) Who has diabetes? Best estimates of diabetes prevalence in the Department of Veterans Affairs based on computerized patient data. Diabetes Care 27 Suppl 2: B10–21.
- 12. West AN, Lee RE, Shambaugh-Miller MD, Bair BD, Mueller KJ, et al. (2010) Defining “rural” for veterans' health care planning. J Rural Health 26(4): 301–9
- 13. Egede LE, Gebregziabher M, Dismuke CE, Lynch CP, Axon RN, et al. (2012) Medication nonadherence in diabetes: longitudinal effects on costs and potential cost savings from improvement. Diabetes Care 35(12): 2533–9
- 14.
White House website. Available at http://www.whitehouse.gov/the-press-office/2012/08/31/executive-order-improving-access-mental-health-services-veterans-service 2013. Accessed February 23, 2013.
- 15. Jimenez-Garcia R, Martinez Huedo MA, Hernandez-Barrera V, Lopez de Andres A, Martinez D, et al. (2012) Psychological distress and mental disorders among Spanish diabetic adults: a case-control study. Prim Care Diabetes 6(2): 149–56
- 16. Dixon LB, Kreyenbuhl JA, Dickerson FB, Donner TW, Brown CH, et al. (2004) A comparison of type 2 diabetes outcomes among persons with and without severe mental illnesses. Psychiatric services 55(8): 892–900
- 17. Kreyenbuhl J, Dixon LB, McCarthy JF, Soliman S, Ignacio RV, et al. (2010) Does adherence to medications for type 2 diabetes differ between individuals with vs without schizophrenia? Schizophrenia bulletin 36(2): 428–35
- 18. Kreyenbuhl J, Medoff DR, Seliger SL, Dixon LB (2008) Use of medications to reduce cardiovascular risk among individuals with psychotic disorders and Type 2 diabetes. Schizophr Res 101(1–3): 256–65
- 19. Druss BG, von Esenwein SA, Compton MT, Rask KJ, Zhao L, et al. (2010) A randomized trial of medical care management for community mental health settings: the Primary Care Access, Referral, and Evaluation (PCARE) study. The American journal of psychiatry 167(2): 151–9
- 20. McGuire J, Gelberg L, Blue-Howells J, Rosenheck RA (2009) Access to primary care for homeless veterans with serious mental illness or substance abuse: a follow-up evaluation of co-located primary care and homeless social services. Administration and policy in mental health 36(4): 255–64
- 21. Pirraglia PA, Rowland E, Wu WC, Friedmann PD, O'Toole TP, et al. (2012) Benefits of a primary care clinic co-located and integrated in a mental health setting for veterans with serious mental illness. Preventing chronic disease 9: E51.
- 22. Saxon AJ, Malte CA, Sloan KL, Baer JS, Calsyn DA, et al. (2006) Randomized trial of onsite versus referral primary medical care for veterans in addictions treatment. Medical care 44(4): 334–42
- 23. Frueh BC, Monnier J, Yim E, Grubaugh AL, Hamner MB, et al. (2007) A randomized trial of telepsychiatry for post-traumatic stress disorder. Journal of telemedicine and telecare 13(3): 142–7
- 24. Godleski L, Darkins A, Peters J (2012) Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006–2010. Psychiatric services 63(4): 383–5
- 25. Morland LA, Greene CJ, Rosen CS, Foy D, Reilly P, et al. (2010) Telemedicine for anger management therapy in a rural population of combat veterans with posttraumatic stress disorder: a randomized noninferiority trial. The Journal of clinical psychiatry 71(7): 855–63
- 26. Ruskin PE, Silver-Aylaian M, Kling MA, Reed SA, Bradham DD, et al. (2004) Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry 161(8): 1471–6
- 27. Dieterich M, Irving CB, Park B, Marshall M (2010) Intensive case management for severe mental illness. Cochrane database of systematic reviews 10: CD007906