Figures
Abstract
We examine effects of government spending on postdoctoral researchers’ (postdocs) productivity in biomedical sciences, the largest population of postdocs in the US. We analyze changes in the productivity of postdocs before and after the US government’s 1997 decision to increase NIH funding. In the first round of analysis, we find that more government spending has resulted in longer postdoc careers. We see no significant changes in researchers’ productivity in terms of publication and conference presentations. However, when the population is segmented by citizenship, we find that the effects are heterogeneous; US citizens stay longer in postdoc positions with no change in publications and, in contrast, international permanent residents (green card holders) produce more conference papers and publications without significant changes in postdoc duration. Possible explanations and policy implications of the analysis are discussed.
Citation: Hur H, Ghaffarzadegan N, Hawley J (2015) Effects of Government Spending on Research Workforce Development: Evidence from Biomedical Postdoctoral Researchers. PLoS ONE 10(5): e0124928. https://doi.org/10.1371/journal.pone.0124928
Academic Editor: Peter van den Besselaar, VU University Amsterdam, NETHERLANDS
Received: September 29, 2014; Accepted: March 11, 2015; Published: May 1, 2015
Copyright: © 2015 Hur et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Our analysis is based on the data from the Survey of Doctoral Recipients (SDR), conducted by the US National Science Foundation (NSF). The Survey is a common source of analysis used in similar studies. Researchers can receive the data directly from NSF at no cost following this link: (http://www.nsf.gov/statistics/license/).
Funding: The National Institute of General Medical Sciences of the National Institutes of Health supported this work (grants 5U01GM094141-02 and 2U01GM094141-05). The grants were awarded to the Ohio State University, the Massachusetts Institute of Technology (MIT), and Virginia Tech. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In 2013, the US government spent more than 130 billion dollars to fund research activities in basic and applied areas in fields as diverse as biomedical sciences, energy, space, environment, and defense [1]. The funding was distributed to research institutions and individuals through several government organizations, such as the National Institutes of Health (NIH) and the National Science Foundation (NSF) [2]. Government research funds are intended to foster research discoveries through training and recruiting researchers as well as developing and equipping related facilities and improving research productivity. Success in these policies can be evaluated by the extent to which funding allocations result in actual outcomes in the forms of science advancement, major breakthroughs, and workforce development, which all ultimately benefit the public.
While we intuitively expect that more government spending in the form of providing research grants should increase the population of researchers, there is a range of mixed arguments about possible effects on each individual researcher. On the positive side, more funding is expected to increase the productivity of individuals through helping equip research laboratories, and providing more opportunities for high-risk, but potentially high-outcome research [3–4]. More funding also helps career advancement in academia and provides opportunities for attending conferences, networking, and collaborations [5–6]. It can also result in more and longer professional training programs which can potentially lead to knowledgeable and productive individuals [7]. With less funding, researchers end up competing for limited resources and consequently spending a major portion of their time writing and submitting grant proposals which negatively affects their ‘actual’ research activities [3,8–9]. Besides, with limited resources, researchers are likely to feel frustrated and dissatisfied by lack of job security [10].
On the other hand, there has been a range of arguments about potential unintended consequences of increasing funding. In the long term, with more funding, universities recruit more students, who will later join the pool of funding applicants and decrease the likelihood of winning a grant for each individual [3,11]. At the institutional level, universities expand their capacities with expecting funding arrival and in turn put more pressure on their faculty and research staff to bring funding [11–12]. Moreover, more funding can potentially result in longer postdoctoral careers and fewer incentives to find permanent positions [13]. Thus, in a tight economy, more funding might only result in creating temporary “holding” positions rather than research opportunities. This can also send the wrong signals to potential PhD applicants. Previous research has shown that interest in PhD programs depends on the market conditions at the time of students' enrollment rather than the expected conditions at the time of graduation [14]. When prospective students see that most graduates are hired, even though some are in temporary positions, they might not accurately understand job market conditions, or might expect funding-based positions to be available for them at the time of graduation.
Given the huge investments that governments make in supporting research activities, the fuzzy and mixed evidence of effects of funding on research outcomes and science workforce development raises important questions for science policy scholars. Understanding changes in research productivity and outcomes is complex. One major reason is that researchers are not homogenous individuals; they differ in their capabilities, incentives, institutions, work colleagues, and family and personal conditions. The latter factors can in fact make a huge difference, given that preferences and limitations that individuals with different family and immigration statuses face are different. As shown in previous studies, such personal factors affect individual decision making [15–16]. Furthermore, established versus young researchers can behave differently in response to changes in funding.
Taking individual factors into account, we investigate effects of change in research funding on researchers’ productivity. In order to provide a more focused analysis and decrease several sources of variation in researchers’ behavior, activities, and incentives, we narrow down our analysis to postdoctoral researchers (postdocs)—a specific subpopulation of researchers who are a relatively young and unestablished body of the workforce. Performance of postdocs is arguably central for future performance of science and engineering, and due to the growing number of this population, there has been an increasing concern about productivity and career prospects of this population [17]. We will consider effects of several individual level factors that influence postdocs’ performance including their citizenship status.
In this study, we take the doubling of NIH funding in 1998–2003 as a unique natural experiment opportunity to analyze change in productivity of postdocs. The event is described in the methodology section in more details. We will later discuss policy implications of our findings.
Postdocs and the Question of Productivity
There are different stages of research workforce development, one of which is postdoctoral training. A postdoc is an individual that holds a doctoral degree and works in a temporary research position, usually in a university, research center, or other research institutions and laboratories [18]. As stated by the National Postdoctoral Association, postdoc positions are aimed at helping individuals acquire technical and professional skills that they need in order to pursue research careers [18].
The majority of postdocs are interested in tenure-track academic jobs [19]. The job market for tenure-track positions has been very competitive since the rate of job openings is less than the rate of PhD graduation [13,20]. As a result, every year some of the individuals who do not land academic positions take temporary postdoc positions to do more research and improve their chances in the future job market. This trend has resulted in a continuous growth in the number of postdocs, which has almost doubled since 1995 with domestic researchers staying longer in postdoc positions [7]. These days, most individuals stay three or more years in postdoc positions [21] and at the end of the experience, only 21% of them land tenure-track positions [19].
Here, we explore possible effects of change in research funding on postdocs.
Holding and Training Positions
In order to investigate potential effects of an increase in research funding on postdocs’ productivity, we should consider career characteristics of postdoc positions. The literature considers postdoc positions as both holding positions and training opportunities.
Postdoctoral positions play the role of holding positions and exist because there is a shortage of tenure-track opportunities [13]. They help individuals take advantage of relatively low-paid academic positions while they seek better permanent positions [22–23]. When due to economic conditions, permanent jobs are limited, researchers more often take such opportunities [24–25]. Several recent studies argue that permanent faculty positions have become very difficult to find. Every year many researchers extend the duration of postdoctoral positions, and as a result the number of postdocs has been increasing [13,23,26]. Of course, waiting durations depend on the availability of funding; in the absence of funding and with fewer postdoc positions, people have to make a choice quickly and probably leave academia.
If we follow this line of reasoning, one expects that more government spending should expand the holding positions, resulting in longer postdoctoral careers. With more funding, individuals will have more opportunities to wait in the line before their desired jobs come up. Principal investigators are also likely to keep their trained and experienced postdocs when financial resources are more available. Thus we propose to investigate the following hypothesis:
H1: More funding results in longer postdoc durations.
On the other hand, postdoc positions are training opportunities. There are various studies that provide evidence on the positive effects of postdoctoral training [27], which suggests more investments and government spending in this regard. Postdoc opportunities can help individuals get additional training and get prepared for new research activities [28]. Studies show that postdoctoral training improves long term productivity of researchers reflected in their higher number of publications [29] and shorter time to secure first research grants [30]. Su (2013) shows that long postdoctoral research positions up to three years increase the chance of landing prestigious academic careers [31]. Success in the job market, publications, and grant applications is critical in researchers’ promotion in academic positions and getting tenured [4–6]. Thus, postdoc trainings can help long-term achievements of individuals in academia.
Postdocs can also help team learning. They usually work in research teams with students, professors, and other staff members. In comparison to other team members, especially students, postdocs are already trained at a PhD level and can quickly start contributing to research projects. They can help by supervising students and managing different parts of research projects [13]. Furthermore, through postdoc opportunities, young scholars usually learn to direct groups of research students and help collaborative activities [17]. Their performance, thus, influences research teams and contributes to higher team performance [32].
Therefore, we expect that more funding should result in increased positive effects of postdoctoral positions. With more and larger grants, postdocs feel more job security and can take higher risk initiatives. Also, with more funding resources, postdocs can attend more conferences and training sessions (such as method workshops), work in better equipped laboratories, and work in larger inter-disciplinary teams. Especially in experimental fields, research activities are costly and resource dependent. Thus, we offer the following hypothesis:
H2a: More funding increases average productivity of postdocs.
While we intuitively expect to find support for the positive effects on productivity (H2a), there is a possibility of observing a reverse relationship between funding and average productivity due to a set of secondary effects. For example, with more funding it is likely that more new postdocs will be hired. More postdoc openings can potentially mean weaker competition which can lead to lower quality postdocs who might not be as productive as previous ones. Furthermore, in short term, new hires can take more time from their supervisors and other team members to catch up and get initial training if needed. We therefore offer a competing hypothesis:
H2b: More funding decreases average productivity of postdocs.
Asymmetric Effects of Funding
The complexity of understanding how researchers in general and postdocs in particular react to change in funding goes beyond difficulties of predicting how the population changes and whether in aggregate it becomes weaker or stronger. In fact, we should make sure that one does not assume that the population of postdocs is a homogenous population with similar preferences. These individuals differ in their capabilities, incentives, institutions, and family and personal conditions. Some of these factors can influence how researchers’ productivity changes when more resources are provided. While in our analysis we will control for various individual factors, such as family status, age, and institutions, we hypothesize that the effects of change in funding can be asymmetric when it comes to citizenship status.
US citizens may react differently from permanent residents (green card holders) when more resources are provided, and they all might react differently in comparison to temporary residents (visa holders). Former studies have shown that students or researchers with different citizenship or residency statuses behave differently [7,15,33]. Heterogeneities in preferences of these groups are rooted in their varying opportunities and environmental restrictions, backgrounds, and the selection processes they have faced.
In terms of opportunities and environmental restrictions, temporary residents are the most restricted individuals. Temporary residents face visa-related limitations which affect duration of stay in the US as well as the types of research positions that they can take. For example, there is a six-year cap for H1B visas which influences how long visa holders can remain in a postdoc position. As a result, temporary residents might have more incentives to move on to permanent jobs which are more likely to facilitate their transition to a permanent residency status. In simple words, due to visa restrictions, postdoc positions are by nature temporary positions for visa holders, but they can turn to quasi-permanent positions for other groups who face little or no residency limitations. Thus, we specifically expect that more funding should increase postdoc duration for US citizens and permanent residents as stated in the following hypothesis:
H3: For US citizens and international permanent residents, more funding results in longer postdoc durations.
In terms of educational background, US citizens are different from the other two groups since they are highly likely to have gotten their high school and undergraduate studies in the US. International citizens usually complete their high school and undergraduate studies in their home countries. Thus, one might expect citizenship to be a proxy for different educational backgrounds. Whether or not the difference in high school and undergraduate backgrounds of internationals versus US citizens makes one group more competitive than the other is an empirical question.
In terms of selection processes, permanent residents and temporary residents have been through several self-selection stages. The decision to take the risks of short term or long term immigration has potential dollar and social costs for individuals. In particular, permanent residents experience a few more selection processes; for example, there is an implicit analysis done to quantify the potential green card holders’ productivity. However, when we look at US citizen researchers, we are talking about the whole spectrum.
Environmental restrictions can also incentivize productivity for temporary residents. NSF reports that more than 77% of international PhD students in the US are interested in staying in the US after graduation [34]. This group should have more incentives to use every opportunity for improving their performance in order to find permanent positions.
Thus, due to the differing backgrounds and selection processes for international residents (temporary and permanent) and US citizens, as well as restrictions for temporary residents, we expect international workers to have more incentives to improve their performance than citizens, and to benefit the most from available resources. This can be reflected in their productivity measures. Thus, we offer the following hypothesis:
H4: For international permanent residents and temporary residents, more funding results in higher productivity.
Methodology
The Doubling Event
In 1997, the Congress doubled NIH budget in a very short period of five years, from $13.6 billion in 1998 to $27.1 billion in 2003, responding to higher demands for scientific studies in health sector [35]. The goal was to significantly boost research activities in biomedical fields and to increase the domestic capacities for scientific research, including maintaining and improving research infrastructure (such as laboratories) and empowering research workforce development.
The doubling event was a surprise [36–37]. Historically, predicting annual research budget in the US has been very difficult [11–12]. One reason is that every year all allocated budget should be spent and cannot be carried over to the next year [8,12]. Furthermore, there are always last minute decisions that the government and the congress should make in order to balance the whole budget, which can affect research spending, a recent example being the research budget sequestration [10]. Specifically, in regards to the doubling event, the change in budget was huge and unique in the history of US science which made it further unpredictable [36]. Evidence suggests that the biomedical science community in general and biomedical postdocs in particular did not predict the huge size and the short term period of change in the budget [36–37].
The event had several immediate impacts, as depicted in Table 1. In comparison to other fields, the ratio of NIH funding to all other federal research funding in the US increased dramatically from 0.54 to 0.81 in 2001 and 0.92 in 2003. In this period, the average grant size increased by 23% (inflation-adjusted), giving more spending flexibility to principal investigators. The number of awards increased from 7,080 to 10,393 in 2003. In the same time period, the number of applications for research project grants (RPG) in NIH also increased from an average of 24,355 to 34,710 in 2003, maintaining the overall success rate of applications (the ratio of grant applications funded) around 30% in this time period. Change in funding also resulted in more PhD admission in biomedical sciences; in five years the number of admissions increased by 23% [38].
The doubling event, both in terms of numbers and the created attention, has been a major shift in US biomedical research. At the outcome level, it is expected that such a huge surge in research funding in a relatively short time period should have major impacts on the biomedical research community. The event provides a unique natural experiment context to study effects of rapid change in funding on research enterprises and workforce. We focus on the specific question of productivity as affected by funding, and study the effects of doubling on postdocs in biomedical fields. The dataset and variables are described in the following.
Data
Our analysis is based on data from the 1995, 2001, and 2003 of Survey of Doctoral Recipients (SDR), conducted by NSF. The dataset is a nationally representative sample of the population of PhD graduates living in the US. Participants’ consent is obtained through NSF. The dataset is anonymized and de-identified by NSF, and is made publicly available for research purposes upon request from NSF. The dataset is IRB exempt by behavioral and social sciences committee of IRB at the Ohio State University.
For the purpose of our analysis, we draw data on current postdoctoral researchers from these datasets. We code the following fields as related to biomedical sciences: biochemistry and biophysics, biology, cell and molecular biology, genetics, animal and plant, microbiological sciences and immunology, nutritional sciences, pharmacology, human and animal, physiology and pathology, human and animal, and zoology [38].
Outcome Variables
We use four outcome measures. The first measure is time in the latest postdoc. This variable is calculated by using the starting month and year of the recent postdoctoral job and the time that the survey is conducted. The second measure is time since graduation. Time since graduation is calculated by using the awarding month and year of US PhD and the time that the survey is conducted. The third measure is the number of papers presented at conferences during the last five years. The fourth measure is the number of (co)authored articles accepted for publication during the last five years. Table 2 reports descriptive statistics for the outcome variables.
Individual Controls
Apart from the four variables mentioned, we control for age, gender, race, marriage, children, working hours, cohort, and research focus. These variables are provided in the survey. The variable research focus is a dummy variable that is set to be one for any respondent that works in basic or applied research, and zero otherwise.
Organizational Controls
We use the 1994 Carnegie classification code to define institutional rank scale. Postsecondary education institutions are classified as following: Research University I, Research University II, Doctorate Granting I, Doctorate Granting II, and others (Master’s Universities and Art College I, II, Baccalaureate College I, II, Associate of Arts College, Professional Schools and Specialized Institutions). We include this measure because of the assertion that higher ranking university postdoctoral researchers have more chances to contact and work with other professors in their university or other universities. Moreover, as NIH grants are highly concentrated geographically and institutionally, it is important to control for institutions in the analyses to allow for comparison by organization.
Model
We conduct two rounds of analyses. In the first round, we look at the whole sample of postdoctoral researchers regardless of their citizenship. In the second round of analysis, we run models separately for different citizenship statuses (US citizens, permanent residents (green card holders), and temporary residents (visa holders)).
Since doubling NIH funding started in 1998, we compare data in 1995 with 2001 and 2003. Due to NIH’s mission, its funding is more concentrated among biomedical researchers than other fields. More funding is expected to increase resources that ultimately improve productivity of scientists in biomedical sciences than scientists in other fields. We will compare the productivity of postdoctoral researchers in biomedical fields against postdoctoral researchers in non-biomedical fields in a classic difference-in-difference analysis.
Specifically, we run the following model with four dependent variables.
In this equation, yit is the outcome of interest for individual i given time t. The dummy variable BiomedicalField captures possible differences between the treatment (biomedical postdocs) and comparison groups (non-biomedical postdocs). The variable DoublingFunding is equal to 1 for the second time period (2001 and 2003), and equal to zero for year 1995. The variables k represent other control variables (age, gender, race, marriage, children, research activity, working hours, research focus, cohort, and institutional rank). Finally, duration of postdoctoral career is added as a control variable when we study conference papers and publications as dependent variables. In a difference-in-difference analysis, the coefficient of the interaction term (β3) represents the effect.
We use the ordinary least-square regression for the first two dependent variables and test the first and third hypotheses. For testing the second and forth hypotheses, since the dependent variables (number of conference papers and publications) are non-negative integers with a skewed distribution (there are many 0 or 1 values), we conduct negative binomial regression analysis.
Results
Effects on the Whole Population
Graphs in Fig 1 compare the dependent variables before and after doubling for non-biomedical and biomedical postdocs. In Fig 1A, the average time in latest postdoc in biomedical fields increases about four months after doubling funding. In non-biomedical fields, this number decreases by about two months. As Fig 1B shows, the variable time since graduation is increased in biomedical fields by about two months after doubling funding, while in non-biomedical fields, it decreased by about three months. Fig 1C shows that the average number of conference papers behaved similarly in biomedical and non-biomedical fields after doubling. Fig 1D shows a relatively steady number of publications per person in biomedical fields and a declining trend in non-biomedical fields.
Error bars represent 95% distribution.
Table 3 depicts effects of doubling NIH funding on the whole population of postdoctoral researchers. It summarizes the results of regressions reporting β1, β2 and β3 from Eq 1. β1 (the coefficient of Biomedical Field) represents the overall difference between treatment group and control group, β2 (the coefficient of Doubling Funding) represents the overall trend for all fields, and β3 (the coefficient of Doubling Funding*Biomedical Field) represents the main effect from a difference-in-difference analysis: the effect of doubling funding for the treatment group controlling for secular and between field effects.
The results show that β3 is significant in two of the analyses. Comparing with non-biomedical researchers, time in the latest postdoc has significantly changed after the start of the funding policy. Postdoctoral researchers in biomedical fields stayed around six more months (p <.01) in comparison to non-biomedical researchers. Moreover, after doubling funding policy, time since graduation of doctoral recipients in biomedical fields was increased by approximately four months compared to non-biomedical fields (p<.05). As shown in Table 3, the numbers of conference papers and publications per person have not significantly changed after the doubling policy.
Among the control variables (not shown in Table 3), gender is significant in the third and fourth models (p<0.1), implying males write more papers and go to more conferences. Postdocs who are parents are also less likely to stay as long in postdoc positions, which can be due to lower average wage of such positions and the need to take permanent jobs.
In summary, the analysis shows that doubling funding increased time in the recent postdoc position and time since graduation among biomedical postdocs (support for hypothesis H1) with no observable effects on productivity (no support for hypotheses H2a or H2b). To examine the robustness of our findings, we conduct two additional analyses with the same data reported in the supporting information (S2 and S3 Tables). First, we limit the data points to individuals who have government funding and, second, use different weights provided in the dataset. Similar effects appear; variables related to postdoc duration increase in treatment groups with no significant change in conference papers and publications.
Effects on Different Citizen Groups
Table 4 summarizes the main results of the analysis on different citizen groups of postdocs. We control for a wide range of individual and organizational variables as described in the method section.
The first column in Table 4 shows the effects of increased funding on the time in the latest postdoc. As depicted, the duration has increased for US citizens as a result of the doubling policy by about eight months (p <.01). Change in time in recent postdoc among permanent residents is not statistically significant and for visa card holders is in a negative direction and modestly significant (p <.10). The second column in Table 4 shows the effects of more funding on time since graduation of postdocs. The variable has increased for US citizens, approximately by five months (p <.05). Overall, the analysis shows that as a result of doubling funding, US citizens wait longer in the pipeline in postdoc positions.
The third column in Table 4 reports results for the average number of conference papers. Permanent residents are the only ones that show a significant change in number of conference papers per individual (p <.05). The fourth column in Table 4 shows change in publications. Again, permanent residents show a significant positive change (p <.05). We also estimated the size of the effect by looking at incidence-rate ratios. The analysis reveals that the magnitude of the effect is meaningful: for permanent residents, the numbers of conference papers and publications per person have increased by 44.4% and 43.7% as results of doubling. The analysis on US citizens does not show any significant change in conference papers and publications despite them staying longer in their latest postdoc positions.
In summary, in regards to the effects of doubling policies on postdoctoral researchers, our analysis shows that:
- More funding resulted in US citizens staying longer in recent postdoctoral positions and extended time since graduation, with no significant effect on other groups (support for hypothesis H3 for US citizens only).
- More funding increased number of conference papers for permanent residents with no significant effect on other groups (support for hypothesis H4 for permanent residents only).
- More funding increased number of publications for permanent residents with no significant effect on other groups (support for hypothesis H4 for permanent residents only).
Similar effects can be found in the sensitivity analysis reported in the supporting information (S2 and S3 Tables), which shows an increase in the number of both conference papers and publications for permanent residents and longer postdocs for US citizens.
Discussion and Conclusion
We analyzed the effects of government research spending on researchers’ productivity by focusing on the population of postdoctoral researchers in biomedical sciences and using data from Survey of Doctorate Recipients. We specifically focused on analyzing changes in productivity of postdocs before and after the US government’s decision to increase NIH funding in 1997. The main hypothesis was that, with more funding, productivity of researchers should increase as they are able to attend more conferences, are better equipped with research facilities and laboratories, and have more supporting students and staff in their teams. Productivity was measured in terms of number of publications and conference papers per researcher. In addition, we looked at the duration of postdoctoral training by focusing on time in latest postdoc positions and time since graduation. The analysis was directed toward understanding the effects on the whole group of postdoctoral researchers in biomedical sciences and on different citizen groups, hypothesizing that the effects might be different for domestic versus international researchers.
Our results show little change in productivity but more significant changes in duration of postdocs and waiting time in the pipeline until landing permanent positions. In comparison to non-biomedical fields, more government spending has resulted in longer postdoc durations. It seems that the first reaction to increase in funding for postdocs is to stay in their current positions, and for supervisors is to keep their currently trained and experienced researchers. One reason for staying longer in a postdoc position despite the relatively lower pay is that individuals can wait for and seek their desired permanent position (which might be a tenure-track academic position) for a longer time period. This observation corroborates with theories that consider postdoc positions as holding positions.
When the population of analysis is segmented by citizenship, we find heterogeneous effects on different groups of biomedical researchers based on their citizenship. Specifically, US citizen postdocs stayed longer in their recent postdoc positions while their productivity was unchanged. In contrast, international permanent residents improved their productivity without significantly extending their postdoc duration. Overall, the results support the idea that different groups of researchers might react differently to change in level of funding due to their differing preferences.
This study contributes to the studies of research workforce development, and our findings corroborates with recent argument about systematic problems in science education especially in biomedical fields [7–13,39]. Our study is different from others since it moves the focus from performance of the population of scientists to individual level productivity. We also show how the reactions of the research community can be complex and asymmetric to change in funding. In other words, our results imply that the question of “what are the effects of change in funding” should be supplemented with “for which sub-population of researchers.”
This study has several limitations that suggest new avenues of research. While one of the richest available data sources is used in this study, we cannot rule out all potential variables affecting postdoctoral researchers’ behavior. We have tried to use most of the available variables at the individual and institutional level to control for a wide range of effects. In terms of productivity, we narrowed down the concept to the more objective variables of number of publications and conference papers. We acknowledge limitations of this operationalization. One would benefit from having data on quality of papers, journals’ rankings, and impacts (in terms of citations or other measures) in their analysis. These limitations suggest cross-checking the results with other data sources and other methodological approaches to better inform policy makers [40–41]. Furthermore, our analysis was focused on the time frame of 1995 to 2003. One would benefit from analyzing longer-term effects, maybe over a decade. This will require disentangling effects of funding on PhD admission decisions and on PhD training from sole effects on postdocs’ productivity, which can be explored in future research.
Policy Implications
As noted in the introduction, identifying “policy tools” to increase the productivity of postdoctoral researchers is a critical question for US science and workforce development policy. The primary tool for the federal government to improve research output is to supply more resources through federal agencies such as NIH and NSF. However, aggregate levels of funding seem to be relatively blunt policy tools, in that they may increase resources but do not have much impact on the distribution of funds or productivity of researchers. Some of the key questions that policymakers should consider are: Who gets the funding? Are some researchers benefiting more than other groups? What fields are being funded?
Another implication is the need to consider multiple measures of performance for research policy evaluation. This analysis shows that dramatic increases in funding led to changes in the amount of time postdoctoral researchers serve in these roles, providing validation of a sort for policy makers that more funding can lead to increased opportunity to conduct scientific research. However, on the outputs side, simply increasing funding does not appear to have accelerated the publication or presentation of papers by each biomedical scientist. This is despite increased opportunity to do research. The reality is that the quality of research is a much more difficult outcome to manipulate through policy. Quality science is generally conceptualized as both the result of consistent effort over time, as well as flashes of brilliance and discovery [42]. In sum, we should judge the effectiveness of these policies based on the science output as well as quality of the results.
Moreover, it seems funding policies should be supplemented by other policy actions in order to lead to the desired goals. In other words, increasing funding is not a “silver bullet.” If more funding mainly results in longer postdoc positions, one might doubt if such a reaction from the science community is optimal while many new PhD graduates are looking for postdoctoral training positions. At the end, the number of tenure-track positions is limited and has been constant in the past years, so longer postdocs might translate to longer waiting times before eventually deciding to leave academia for industry—an inefficient policy.
Furthermore, our study provides evidence that the reaction of domestic and international researchers to change in research funding varies. US science policy has depended for decades on international researchers [33]. A substantial fraction of researchers in virtually every science sub-discipline were educated overseas and moved to the US either for doctoral or postdoctoral training [43]. Visa policies restrict non-native researchers’ job mobility, leading to potentially different motivations and providing support for the hypothesis that international researchers behave differently and might respond in unique ways to increases in government funding [15,43]. This implies that the science community is not homogenous in preferences and the same policy can have different results for different groups of the population, a fact that policy makers should consider.
Moving forward, the US government is facing increasing resource pressures and, after the American Recovery and Investment Act in 2009, is trying to readjust the science community to lower levels of resources for basic and applied work. It is necessary to come up with better policies that make science funding more effective in output and quality metrics.
Supporting Information
S1 Table. Description of variables.
This table complements Table 2 in the paper by presenting descriptive statistics, definitions, and sources of variables.
https://doi.org/10.1371/journal.pone.0124928.s001
(PDF)
S2 Table. Difference-in-Difference estimates within only government-funded postdocs.
We report extra analysis to demonstrate robustness of our main results (Tables 3 and 4) to a major change in assumptions. S2 Table limits the sample size to only individuals who have government funding (excluding postdocs with no government funding).
https://doi.org/10.1371/journal.pone.0124928.s002
(PDF)
S3 Table. Difference-in-Difference estimates, weighted.
We report an analysis to demonstrate robustness of our main results (Tables 3 and 4) to another major change in assumptions. This table uses a different survey weight system (provided in the dataset).
https://doi.org/10.1371/journal.pone.0124928.s003
(PDF)
Acknowledgments
The discussion and conclusions in this paper are those of the authors and do not necessarily represent the views of the National Institutes of Health, the Ohio State University, MIT, or Virginia Tech. The authors thank Richard Larson of MIT and Hamed Ghoddusi of Stevens Institute of Technology for helpful comments on an earlier draft.
Author Contributions
Analyzed the data: HH NG JH. Contributed reagents/materials/analysis tools: HH NG JH. Wrote the paper: HH NG JH.
References
- 1.
Office of Management and Budget (2014) Historical tables: budget of the United States government. Fiscal Year 2015. Washington, DC, Available: http://www.whitehouse.gov/sites/default/files/omb/budget/fy2015/assets/hist.pdf Accessed 1 Dec 2014.
- 2.
Mervis J (2014) U.S. Science agencies get some relief in 2014 budget. Science Insider. Available: http://news.sciencemag.org/funding/2014/01/u.s.-science-agencies-get-some-relief-2014-budget Accessed 1 Dec 2014.
- 3. Zerhouni EA (2006) NIH in the post-doubling era: realities and strategies. Sci 314(5082):1088–1090.
- 4.
Laudel G, Gläser J (2012) The ERC's impact on the grantees' research and their careers (EURECIA, Work package 4 summary report). Available: http://www.laudel.info/wp-content/uploads/2013/12/EURECIA-WP4-report-final-Jan2012.pdf Accessed 1 Dec 2014.
- 5. van Arensbergen P, van der Weijden I, van den Besselaar P (2014) Different views on scholarly talent: what are the talents we are looking for in science? Res Eval 23(4):273–284.
- 6. Bloch C, Graversen EK, Pedersen HS (2014) Competitive research grants and their impact on career performance. Minerva 52(1):77–96.
- 7. Ghaffarzadegan N, Hawley J, Desai A (2014) Research workforce diversity: the case of balancing national versus international postdocs in US biomedical research. Syst Res Behav Sci 31(2):301–315. pmid:25368504
- 8.
Diaz MG (2012) Unintended effects of changes in NIH appropriations: challenges for biomedical research workforce development, electronic. M.Sc. Thesis, Massachusetts Institute of Technology. Available: http://dspace.mit.edu/handle/1721.1/72892 Accessed 1 Dec 2014.
- 9. Alberts B, Kirschner MW, Tilghman S, Varmus H (2014) Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci 111(16):5773–5777. pmid:24733905
- 10. Alberts B (2013) Am I wrong? Sci 339(6125):1252–1252. pmid:23493681
- 11. Teitelbaum MS (2008) Structural disequilibria in biomedical research. Sci 321(5889):644–645. pmid:18669847
- 12. Larson RC, Ghaffarzadegan N, Diaz MG (2012) Magnified effects of changes in NIH research funding levels. Serv Sci 4(4):382–395. pmid:24489978
- 13.
Ghaffarzadegan N, Hawley J, Larson R, Xue Y (2014) A note on PhD population growth in biomedical sciences. Syst Res Behav Sci: In press. https://doi.org/10.1002/sres.2324
- 14. Blume-Kohout ME, Clack JW (2013) Are graduate students rational? evidence from the market for biomedical scientists. PLoS ONE 8(12):e82759. pmid:24376573
- 15. Lan X (2012) Permanent Visas and temporary jobs: evidence from postdoctoral participation of foreign PhDs in the United States. J Policy Anal Manage 31(3):623–640.
- 16. Lan X (2013) The effects of green cards on the wages and innovations of new PhDs. J Policy Anal Manage 32(4): 807–834.
- 17. Schillebeeckx M, Maricque B, Lewis C (2013) The missing piece to changing the university culture. Nat Biotechnol 31(10):938–941. pmid:24104758
- 18.
National Postdoctoral Association (2014) What is a postdoc. Available: http://www.nationalpostdoc.org/policy/what-is-a-postdoc Accessed 1 Dec 2014.
- 19. Bonetta L (2010) The postdoc experience: taking a long term view. Sci 329(5995):1091–1091.
- 20. Larson RC, Ghaffarzadegan N, Xue Y (2014) Too many PhD graduates or too few academic job openings: the basic reproductive number R0 in academia. Syst Res Behav Sci 31(6):745–775. pmid:25642132
- 21.
De Jesus, M (2012) Opinion: The postdoc challenge. The Scientist. Available: http://www.the-scientist.com/?articles.view/articleNo/32445/title/Opinion—The-Postdoc-Challenge. Accessed 1 Dec 2014.
- 22.
Stephan P (2012) How economics shapes science. Cambridge, MA: Harvard University Press.
- 23. Stephan P (2013) How to exploit postdocs. Bioscience 63(4), 245–246.
- 24. Zumeta W (1984) Anatomy of the boom in postdoctoral appointments during the 1970s: troubling implications for quality science? Sci Technol Human Values 9(2):23–37.
- 25.
Zumeta W (1985) Extending the educational ladder: the changing quality and value of postdoctoral study. Lexington, MA: Lexington Books.
- 26.
Davis G (2009) Improving the postdoctoral experience: an empirical approach. In: Freeman RB, Goroff DL, editors. Science and engineering careers in the United States: an analysis of markets and employment. University of Chicago Press. pp. 99–127.
- 27. Levey GS, Sherman CR, Gentile NO, Hough LJ, Dial TH, et al. (1998) Postdoctoral research training of full-time faculty in academic departments of medicine. Ann Intern Med 109(5):414–418.
- 28.
Kolb DA (1984) Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.
- 29. Steiner JF, Lanphear BP, Curtis P, Vu KO (2002) Indicators of early research productivity among primary care fellows. J Gen Intern Med 17(11):854–860.
- 30.
Gentile NO, Levey GS, Sherman CR, Hough LJ, Dial TH, et al. (1989) Post-doctoral research training of full-time faculty in departments of medicine. Washington, DC: Association of American Medical Colleges.
- 31. Su X (2013) The impacts of postdoctoral training on scientists' academic employment. J High Educ 84(2):239–265.
- 32. Scribner S, Beach K (1993) An activity theory approach to memory. Appl Cogn Psychol 7(3):185–190.
- 33.
Bound J, Turner SE, Walsh P (2009) Internationalization of U.S. doctorate education. In: Freeman RB, Goroff DL, editors. Science and engineering careers in the United States: an analysis of markets and employment. University of Chicago Press. pp. 59–97.
- 34.
National Science Foundation (2012) Science and engineering indicators 2012. Available: http://www.nsf.gov/statistics/seind12/ Accessed 1 December, 2012.
- 35.
Smith PW (2006) The national institutes of health (NIH): organization, funding, and congressional issues. Washington, DC: Congressional Research Service. Available: http://www.nih.gov/about/director/crsrept.pdf Accessed 1 Dec 2014. https://doi.org/10.1002/pam.21639
- 36. Mervis J (1997) Senate bills back huge increases. Sci 275(5300):608–608. pmid:9019814
- 37. Lawler A (1997) Smoother road for R&D spending? Sci 275(5302):916–919.
- 38.
Garrison HH, Ngo K (2011) Education and employment of biological and medical scientists: Data from national surveys. Available: http://report.nih.gov/investigators_and_trainees/ACD_BWF/pdf/FASEB_PPT-Slide_Set.ppt Accessed 1 Dec 2014.
- 39.
Hoots WK, Abkowitz JL, Coller BS, DiMichele DM (2015) Planning for the future workforce in hematology research. Blood: In press. https://doi.org/10.1182/blood-2015-01-617811
- 40. Weinberg BA, Owen-Smith J, Rosen RF, Schwarz L, Allen BM, Weiss RE, Lane J (2014) Science funding and short-term economic activity. Sci 344(6179): 41–43. pmid:24700844
- 41. Kim H, MacDonald RH, Andersen DF (2013) Simulation and managerial decision making: a double-loop learning framework. Public Adm Rev, 73(2): 291–300.
- 42.
Root-Bernstein RS (1992) Discovering: inventing and solving problems on the frontiers of scientific knowledge. Cambridge, MA: Harvard University Press.
- 43. Garrison HH, Gerbi SA, Kincade PW (2003) In an era of scientific opportunity, are there opportunities for biomedical scientists? FASEB J 17(15):2169–2173. pmid:14525935