Building environmental literacy (EL) in children and adolescents is critical to meeting current and emerging environmental challenges worldwide. Although environmental education (EE) efforts have begun to address this need, empirical research holistically evaluating drivers of EL is critical. This study begins to fill this gap with an examination of school-wide EE programs among middle schools in North Carolina, including the use of published EE curricula and time outdoors while controlling for teacher education level and experience, student attributes (age, gender, and ethnicity), and school attributes (socio-economic status, student-teacher ratio, and locale). Our sample included an EE group selected from schools with registered school-wide EE programs, and a control group randomly selected from NC middle schools that were not registered as EE schools. Students were given an EL survey at the beginning and end of the spring 2012 semester. Use of published EE curricula, time outdoors, and having teachers with advanced degrees and mid-level teaching experience (between 3 and 5 years) were positively related with EL whereas minority status (Hispanic and black) was negatively related with EL. Results suggest that school-wide EE programs were not associated with improved EL, but the use of published EE curricula paired with time outdoors represents a strategy that may improve all key components of student EL. Further, investments in teacher development and efforts to maintain enthusiasm for EE among teachers with more than 5 years of experience may help to boost student EL levels. Middle school represents a pivotal time for influencing EL, as improvement was slower among older students. Differences in EL levels based on gender suggest boys and girls may possess complementary skills sets when approaching environmental issues. Our findings suggest ethnicity related disparities in EL levels may be mitigated by time spent in nature, especially among black and Hispanic students.
Citation: Stevenson KT, Peterson MN, Bondell HD, Mertig AG, Moore SE (2013) Environmental, Institutional, and Demographic Predictors of Environmental Literacy among Middle School Children. PLoS ONE 8(3): e59519. doi:10.1371/journal.pone.0059519
Editor: Randen Lee Patterson, UC Davis School of Medicine, United States of America
Received: December 21, 2012; Accepted: February 15, 2013; Published: March 22, 2013
Copyright: © 2013 Stevenson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was supported by a grant from North Carolina SeaGrant (http://www.ncseagrant.org, NCSU Project (1504) 2012-0080). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Direct responses to global environmental crises can slow the tide of environmental degradation, but reversing the trend will require an environmentally literate citizenry. These direct responses have included converting nearly 12% of the earth’s land base to protected areas, which has undoubtedly altered the trajectory of species extinction . Similarly, international climate change treaties guided by the UN Framework Convention on Climate Change (UNFCCC) and carbon markets represent the seeds of a response to global warming . Comparable initiatives are associated with marine protected areas , water conservation and quality , and erosion mitigation . The existence of these programs and development of future responses, however, depend entirely on publics who understand ecology, care about the environment, possess skills to assess environmental risk, and share a commitment to sustainability.
These four attributes of a citizenry capable of achieving sustainability are reflected by the four components of environmental literacy (EL): Knowledge, Affect, Cognitive Skills, and Behavior . The environmental education (EE) movement revolves around promoting EL. Precursors to environmental education included both nature study and outdoor education early in the 19th century, but environmental education as a field gained momentum in 1977 with the first intergovernmental conference on environmental education organized by the United Nations Education, Scientific, and Cultural Organization (UNESCO). The ensuing Tbilisi Declaration established EE objectives, and subsequent collaborations between environmental literacy experts have focused the definition of environmental literacy on the following topics: ecological knowledge, environmental attitudes and sensitivity, issue and action skills, and verbal and actual commitment to proenvironmental behavior , .
Environmental literacy helps to create a citizenry equipped to tackle current and emerging environmental concerns worldwide , and each component of EL is critical to this goal. Lack of empirical ecological knowledge about problems like extinction and climate change makes the problems insurmountable. For instance, a 2011 national poll found that 63% of adults in the United States “think there is solid evidence that the earth is warming,” which was down from 71% who saw solid evidence in 2008 . If citizens do not accept overwhelming scientific evidence of warming trends, behavior to mitigate effects of global warming and support for government policies is highly unlikely . Similarly, even an empirically aware public must care about the environment, have skills required to identify problems and solutions, and a willingness to act before the problems can be adequately addressed. That these components of EL are necessary to ensure pro-environmental behavior is both intuitive and supported by research. Hines, Hungerford, & Tomera  found that knowledge together with pro-environmental attitudes are requisites to environmentally responsible behavior, and a model has been further refined that includes in-depth knowledge about issues, personal investment in the environment, knowledge of and skill in using action strategies, and intention to act , all key components of EL .
Classroom activities have limited ability to change some components of EL, notably emotional connection to the environment (affect) and environmental behavior , and outdoor education has been promoted as a solution to this challenge. Time outdoors has been linked with the affective components of EL, a key predictor for proenvironmental behavior . Duerdon and Witt  found that classroom activities were associated with improved environmental knowledge whereas field experiences were correlated with improved affect which indirectly improved behavior by activating knowledge. Skelly and Zajicek  also found time outdoors was a key predictor of pro-environmental attitudes in their evaluation of a gardening program. In Louisiana, horticulture teachers found that students participating in a program with an outdoor component were more aware of their role in the environment than students who did not participate in the program .
The dire need for EL has spurred legislation in several nations as well as attention at international conferences. In the United States, the Environmental Education Act of 1970 was one of the first major pieces of EE legislation and established the US Office of Environmental Education. Internationally, both the 1975 Belgrade Charter out of an International Workshop on Environmental Education hosted by the United Nations Environmental Program (UNEP) and Educational, Scientific and Cultural Organization (UNESCO), as well as the 1977 Tbilisi Declaration, brought international attention to the field of environmental education. Legislation stemming from these conferences included resolutions of the European Union Council of Environmental Education in 1988, the 1995 National Environment Statute in Uganda , and the Environmental Education Act of 1990 in the United States, which reestablished the Office of Environmental Education after its elimination in the 1980s . Most recently, a groundswell of support for connecting children to nature spurred the No Child Left Inside Act of 2011 (NCLI) in the United States, which would have provided $100 million in funding for state EE efforts if passed. Though much of the aforementioned legislation supported specific programs, this act emphasized the role of outdoor education, integrated EE into formal schooling, and required the development of state EE standards, assessment, and teacher training through state-wide EL plans adopted by state boards of education .
Given the potential role of EL in addressing global environmental crises and the rapid expansion of EE around the world, there is surprisingly little empirical research addressing how EL is formed . Further, even less research utilizes before-after, treatment-control designs. Blumstein and Saylan  suggest a “bunker mentality” within the environmental community may explain the reluctance to more formally evaluate the drivers of EL. Whatever the cause for limited evaluation, research addressing the drivers of EL is critical . Although many studies examine factors contributing to at least one component of EL, , , , few if any address all four components or evaluate a broad suite of drivers. Efforts to standardize the way programs target and measure progress in all four elements include the National Environmental Literacy Assessment (NELA) project ,  and the 2011 Framework for Environmental Literacy Assessment . The NELA team developed the first major assessment tool for middle school students in the United States and the Framework project established guidelines for use in developing future assessment tools worldwide. Although these latest efforts have made progress in standardizing and measuring EL, no studies we are aware of have controlled for confounding factors including ethnicity, socio-economic background, school quality, and teacher training or addressed the degree to which specific EE efforts are effective in a school setting.
This study begins to fill the gap in EE research with an examination of participation in school-wide EE programs and how time spent in nature relates to EL in North Carolina, USA middle schools. We also accounted for use of published EE curricula; student attributes of gender, age, and ethnicity; teacher attributes of education level and years of experience; and school attributes of economic status, urban status, school type (charter or private vs. traditional public) and student-teacher ratio. We hypothesized that students’ EL scores would be: (1) positively related to participation in school-wide EE programs and use of published EE curricula, (2) positively related to class time spent in nature, (3) positively related to teacher development, measured by both having an advanced degree and years of teaching experience, (4) negatively related to student attributes historically associated with low academic performance (e.g., interest and performance in science wanes with student age , girls score lower than boys in science , and minority students score lower than white students in other academic areas ), and (5) negatively related to student-teacher ratios and enrollment in lower-income schools.
Materials and Methods
The North Carolina State University institutional review board (IRB # 2212) approved this study. All participants provided written informed consent. Students and their parents/guardians were given either a Passive Consent form or an Active Consent form, per the preference of the teacher and/or school. The Passive Consent form was only signed and returned if the parents/students did not want to participate. The Active Consent form was signed and returned to indicate consent to participate in the study.
We targeted 6th and 8th grade students because middle school students are developing cognitive abilities linked to the goals of EL and represent the latest and prime stage for influencing how students engage in society as citizens and environmental decision makers . Additionally, we chose 6th and 8th graders so we could compare our results with the only other large scale assessment of EL available at the time that our study was conducted – Phase 1 of the National Environmental Literacy Assessment Project .
Sampling occurred in three stages: schools, teachers, and classrooms. We followed the three stage sampling model for two groups – an EE group randomly selected from schools registered with the NC Office of Environmental Education as having school-wide EE programs, and a control group randomly selected from 6th and 8th grade science classes in North Carolina that were not registered with the Office of Environmental Education. We generated the first stage of the EE sample from a list of all schools registered with the NC Office of EE (n = 40). We generated the first stage of the control sample from a list of all 665 middle schools in North Carolina. We omitted the 40 schools already included in the EE group and then randomly selected 40 schools for the control group from the remaining 625 schools.
From the two lists of 40 schools, we generated a list of 6th and 8th grade science teachers at each school. This process resulted in 135 teachers in the control group and 95 teachers in the EE group. We randomly selected 85 teachers from each group to recruit for the study and e-mailed each a letter of introduction and a brief recruitment survey. We sent up to four survey reminders to each teacher in 4–7 day increments. Of the 170 teachers contacted, 59 (34.7%) responded, and of these, 21 declined to participate and 38 consented, representing 20 members of the control group and 18 members of the EE group. Two teachers from each group later withdrew from the study. The 64% compliance rate among teachers who we successfully contacted could allow for bias if participating teachers were more environmentally oriented than other teachers. Because evaluating drivers of EL was more important than extrapolating findings to general assessments of EL in NC, this potential bias should not be problematic. Further, we measured teachers’ environmental orientations using the New Ecological Paradigm scale  and their scores were not related to student scores on the MSELS (r = −0.0186, p = 0.614), so if participating teachers were more environmentally oriented than others it should not bias student scores. Entering the pretest, we had 18 teachers participating from the control group (ten 6th grade and eight 8th grade teachers) and 16 teachers participating in the EE group (four 6th grade teachers and twelve 8th grade teachers). Though we contacted equal numbers of teachers in the control and EE groups and in 6th and 8th grades, differential teacher response rates prevented equal classroom numbers. If a teacher taught more than one 6th or 8th grade class, we asked them to flip a coin to randomly select one class for participation.
All students in the study were given the Middle School Environmental Literacy Survey (MSELS) developed by the NELA team , . The MSELS consists of eight sections that contribute to an overall environmental literacy score (Table 1). The MSELS was based on four instruments that reported established high validity and reliability – the Middle School Environmental Literacy Instrument , Children’s Environmental Attitude and Knowledge Scale , the Secondary School Environmental Literacy Assessment Instrument , and the Ecology Attitude Inventory . In the 2008 national survey using the MSELS, the Chronbach alpha coefficients for each component of the MSELS fell between 0.701 and 0.869 with the exception of a three-item Issue Identification scale with an alpha coefficient of 0.389. The total reliability for the MSELS in this 2008 study was 0.82 . Teachers were given a separate survey that asked about their use of EE, the degree to which they take their students outside, and their own training. Key questions included whether they use a published EE curriculum (e.g. Project WET, Project WILD, Project Learning Tree), if they visit natural areas during class time, and their experience in education and EE.
Table 1. Summary of MSELS contents and average pretest scores .doi:10.1371/journal.pone.0059519.t001
In January of 2012, we visited all 34 classrooms. We administered the MSELS to the students and asked each teacher to complete the teacher survey. From April 11 to June 6, 2012, we returned to the same classrooms and administered the post-test (using the MSELS again) to the students. While visiting each classroom for the post-test, we also asked teachers to complete a shorter follow-up survey that asked about the time they spent outdoors with the participating class, their use of EE, and any further EE training they had received since the pretest.
We surveyed 856 students during the pretest and 846 during the post-test. We eliminated students who were absent during either the pretest or the post-test from the pre/post-test comparison yielding 739 students in the comparison. Use of published EE curricula and time spent outdoors were reported by the teachers in the teacher surveys at the time of the pretest and confirmed by the follow-up survey at the time of the post-test. Teacher attributes of higher education and years teaching experience were self-reported as questions in the teacher survey. Student attributes of age, gender, and ethnicity were self-reported as questions within the MSELS. For school attributes of locale, school-level socio-economic status (SES) measured by Title I status, student/teacher ratios, and charter or private school designation, we used data from the National Center for Education Statistics. The Title I program is authorized by the Elementary and Secondary Schools Act to give additional funding to schools with high percentages of low-income students , and Title I status can be used as a measure of school-level SES. Locale includes 12 categories: Large city, midsize city, small city, large suburb, midsize suburb, small suburb, fringe town, distant town, remote town, fringe rural, distant rural, and remote rural areas . We collapsed these categories into urban (including all size cities and suburbs) and rural (including all size towns and rural areas).
We analyzed data using STATA software, version 12.1. Multiple linear regression analysis was used to model component scores for the pretest and the difference between the post-test and pretest as a function of membership in the EE group, spending time outdoors as reported by the teacher at the end of the study, use of published EE curriculum, teacher education level (Master’s degree or higher), years teaching experience, student attributes (age, gender, ethnicity) and school attributes (student/teacher ratio and Title I program). We also controlled for school locale and type of school (private or charter vs. traditional public) in all models. To account for the fact that students within the same classroom also are exposed to the same teacher and school attributes, we included a random effect for class. This approach captures the likelihood that students within the same classroom may have similar EL levels as opposed to independent random deviations of student scores. Additionally, we calculated robust standard errors to account for the possibility of unequal variances between individual students .
For each analysis (pretest and difference in scores), we modeled each component of the MSELS (Knowledge, Affect, Cognitive Skills, and Behavior) as well as Overall EL using the aforementioned independent variables. We calculated each of these scores according to the guidelines listed in the Phase One NELA study . This method weighted the four components in the MSELS to contribute equally to the Overall EL score. When modeling the change in MSELS scores, we also included the pretest score as a predictor to control for the fact that students scoring high in the pretest had less potential for improvement in the post-test.
Modeling both pretest scores and modeling how scores changed between the pretest and post-test was necessary because pretest scores reflected EL levels when students entered the study in January, half way through the school year. Spring testing was conducted to facilitate comparisons with other studies using the MSELS , . This meant that students had already been in their respective classes for several months – with the same teacher, often employing the same curricular strategies including use of environmental and outdoor education. Some variables may predict pretest scores and not change in EL, because the impact of that particular variable was exerted during first semester of the school year. Thus the pretest models allow assessment of variables with relatively rapid impacts whereas the change model allows assessment of the variables that may not have exerted a discernible effect during the first semester.
There were 415 students in the control group (240 6th graders and 175 8th graders) and 324 students in the EE group (70 6th graders and 254 8th graders). Almost half (49.1%) of the students in the study spent some class time outdoors and 42.1% were exposed to published EE curricula. The average student age was 12.7 years, and there were slightly more females (53.1%) than males. The students were primarily white (70.6%) and black (14.6%), with smaller percentages of Hispanic students (8.2%), Asian students (3.4%) and Native American/Alaskan students (3.3%). Half (50.0%) of the teachers in the study held Masters degrees, and the average experience level was 9.2 years teaching. Over half of the students were from schools with Title I programs (56.3%), and from rural communities (66.0%). The average school-wide student-teacher ratio was 14.6, and 26.8% of the students in the study were enrolled in Charter or private schools. Average pretest Cognitive Skills were considerably lower than the other three components measured in the MSELS (Table 1).
EE and Outdoor Education
Our results partially support hypothesis 1 because attendance at EE schools often had either a non-significant or negative relationship with EL, but classroom engagement in published EE curriculum was positively related to EL. Students in the EE schools pretested lower in Affect and Behavior (Table 2) and changed in the same ways as the control group over the course of the semester (Table 3). Further, membership in the EE schools was the most important negative predictor of pretest scores for the Affect component (Table 2). The explanatory power of the Affect and Behavior pretest models was low (R2 = 0.071 and 0.054, respectively), suggesting that although there was a negative relationship between school-wide EE programs and Affect and Behavior scores, important drivers of Affect and Behavior (e.g., behavioral norms, values , parental attributes, presence of a role model for environmental stewardship ) were not accounted for in the models. Although all schools in the EE group were registered with the NC Office of Environmental Education as having EE programs, only four of the teachers in this group indicated that their students participated in a formal EE program. Similarly, none of the teachers in the control group were associated with schools registered with the Office of EE, but three reported their students participated in a formal EE program.
Table 2. Pretest MSELS Scores.doi:10.1371/journal.pone.0059519.t002
Table 3. Difference in MSELS Scores between pretest and posttest.doi:10.1371/journal.pone.0059519.t003
Students exposed to published EE curricula improved more than students who were not, especially with respect to Cognitive Skills. Four teachers in the control group and six teachers in the EE group reported use of a published EE curriculum. The most frequently used EE curriculum was Project WILD (29.1% of students exposed) followed by Project WET (28.2%) and Project Learning Tree (12.8%), although most teachers reported to using multiple curricula and at least 13 curricula were listed. Students whose teachers used at least one published EE curriculum entered the study pretesting higher on Overall EL (Table 2). Students engaged in a published EE curriculum also improved more than other students in the Cognitive Skills component and Overall (Table 3, Figure 1). The use of published EE curricula was the only positive predictor of the change in the Cognitive Skills component measured in this study (Table 3).
Figure 1. Significant predictors of pretest and change in EL.
Variables present represent the significant predictors of pretest and/or change in Overall environmental literacy from the full models represented in Table 2 and Table 3. Pretest scores represent total MSELS score at time of pretest and change in scores represents difference between pretest and post-test scores attributed to each variable independent of the others in the model.doi:10.1371/journal.pone.0059519.g001
Both the pretest and change in EL models support hypothesis 2 by suggesting that taking students outside was positively related to EL. Spending time in nature was positively related to the pretest scores for all components except Cognitive Skills (p = 0.09) and was the most important factor predicting the Affect pretest scores in terms of significance and magnitude (Table 2). Time outdoors was the most important positive predictor of change in the Behavior component along with whether the teacher had a Master’s degree (Table 3).
Our results support hypothesis 3 by suggesting teacher development, primarily through advanced degrees, is associated with improved EL among students. Teachers with Master’s degrees had students who pretested higher in all components of EL except for the Cognitive Skills component (Table 2), and those same students improved more than others in the Behavior component (Table 3). Of the teachers that had Master’s degrees, most (75%) held a degree in education or science education. Advanced degrees for teachers had the most important positive relationship with the pretest of Knowledge in terms of magnitude and significance (Table 2). The relationships between student EL and teacher experience were mixed. Teachers with 3–5 years of teaching experience had students that outperformed those with fewer than 3 years of teaching experience in the pretest of Cognitive Skills and Overall EL (Table 2, Figure 1). Teachers with 6 or more years of experience had students that pretested lower in Behavior than teachers with fewer than three years of experience (Table 2). Students with teachers of 6–8 years teaching experience also did not improve as much in Affect as those with teachers with fewer than 3 years of experience.
Our results support hypothesis 4 in terms of ethnicity and age but are mixed with regards to student gender. Girls pretested lower than boys in Knowledge but higher in Affect and Cognitive Skills (Table 2), although they improved more than boys in Knowledge (Table 3). Older students performed the same as younger students in the pretest (Table 2) and improved less than younger ones in Behavior and Overall scores (Table 3, Figure 1). Minority students fell behind in several components of EL. Native American students pretested lower than white students in the Knowledge and Cognitive Skills components as well as Overall (Table 2, Figure 1) and improved more slowly than white students in Cognitive Skills during the semester (Table 3). Hispanic students followed the same pretest patterns as Native American students (Table 2, Figure 1), but improved as much as white students over the course of this study (Table 3). Black students entered the study behind white students in the same areas as Native American and Hispanic students (Table 2, Figure 1), however, they improved more slowly than white students in all three of these areas (Table 3). Students self-identifying as black compared to white was the most important negative predictor in Knowledge, Cognitive Skills, and Overall EL in terms of significance and magnitude (Table 3, Figure 1).
Because we saw pronounced differences in EL levels related to ethnicity (Table 2, Table 3), we also tested interaction terms between ethnicity and time outdoors. Though the full models were minimally impacted, several of the interaction terms between time outdoors and Hispanic ethnicity were significant. Specifically, these interaction terms were significant in the pretest of Affect (Beta = 3.639, p = 0.012) and Behavior (Beta = 4.893, p = 0.016). Additionally, the interaction term between time outdoors and black ethnicity approached significance at the alpha = 0.05 level (Beta = 1.412, p = 0.099) for the Behavior pretest. These findings suggest that time outdoors may impact some components of EL more for Hispanic and black students than for white students. The main effect of time outdoors approached significance (Beta = 0.667, p = 0.058) in the pretest Affect model with the inclusion of the interaction terms, suggesting that although the relationship is stronger among minority students, Affect scores are still positively associated with time outdoors among white students. The time outdoors main effect was not significant in the pretest Behavior scores, suggesting that time outdoors is positively related to Behavior scores among Hispanic students but not significantly among white students. None of the interaction terms were significant for the change in EL models.
The type of school students attended was related to some areas of EL, supporting hypothesis 5. In the pretest, schools with a higher student/teacher ratio were behind in Knowledge, Title I schools were behind in Behavior, and Charter and Private schools outperformed public schools in Behavior (Table 2). None of these school attributes were significantly related to change in EL over the semester (Table 3). Urban and rural schools performed the same in the pretest, although students in urban schools improved more slowly than those in rural schools over the semester in Cognitive Skills (Table 3). Overall, school attributes had the weakest relationships with EL (Table 3, Figure 1).
EE and Outdoor Education
The surprising relationship, or lack thereof, between EL and school-wide EE programs (hypothesis 1) may be explained by how EE programs are defined and operationalized. The EE schools were drawn from a list of school-wide EE programs maintained by the NC Office of Environmental Education. Although there are specific qualifications listed on the website, teachers can register their own school based on a personal assessment of their school’s EE program . The low percentage of teachers in EE schools as well as similar percentage of teachers in control schools that stated their students participated in an EE program during the study suggests attendance at an EE school had limited impact on whether a given student was actually exposed to EE programming. Further, teachers may enroll their schools because they perceive deficiencies in EE programming. This may explain why we found a negative association between membership in the EE schools and pretest Affect and Behavior scores. Ultimately, we suspect our findings related to attendance at EE schools have less to do with the efficacy of school-wide EE programs than with how those programs are defined. School-wide EE efforts may in fact be highly effective at building EL and other skills , , but our results suggest effective evaluations of school-wide EE programs will need to account for what is happening in individual classrooms.
Although there may be some confusion about what constitutes a school-wide EE program, the use of published EE curricula in classrooms is less ambiguous, and our results suggest their use may improve EL. Use of EE curricula was the only variable that was significantly linked to both higher pretest scores and improved Overall EL over the semester. Most of this impact was in the Cognitive Skills component, which is not surprising as guidelines for K-12 EE curricula emphasize ecological knowledge and awareness as well as cognitive skill building . These skills focus on identifying and analyzing complex issues as well as action planning and forming solutions. While many EE programs are content-specific to wetlands (e.g., Project WET), wildlife (e.g., Project WILD), or forests (e.g., Project Learning Tree) , the fostering of Cognitive Skills in all of these published EE curricula equip students to engage in and respond to more complex issues including climate change, biodiversity loss, and water quality problems. Studies through the State Education and Environment Roundtable suggest Cognitive Skills built through EL improve academic achievement in reading, writing, math, science, and social studies test scores . Accordingly, use of published EE curricula may be an important tool for improving student achievement in key academic areas beyond EL.
Time spent outdoors complements the use of published EE curriculum by addressing all components of EE other than Cognitive Skills. The relationship with the Affect pretest is not surprising as time outdoors has been linked to improvement in environmental attitudes and intentions . Although time outdoors did not predict a change in Affect, time students spent outside in the fall may have already impacted Affect scores. We did, however, see that time outdoors was associated with improvement in Behavior scores. As improvement in attitudes is generally linked to pro-environmental behavior , , our data may show that time outdoors fosters pro-environmental attitudes and higher levels of environmental sensitivity which over time leads to more environmentally friendly behavior. Time outdoors also correlated with higher pretest scores in the Knowledge component. Time outdoors can improve student attention in children  and is linked to elevated creativity and improved problem solving skills in adults , which may explain why contact with nature may improve the more purely academic areas of EL (i.e., Knowledge). Because time outdoors was positively correlated with all areas except Cognitive Skills, use of published EE curriculum and time outdoors together represent a potentially powerful strategy for increasing EL. Our results add to the growing chorus of support for taking kids outside by suggesting outdoor education can promote EL in addition to social , mental , and physical health .
Teacher development may be more important than curriculum in terms of predicting EL and may influence EL faster than curriculum. Further, the benefits of teacher development (having a Master’s and 3–5 years of experience) appear to have been largely expressed in the pretest after only one semester of the course. The recent focus on teacher effectiveness in education reform has resulted in numerous studies on how teacher development affects student performance, and the results are mixed. Advanced degrees in education seem to have little or inconsistent effect on overall student performance , although subject-specific degrees (e.g. a MS in Biology for science teachers) do seem to have significant impact in Math and Science . As most (75%) of the advanced degrees among the teachers in our study are in education or science education, it appears that our results may conflict with those suggesting training in education does not impact student learning . Our results suggest that a strong background in pedagogical theory and technique is associated with improved student EL.
Interestingly, the relationship between teacher experience and student EL was non-linear. Education literature suggests teacher effectiveness, measured by student achievement, increases rapidly in the first three years of teaching, afterward plateauing for teachers who stay in the profession at least five years , . This trend is either attributed to the benefits of on-the-job training provided for early career teachers or the possibility that less-effective teachers leave the profession quickly while more-effective teachers continue teaching past the first few years . Our results add to this research by suggesting that the relationship between teaching experience and student achievement in EL plateaus (e.g., for Cognitive Skills scores in the pretest) after reaching a threshold around 5 years. Specifically, moderately experienced teachers (3–5 years of experience) seem to be more effective at fostering EL, particularly Cognitive Skills, than new teachers, but this effect is not present when considering teachers with more than five years of experience. As this finding parallels research related to teacher experience and other areas of student achievement, perhaps the similar factors of on-the-job training and attrition affect EL. With respect to the pretest of Behavior and change in Affect scores, however, teachers with more than 5 years of experience had students that underperformed those with new teachers. One explanation may be related to loss of idealism and increasing burnout associated with more years of teaching experience . Several studies have suggested inadequate administrative support poses a barrier to inclusion of EE  and may contribute to waning enthusiasm and commitment to EE among teachers. Teachers can become increasingly discouraged when not given support to expand use of EE through training or curriculum development . Efforts to promote teacher enthusiasm among veteran EE teachers may be just as important as similar efforts in other disciplines , despite the enthusiasm demonstrated by many EE teachers.
Our results suggest student attributes have strong relationships with EL, and challenges associated with ethnicity and early education identified in general education research apply to EE efforts as well. Age was negatively related to the change in Behavior and Overall EL, but had no relationship to the pretest in EL, suggesting that while middle school students have similar EL levels among grade levels, older students are slowing down in terms of improvement. This trend is mirrored in research that suggests early education has more bearing on student achievement and even certain measures of life achievement than education efforts with older students . Additionally, older students tend to wane in their interest in science and math in the middle school years , which could also explain the slower rate of EL improvement among eighth graders. These findings suggest middle school grades may include an age related tipping point where EE efforts start becoming less effective in promoting EL. We are not arguing that older students cannot benefit from these efforts; rather younger students may have the greatest capacity for learning. Further, the most readily available EE curricula may be more effective for 6th grade students than 8th grade students.
Gender was related to EL in complex ways. Although girls underperformed boys in the pretest for Knowledge, they outperformed them in Affect and Cognitive Skills and improved faster in Knowledge over the course of the semester. The gap in the Knowledge pretest is supported by similar gender trends in science . However, as girls entered the study with more pro-environmental attitudes and higher levels of Cognitive Skills, they were perhaps better positioned for improvement in Knowledge. Girls often underperform boys in the sciences , but numerous studies have shown that women and girls hold more positive environmental attitudes and greater levels of concern for the environment , . Although overall EL did not differ based on gender, the internal differences associated with Knowledge, Affect, and Cognitive Skills suggest routes to more effective teaching. For example, teachers could use the differing strengths associated with gender to facilitate boys sharing their higher levels of Knowledge with girls while encouraging girls to express their pro-environmental attitudes and utilize their Cognitive Skills to help boys put their Knowledge to use. This strategy could be particularly effective in problem-based group work simulating environmental decision-making.
Ethnicity related differences in EL seem to mirror general education trends; however, the implications may be more complex in EL contexts. Much attention has been paid to achievement gaps between minority students and white students in education literature, and most studies find that individual and school level socio-economic status (SES) are confounded with ethnicity –. These scholars suggest achievement differences are more an issue of poverty than culture. We controlled for school level SES (Title I status), but the differences in EL scores predicted by ethnicity may be at least partially explained by individual SES data. Other explanations for these achievement gaps range from how views of schooling fit into cultural narratives  to how teacher expectation bias affects student performance .
Although it is possible that achievement gaps in EL are rooted in the same causes as those in other academic areas, differences in cultural perception of the outdoors and access to natural areas may also come into play. Minority groups experience more constraints to natural area access and can be culturally excluded from outdoor recreation , . In considering outdoor recreation, safety is of particular concern to some minority groups, including blacks and Hispanics, which may lead to minority children spending less time outdoors than their white counterparts . Additionally, a disproportionately high exposure to environmental risk and decreased availability of natural areas among minority children offers another explanation for the disparities in EL shown in our results . If minority students are exposed to the outdoors less than white students, it would follow that in-school outdoor experiences could have more impact on EL among minority students. Outdoor experiences and contact with nature can be particularly effective in closing gaps in environmental attitudes and awareness associated with ethnicity . The interaction between time outdoors and Hispanic and black students suggest that time outdoors was particularly important in predicting the pretest of Affect and Behavior components among Hispanics and the pretest of Affect among black students. Exposure to nature could help mitigate EL gaps associated with ethnicity, at least among Hispanic and black students. This relationship could be a fruitful area of future study, including the amount, type, or quality of outdoor experience and its impact on EL levels of minority students.
School characteristics were related to EL in somewhat expected ways. Lower socio-economic status (SES) is generally associated with lower academic achievement , but we did not detect this relationship for any dimension of EL except Behavior. Generally, income has been positively associated with environmental behavior , . Schools with Title I programs have a higher proportion of lower-income students who, according to these studies, may be less inclined toward environmental concern and behavior. However, more recent literature shows that the link between income and environmental concern is fading and is context dependent , , which may explain why we found no difference in most EL components associated with SES. In addition to lower SES, bigger class sizes are generally associated with lower academic performance , . Our results support this research with a weak negative association between student-teacher ratios and the Knowledge pretest. The relatively low importance of school attributes is encouraging, because curriculum and teacher development are more easily changed than poverty, school types (e.g. charter vs. traditional) or locales.
Achieving EL through K–12 education is a critical step to creating a public equipped to meet and solve environmental challenges. This study highlights key ways to ensure investments in EE promote EL and ultimately lead to a sustainable future. First, we suggest using published EE curricula and time outdoors in tandem because taken together they foster all four components of EL among middle school students. Published EE curricula including Project WILD, Project WET, and Project Learning Tree were particularly effective at building Cognitive Skills. Future research should address whether other curricular innovations (e.g., service learning) could better engage older students. Time outdoors is one of the only factors that significantly impacts Knowledge, Affect, and Behavior. School-wide EE programs will fail to achieve EL gains unless these programs include tangible changes to curricula in classrooms. Second, despite the informal and volunteer nature of many EE efforts, advanced degrees and years teaching experience are as important in EE as they are in other academic disciplines. EE efforts may improve if advanced degrees among teachers are promoted and teachers are retained for longer periods. The stabilization or even decline in teacher effectiveness after 5 years of experience highlights the need for training, administrative, and structural support for teachers that maintains their enthusiasm and commitment to EE over the long run. Gender based differences in EL appear to complement one another (with each gender excelling in areas where the other does not), suggesting teachers have synergistic opportunities to raise EL levels among both boys and girls. The relationship between ethnicity and EL reveal that although time outdoors may be effective for all students, it may be especially effective for engaging black and Hispanic students.
We wish to thank members of the North Carolina Environmental Literacy Plan working group for valuable feedback and logistical support. Especially, we would like to thank Renee Strnad Dr. Laurell Malone, Lisa Tolley, and Marty Wiggins.
Secured funding: MNP KTS SEM. Conceived and designed the experiments: KTS MNP HDB AGM SEM. Performed the experiments: KTS MNP. Analyzed the data: KTS MNP HDB. Contributed reagents/materials/analysis tools: KTS MNP HDB. Wrote the paper: KTS MNP HDB AGM SEM.
- 1. Convention on Biological Diversity (2010) Global Biodiversity Outlook 3. Montreal, Canada. Global Biodiversity 3. Available: http://gbo3.cbd.int/. Accessed 2013 Feb 25.
- 2. Schreurs MA (2012) Breaking the impasse in the international climate negotiations: The potential of green technologies. Energy Policy 48: 5–12 doi:10.1016/j.enpol.2012.04.044.
- 3. Agardy T (1994) Advances in marine conservation: The role of marine protected areas. Trends Ecol Evol 9: 267–270. doi: 10.1016/0169-5347(94)90297-6
- 4. Brezonik P, Cooper W (1994) Reauthorization of the Clean Water Act: Important Issues for Water Quality Scientists. J Contemp Water Res Educ 94: 47–51.
- 5. Liu J, Li S, Ouyang Z, Tam C, Chen X (2008) Ecological and socioeconomic effects of China’s policies for ecosystem services. Proc Natl Acad Sci USA 105: 9477–9482. doi: 10.1073/pnas.0706436105
- 6. Hollweg KS, Taylor J, Bybee RW, Marcinkowski TJ, McBeth WC, et al.. (2011) Developing a Framework for Assessing Environmental Literacy. Washington, DC. Available: http://www.naaee.net/framework. Accessed 2013 Feb 25.
- 7. McBeth B, Hungerford H, Marcinkowski T, Volk T, Meyers R (2008) National Environmental Literacy Assessment Project: Year 1, National Baseline Study of Middle Grades Students. Final Report. Available: http://www.epa.gov/education/pdf/MASTERNELA_Year1Report_081208_.pdf. Accessed 2013 Feb 25.
- 8. Bickford D, Posa MRC, Qie L, Campos-Arceiz A, Kudavidanage EP (2012) Science communication for biodiversity conservation. Bio Cons 151: 74–76. doi: 10.1016/j.biocon.2011.12.016
- 9. The Pew Research Center for the People & the Press (2011) More Moderate Republicans See Evidence of Warming. Washington, DC.
- 10. Tobler C, Visschers VHM, Siegrist M (2012) Consumers’ knowledge about climate change. Clim Change 114: 189–209 doi:10.1007/s10584-011-0393-1.
- 11. Hines JM, Hungerford HR, Tomera AN (1987) Analysis and Synthesis of Research on Responsible Environmental Behavor: a Meta-Analysis. J Environ Educ 18: 1–8. doi: 10.1080/00958964.1987.9943482
- 12. Hungerford HR, Volk T (1990) Changing Learner Behavior through Environmental Education. J Environ Educ 21: 8–21. doi: 10.1080/00958964.1990.10753743
- 13. Rickinson M (2001) Learners and Learning in Environmental Education: a critical review of the evidence. J Environ Educ 7: 207–320 doi:10.1080/1350462012006523.
- 14. Cheng JC-H, Monroe MC (2010) Connection to Nature: Children’s Affective Attitude Toward Nature. Environ Behav 44: 31–49 doi:10.1177/0013916510385082.
- 15. Duerden MD, Witt PA (2010) The impact of direct and indirect experiences on the development of environmental knowledge, attitudes, and behavior. J Enviro Psychol 30: 379–392 doi:10.1016/j.jenvp.2010.03.007.
- 16. Skelly SM, Zajicek JM (1998) The Effect of an Interdisciplinary Garden Program on the Environmental Attitudes of Elementary School Students. Horttechnology 8: 579–583.
- 17. Karsh K, Bush E, Hinson J, Blanchard P (2009) Integrating Horticulture Biology and Environmental Coastal Issues into the Middle School Science Curriculum. Horttechnology 19: 813–817.
- 18. Palmer J (1998) Environmental Education in the 21st century: Theory, practice, progress and promise. Routledge.
- 19. H.R. 2547 (2011) No Child Left Inside Act: 1–25.
- 20. Keene M, Blumstein DT (2010) Environmental education: a time of change, a time for change. Evaluation and program planning 33: 201–204 doi:10.1016/j.evalprogplan.2009.07.014.
- 21. Blumstein DT, Saylan C (2007) The failure of environmental education (and how we can fix it). PLoS Biol 5: e120 doi:10.1371/journal.pbio.0050120.
- 22. Palmer JA (1999) Research Matters: a call for the application of empiracal evidence to the task of improving the quality and impact of environmental education. Camb J Educ 29: 379–395. doi: 10.1080/0305764990290308
- 23. Lisowski M, Disinger JF (1991) The Effect of Field-Based Instruction on Student Understandings of Ecological Concepts. J Environ Educ 23: 19–23. doi: 10.1080/00958964.1991.9943065
- 24. Bradley JC, Waliczek TM, Zajicek JM (1999) Relationship Between Environmental Knowledge and Environmental Attitude of High School Students. J Environ Educ 30: 17–21. doi: 10.1080/00958969909601873
- 25. Mcbeth B, Hungerford H, Marcinkowski T, Volk T, Cifranick K (2011) National Environmental Literacy Assessment, Phase Two: Measuring the Effectiveness of North American Environmental Education Programs with Respect to the Parameters of Environmental Literacy. Final Report. Available: http://www.naaee.net/sites/default/files/programs/research/NELA_Phase_II_Report.pdf. Accessed 2013 Feb 25.
- 26. Fouad NA, Smith PL (1996) A test of a social cognitive model for middle school students: Math and science. J Couns Psychol 43: 338–346 doi:10.1037//0022-022.214.171.1248.
- 27. Neathery MF (1997) Elementary and secondary students’ perceptions toward science: correlations with gender, ethnicity, ability, grade, and science achievement. Electron J Sci Educ. Available: http://ejse.southwestern.edu/article/view/7573/5340. Accessed 2013 Feb 25.
- 28. Osborne JW (2001) Testing Stereotype Threat: Does Anxiety Explain Race and Sex Differences in Achievement? Contemp Educ Psychol 26: 291–310 doi:10.1006/ceps.2000.1052.
- 29. Dunlap RE, Liere KD Van, Mertig AG, Jones RE (2000) Measuring Endorsement of the New Ecological Paradigm: A Revised NEP Scale. J Soc Issues 56: 425–442. doi: 10.1111/0022-4537.00176
- 30. Bluhm WJ, Hungerford HR, McBeth WC, Volk TL (1995) A final report on development and pilot-testing of the “Middle School Environmental Literacy Instrument.” In: Wilke R, editor. Environmental Literacy/Needs Assessment Project: Final Report. Stevens Point, WI: University of Wisconsin, Stevens Point.
- 31. Leeming FC, Dwyer WO, Bracken BA (1995) Children’s environmental attitude and knowledge scale: Construction and validation. J Environ Educ 26: 22–31. doi: 10.1080/00958964.1995.9941442
- 32. Marcinkowski TJ, Rehrig L (n.d.) The Secondary School Report: A Final Report on the Development, Pilot Testing, Validation, and field Testing of the Secondary School Environmental Literacy Assessment Instrument. In: Wilke R, editor. Environmental Education Literacy/Needs Assessment Project: Assessing Environmental Literacy of Students and Environmental Education Needs of Teachers; Final Report for 1993–1995. Stevens Point, WI: University of Wisconsin, Stevens Point. 30–70.
- 33. Maloney M, Ward M, Braucht G (1975) A revised scale for the measurement of ecological attitudes and knowledge. American Psychologist 30: 787–790. doi: 10.1037/h0084394
- 34. th Congress (2002) No Child Left Behind (NCLB) Act: 1–670.
- 35. National Center for Education Statistics (2006) Identification of Rural Locales. Available: http://nces.ed.gov/ccd/rural_locales.asp. Accessed 23 October 2012.
- 36. Stock JH, Watson MW (2007) Introduction to Econometrics. 2nd ed. Boston: Person Eduction, Inc.
- 37. Stern PC (2000) Toward a Coherent Theory of Environmentally Significant Behavior. J Soc Issues 56: 407–424. doi: 10.1111/0022-4537.00175
- 38. Chawla L (1999) Life Paths into Effective Environmental Action. J Environ Educ 31: 15–26. doi: 10.1080/00958969909598628
- 39. NC Office of Environmental Education (2011) EE School Guidelines. Available: http://web.eenorthcarolina.org/net/org/add.aspx?tid=38000&glid=54764&s=0.0.0.37430. Accessed 30 October 2012.
- 40. Ernst J (2007) Factors Associated With K-12 Teachers’ Use of Environment-Based Education. J Environ Educ 38: 15–32 doi:10.3200/JOEE.38.3.15-32.
- 41. State Education and Environment Roundtable (SEER) (2000) The Effects of Environment-based Education on Student Achievement. Available: http://www.seer.org/pages/research/educeff.pdf. Accessed 2013 Feb 25.
- 42. North American Association for Environmental Education (2004) Environmental Education Materials: Guidelines for Excellence. Washington, DC.
- 43. Eick CJ, Carrier S, Perez K, Keasal DE (2010) Summer Methods in Summer Camps: Teaching Projects WILD, WET, and Learning Tree at an Outdoor Environmental Education Center. In: Bodzin AM, Shiner Klein B, Weaver S, editors. The Inclusion of Environmental Education in Science Teacher Education. Dordrecht: Springer Netherlands. 173–189. doi:10.1007/978-90-481-9222-9.
- 44. Cheng JC-H, Monroe MC (2010) Connection to Nature: Children’s Affective Attitude Toward Nature. Environ Behav 44: 31–49 doi:10.1177/0013916510385082.
- 45. Guagnano GA, Stern PC, Dietz T (1995) Influences on Attitude-Behavior Relationships: A Natural Experiment with Curbside Recycling. Environ Behav 27: 699–718. doi: 10.1177/0013916595275005
- 46. Poortinga W, Steg L, Vlek C (2004) Values, Environmental Concern, and Environmental Behavior: A Study into Household Energy Use. Environ Behav 36: 70–93. doi: 10.1177/0013916503251466
- 47. Taylor AF, Kuo FE (2009) Children with attention deficits concentrate better after walk in the park. J Atten Disord 12: 402–409 doi:10.1177/1087054708323000.
- 48. Atchley RA, Strayer DL, Atchley P (2012) Creativity in the Wild: Improving Creative Reasoning through Immersion in Natural Settings. PLoS ONE 7: e51474. Available: http://dx.plos.org/10.1371/journal.pone.0051474. Accessed 13 December 2012.
- 49. Burdette HL, Whitaker RC (2006) Resurrecting Free Play in Young Children: Looking Beyond Fitness and Fatness to Attention, Affiliation, and Affect. Arch Pediatr 159: 46–50. doi: 10.1001/archpedi.159.1.46
- 50. Wells NM, Evans GW (2003) Nearby Nature: A Buffer of Life Stress Among Rural Children. Environ Behav 35: 311–330 doi:10.1177/0013916503251445.
- 51. Potwarka LR, Kaczynski AT, Flack AL (2008) Places to Play: Association of Park Space and Facilities with Healthy Weight Status among Children. J Community Health. doi:10.1007/s10900-008-9104-x.
- 52. Goldharber D (2002) The Mystery of Good Teaching. Education Next 2: 50–55.
- 53. Goldhaber DD, Brewer DJ (1996) Evaluating the Effect of Teacher Degree Level on Educational Performance. Dev Sch Finance: 197–210.
- 54. Boyd D, Grossman P, Lankford H, Loeb S, Wyckoff J (2008) Teacher Attrition and Student Achievement. 2009. Available: http://www.urban.org/UploadedPDF/1001270_teacher_attrition.pdf. Accessed 2013 Feb 25.
- 55. Clotfelter CT, Ladd HF, Vigdor JL (2007) Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Econ Educ Rev 26: 673–682 doi:10.1016/j.econedurev.2007.10.002.
- 56. Henry GT, Bastian KC, Fortner CK (2011) Stayers and Leavers: Early-Career Teacher Effectiveness and Attrition. Educ Res 40: 271–280 doi:10.3102/0013189X11419042.
- 57. Schamer L, Jackson M (1996) Coping with Stress: Common Sense about Teacher Burnout. Education Canada 36: 28–31.
- 58. Ernst J (2009) Influences on US middle school teachers’ use of environment‐based education. J Environ Educ 15: 71–92 doi:10.1080/13504620802710599.
- 59. May TS (2000) Elements of Success in Environmental Education Through Practitioner Eyes. J Environ Educ 31: 4–11. doi: 10.1080/00958960009598639
- 60. Rosenholtz SJ (2008) Effective Schools: Interpreting the Evidence. Am J Educ 93: 352–388. doi: 10.1086/443805
- 61. Gorey KM (2001) Early Childhood Education: A Meta-Analytic Affirmation of the Short- and Long-Term Benefits of Educational Opportunity. Sch Psychol Q 16: 9–30. doi: 10.1521/scpq.126.96.36.19963
- 62. Ma X (2008) Within-School Gender Gaps in Reading, Mathematics, and Science Literacy. Comp Educ Rev 52: 437–460. doi: 10.1086/588762
- 63. Zelezny LC, Chua P-P, Aldrich C (2000) Elaborating on Gender Differences in Environmentalism. J Soc Issues 56: 443–457. doi: 10.1111/0022-4537.00177
- 64. Stern PC, Dietz T, Kalof L (1993) Value Orientations, Gender, and Environmental Concern. Environ Behav 25: 322–348. Available: http://eab.sagepub.com/cgi/doi/10.1177/0013916593255002. Accessed 30 March 2012.
- 65. Kao G, Thompson JS (2003) Racial and Ethnic Stratification in Educational Achievement and Attainment. Annu Rev Sociol 29: 417–442 doi:10.1146/annurev.soc.29.010202.100019.
- 66. Freyer Jr RG, Levitt SD (2004) Understanding the Black-White Test Score Gap in the First Two Years of School. Rev Econ Stat 86: 447–464. doi: 10.1162/003465304323031049
- 67. Alexander KL, Entwisle DR, Olson LS (2001) Schools, Achievement, and Inequality: A Seasonal Perspective. Educ Eval Policy Anal 23: 171–191. doi: 10.3102/01623737023002171
- 68. Ogbu JU, Simons HD (1998) Voluntary and Involuntary Minorities: A Cultural-Ecological Theory of School Performance with Some Implications for Education. Anthropol Educ Q 29: 155–188 doi:10.1525/aeq.19188.8.131.52.
- 69. De Boer H, Bosker RJ, Van der Werf MPC (2010) Sustainability of teacher expectation bias effects on long-term student performance. J Educ Psychol 102: 168–179 doi:10.1037/a0017289.
- 70. Shores KA, Scott D, Floyd MF (2007) Constraints to Outdoor Recreation: A Multiple Hierarchy Stratification Perspective. Leis Sci 29: 227–246 doi:10.1080/01490400701257948.
- 71. Finney C (2005) Black Faces, White Spaces: African-Americans and the Great Outdoors. Berkeley, CA.
- 72. Strife S, Downey L (2009) Childhood Development and Access to Nature. Organ Environ 22: 99–122 doi:10.1177/1086026609333340.
- 73. Larson LR, Castleberry SB, Green GT (2010) Effects of an Environmental Education Program on the Environmental Orientations of Children from Different Gender, Age, and Ethnic Groups. J Park Recreat Admi 28: 95–113.
- 74. Straughan RD, Roberts JA (1999) Environmental segmentation alternatives: a look at green consumer behavior in the new millennium. J Consum Mark 16: 558–575 doi:10.1108/07363769910297506.
- 75. Gamba RJ, Oskamp S (1994) Factors Influencing Community Residents’ Participation in Commingled Curbside Recycling Programs. Environ Behav 26: 587–612 doi:10.1177/0013916594265001.
- 76. Samdahl DM, Robertson R (1989) Social Determinants of Environmental Concern: Specification and Test of the Model. Environ Behav 21: 57–81 doi:10.1177/0013916589211004.
- 77. Mainieri T, Barnett EG, Trisha R, Unipan JB, Oskamp S (1997) Green Buying: The Influence of Environmental Concern on Consumer Behavior. J Soc Psychol 132: 189–204 doi:10.1080/00224549709595430.
- 78. Darling-Hammond L (1999) Teacher Quality and Student Achievement: A Review of State Policy Evidence. Seattle, WA.
- 79. Schwarts RM, Schmitt MC, Lose MK (2012) Effects of Teacher-Student Ratio in Response to Intervention Approaches. Elem Sch J 112: 547–567. doi: 10.1086/664490