Learning Physical Science through Astronomy Activities: A Comparison between Constructivist and Traditional Approaches in Grades 3-6
September 2, 2014, 1:51:16
Ward, R. B., Sadler, P., Shapiro, I.,, 2008, Astronomy Education Review, 6, 2, 1-19
Evaluation of Project ARIES - an astronomy-based physical science curriculum for upper elementary and middle school children. The project uses simple, affordable materials to carry out hands-on activities, and includes comprehensive teacher support materials and student journals. The ARIES team hypothesize that when students discover for themselves through exploratory activities that their predictions do not match their findings, they are more open to reassessing and changing their personal beliefs. Approximately 750 Grade 3-6 intervention students are compared pre- and post-intervention with approximately 650 Grade 4-6 control students. Hypotheses were: 1) students in classes using ARIES materials will significantly increase their understanding of concepts related to time, astronomy, light and color; 2) students in classes using ARIES materials will increase understanding of these concepts significantly more than students in control classes not using ARIES materials. A detailed analysis identifies concepts where Project ARIES was more effective and equally effective in producing student gains than the control, and those where neither was effective. Project ARIES students were found to achieve approximately four times the gain of the control group, and the gains were greater for concepts that the control students found most difficult. Project ARIES, an astronomy-based physical science curriculum for grades 3-8, uses innovative, simple, and affordable apparatus to carry out hands-on, discovery-based activities. 750 grade 3-6 students in ARIES classrooms are compared with 650 grade 4-6 students in control classrooms through a series of open-ended assessment measures, using a pretest and posttest format. We identify concepts where the ARIES approach is more effective, where both are equally effective, and where neither results in much learning (ARIES was never less effective). Although learning is in evidence for both control and treatment groups, overall, the ARIES students achieve roughly four times the gain of their control counterparts. In particular, ARIES students had much greater gains for the concepts that the control students found most difficult.
Teachers in 15 treatment classrooms with a total of 750 students integrated ARIES modules into their classroom curricula at various times during the academic year. The classrooms were Grades 3-6 level, and in urban (4), suburban (6) And rural (2) schools. Each of the volunteer ARIES teachers had participated in at least one course run by the authors; 8 had participated in field-testing of the ARIES materials. In most cases the teachers used only 1 ARIES module, and in some cases did not complete the module. Evaluators observed ARIES classrooms or interviewed teachers, and confirmed that the implementation was faithful to the intended ARIES pedagogy. Students were tested pre- and post-module.
The 16 control classrooms (Grades 4-6, urban (2), large suburban (1), small suburban (1)) were all in communities with at least one ARIES classroom in the same community, and six of the control classrooms were in the same building as the ARIES classroom. A total of 650 students in the control groups studied the same concepts as the ARIES students but pretests were carried out in the Fall and post-tests in the Spring to be sure that they had studied the same concepts as the ARIES students. It's worth noting, therefore, that the post-test of the control group may have taken place considerably longer post-instruction than for the ARIES group. Also worth noting is that all of the control schools had minority populations > 50%, compared to only 6 of the 15 ARIES schools.
Pre- and post-tests for the ARIES groups were carried out before and after each ARIES module. For the control groups, pre- and post-tests were carried out at the beginning and end of the school year. It's worth noting, therefore, that the post-test of the control group may have taken place considerably longer post-instruction than for the ARIES group. The instruments for the control group were developed in collaboration with the control group teachers, who indicated the topics and concepts that they would be covering throughout the year. The authors then constructed a 17-item "sampler test" using questions from the ARIES tests for each module. This sampler test was administered to the control group as both the pre- and post-test. All test items were open-ended, constructed-answer questions allowing students to show how they arrived at their answers. Tests were scored according to a rubric developed by the evaluators, and cross-checked for interrater reliability. Reliability between scorers is reported as <0.90. Gains were compared using a paired-comparison t-test to check whether gains in the treatment vs control groups were significant. Statistical significance is reported at the 0.05, 0.01 and 0.001 levels (i.e. approx 1, 2 and 3 sigma). Effect sizes are reported in standard deviations.
The evaluators contacted 25 teachers who had taken part in regional ARIES institutes and asked them to take part in the study as the treatment group. 15 agreed. The recruitment of the control classrooms is not reported.
The rate of completed pre- and post-tests from the ARIES teachers varies for each item, as not all students completed all test items, and not all teachers completed the ARIES module they started. The highest rate is 40% valid responses, the lowest is 6.6%. For the control group the response rate for all items was 78%.
For the ARIES students, gains made were significant for all but one item tested (#12 - objects emitting light on their own). For the control students, gains were significant for all but three items (#5, flashlight shadows; #9, Earth terminator, and #11, objects in a dark room). The effect size of using the ARIES materials compared to the control group was found to be statistically significant in 15 out of 17 of the measures tested - i.e. the mean gains made by the ARIES students in 15 items were significantly greater than the mean gains made by the control students. Averaging across all items, the ARIES students displayed a mean gain over 4 times greater than the mean gain of the control students. ARIES students showed much greater gains on the more difficult items (identified by low mean re-test scores) than the easy ones. When interpreting these results, be aware of the notes raised in the response rate, and the control or comparison condition sections of this summary. In-depth findings on each of the items measured are explored in depth in the paper, but the following are the main points: Looking at Day and Night, the results suggest that despite modeling the phenomenon in the lab, and investigating the changes in hours of daylight per season, many of the children in both the ARIES and control groups did not incorporate a spinning Earth into their mental models. The study also finds that students consistently had difficulty identifying sources of light, with many believing that the eye gives off light, and with explaining how prisms work - many students continued to believe that a prism adds colour to white light. The authors note the importance placed on understanding shadows in the ARIES modules, and the particularly high gains made in the items related to shadows. They find that the hands-on modules are likely particularly useful in addressing misconceptions about shadows, which students tend to think of as a fixed object rather than a blocking of light, and whether objects are visible in complete darkness. The authors note, however, that all students (control and ARIES) did poorly when explaining how to use the Earth's shadow to tell time, even after making sundials. They postulate that the cognitive ability needed to transfer shadow patterns to the more abstract concept of using them to tell the time may be too advanced for Grade 3-6 students, and may need to be taught later. The authors also find that seasons are a difficult topic for this age group, noting that while the ARIES students did significantly better than the control students, the full reasons for the seasons are not taught until Grades 7-8, which is likely where they belong.
The authors find that the ARIES discovery-based curriculum yields increased learning gains (up to 4 times) for Grade 3-6 students compared to the Grade 4-6 control group. The difference is particularly large for the more difficult items tested. The authors note that there were many variables that they were unable to control, including student demographics, self-selection of the ARIES teachers, teacher qualification, experience, PD and background, school-based factors, curriculum in the control classrooms, and the timing and total time spent on each of the concepts explored. They nevertheless believe that these are unlikely to have a big enough effect to completely wipe out the disparity in gains between the ARIES and control groups. The study finds that the ARIES materials have little to no effect on learning gains compared to the control group for three concepts: the cause of day & night, how a flashlight and mirror interact to reflect light, and how a prism splits light. They do not recommend replacing existing classroom materials for these concepts. The do recommend adopting the ARIES discovery-based approach for the other concepts, noting that the ARIES approach aligns well with increased calls for teaching students about the nature of science, and how to apply their science knowledge to the world around them. Student journals are an important part of this. Finally, they note that there are 2 concepts that did not lend themselves well to the constructivist approach - using outdoor shadows to tell the time, and describing what makes the number of hours of daylight change across seasons. The authors suspect that these topics may be beyond the cognitive ability of Grades 3-6, and recommend they be taught later.
Upper elementary and middle schools in a cross-section of rural, urban, and suburban schools. The date of the study was not given, but the entire intervention took place an intervals across the school year (see Data Collection and Analysis).
Formal ed: Upper elementary
Formal ed: Middle school
4B/E2bc - The rotation of the earth on its axis every 24 hours produces the night-and-day cycle. To people on earth, this turning of the planet makes it seem as though the sun, moon, planets, and stars are orbiting the earth once a day.
4F/E3** - Light travels and tends to maintain its direction of motion until it interacts with an object or material. Light can be absorbed, redirected, bounced back, or allowed to pass through.
Nature of Science
STEM content learning
Earth, Moon, and Sun: Days
Earth, Moon, and Sun: Seasons
Nature of Science: The scientific process
Physical sciences: Light and/or optics
Nature of Science