GRG is conducting an evaluation of the Boston Harbor Islands (BHI) Youth Engagement project, a two- year joint initiative of Boston Harbor Now, Inc. (BHN), Save the Harbor/Save the Bay (SHSB), the Massachusetts Department of Conservation and Recreation (DCR), and the National Park Service (NPS), with funding from the Massachusetts Environmental Trust (MET). The evaluation has two main components. The first component involves consultation on the Youth Engagement Plan being formulated by the BHI partners and a situation analysis to better understand who is being served by existing programs and assess how the programs could be improved. The second component is a formative evaluation of the Boston Harbor Islands Discovery Camp taking place in summer 2016 and summer 2017, which involves camp observations, student surveys, and discussions with youth service providers.
The PoLAR Learning
PoLAR Climate Partnership, led by Columbia
University, engages adult learners through novel polar educational
approaches to advance their understanding of climate change and to
stimulate engagement in meaningful individual and collective responses.
GRG is serving as external evaluator of this Phase II project, which is
one of six projects in the NSF Climate Change Education Partnership
(CCEP) program. We also conducted evaluation during their Phase I
planning grant. Throughout Phase II, GRG will conduct formative and
summative evaluation as the partnership evolves, and as the cohesive
suite of educational tools and technology are developed and
disseminated. Formatively, we are assessing the likely effectiveness of
the products and provide suggestions for modification. Summatively, we
are assessing project’s effectiveness regarding the communication
efforts and intended audience impacts.
GRG is performing an evaluation of the Council Bluffs Community Education Foundation's STARS scholarship, which aims to help residents of Pottawattamie County that have dependent children earn their associate's and/or bachelor's degrees. GRG is conducting mixed-methods process and outcomes evaluations of the program. This includes a review of existing program literature, data, and scholarship program processes, conducting interviews with current staff members, and online surveys of current scholars and alumni.
Rural Gateways is designed to enhance rural librarians’ self-efficacy and professional self-identification as informal science facilitators through professional development, an online community of practice, and scaffolded experiences of providing adult STEM programming in their communities, with the eventual goal of building their capacity to sustain this programming beyond the life of the project. GRG’s summative evaluation focuses on the quality of the professional development model, the success of the adult science programming, and changes over time in rural librarians’ self-efficacy and identity as informal science facilitators. Our evaluation is one component of a larger research project funded by NSF AISL that will inform the field of informal science learning, with particular emphasis on how to reach new and underserved audiences.
Education Development Center, Inc. (EDC), Waltham, MA
EDC’s STEM Learning and Research (STELAR) Center, created in partnership with the EdLab Group and funded by the National Science Foundation (NSF), is designed to serve the Innovative Technology Experiences for Students and Teachers (ITEST) community of practice. STELAR provides technical support and monitoring of ITEST projects throughout their implementation lifecycle, informs and influences STEM/ICT stakeholders by disseminating project findings nationally, and deepens the impact and reach of the ITEST program by broadening participation to areas and institutions not currently represented in its portfolio. GRG’s formative and summative evaluation of STELAR will assess the implementation, effectiveness, and impact of its technical support, dissemination, and outreach activities.
EDC’s Massachusetts Exploring Computer Science Partnership (MECSP), funded by the National Science Foundation, is working to broaden computer science education by expanding the Exploring Computer Science (ECS) course in high schools across the state, providing professional development to ECS teachers, and creating a cadre of Teacher Leaders to prepare others to teach ECS. Partners include the Advanced Math and Science Academy (AMSA) charter school, UMass Boston’s Broadening Advanced Technical Education Connections (BATEC) program, Massachusetts Technology Leadership Council (MassTLC), Massachusetts Computer Attainment Network (MassCAN), and Code.org. GRG’s evaluation of MECSP will assess the implementation, effectiveness, and impact of the program on new ECS teachers, Teacher Leaders, and students through surveys, observations, embedded assessments, and records review.
Institute, Philadelphia, PA
Franklin Institute in Philadelphia is partnering with five other
organizations to develop a climate change education partnership project
for urban centers in four cities. Climate and Urban Systems Partnership
(CUSP) partners include The University of Pittsburgh Center for
Learning in Out of School Environments (UPCLOSE), The Center for
Climate Systems Research, a unit of the Earth Institute at Columbia
University in New York, Carnegie Museum of Natural History in
Pittsburgh, New York Hall of Science, and Marian Koshland Science
Museum of the National Academy of Science in Washington, DC. GRG is
continuing our role as external evaluator, having evaluated the Phase I
planning grant. In Phase II, GRG is monitoring the progress and
dynamics of the CUSP partnership and evaluating the Urban Learning
Networks (ULN) developed by the informal learning institutions in each
city. Evaluation is focusing on the four separate educational platforms
created, with examination of their goals and functioning,
implementation, audience impacts, and ultimate cross-city and
The Franklin Institute is partnering with the Center for Neuroscience
& Society at the University of Pennsylvania to create
Neuroscience in Your World, an NIH-funded suite of K-12 programs
related to the museum's upcoming exhibition on the human brain and
designed to inspire and inform K-12 students and teachers. Curricular
and professional development resources are also available digitally in
order to have an impact beyond the local initiatives. GRG is serving as
the external evaluator for the project, starting with formative
evaluation of a semester-long course on neuroscience and society for
grades 11-12 and continuing with summative evaluation of an all-day
experiential field trip package for grades 5-8, a neuroscience
mini-course at the museum for grades 9-10, and the elective course in
schools for grades 11-12. GRG is also conducting summative evaluation
of the Franklin-provided professional development for K-10
GRG is conducting process and outcome evaluation of The HistoryMakers Digital Archive project, funded by the PwC Foundation. The HistoryMakers will transcribe and process 2,000 interviews that are not yet accessible to the public via The HistoryMakers Digital Archive. The goal is to increase access for students and teachers in public and charter schools to the resources and lesson plans afforded through these primary source materials. GRG will conduct a case study with three phases of external evaluation intended to assess the effectiveness of the fully populated Digital Archive (i.e., containing full transcripts of all interviews). The three phases will include: key stakeholder interviews, process evaluation including site visits and classroom observations, and outcome evaluation including teacher and student surveys.
Linking Experiences and Pathways Follow-On Study, Cambridge, MA
M-LEAP 2 is a three-year longitudinal research designed to continue exploring how early science experiences in formal and informal settings are related to gender-based differences in science achievement-related choices in middle and high school. The research is guided by the Eccles Expectancy-Value (EV) Model of Achievement-Related Choices. It will serve as a complement to and extension of the Massachusetts Linking Experiences and Pathways (M-LEAP 1) study, a primarily quantitative research project. M-LEAP 2 is heavily qualitative; using in-depth interviews with 84 of the current students and their families, we will start with 6th-9th graders and follow them to grades 8-11. This study will allow us to uncover a deeper understanding of the responses in M-LEAP 1, while broadening the scope of the research. This method will be supplemented by the quantitative surveys used in M-LEAP 1 and student diaries to be introduced in M-LEAP 2.
The senior research team consists of PI Irene F. Goodman, EdD, President of Goodman Research Group, Inc. (GRG), and co-PI/Project Director Karen Gareis, PhD, GRG Senior Research Associate. Dr. Gareis is an experienced research psychologist. Dr. Eccles, a gender studies expert and developer of the E-V Model, will serve on our advisory board, as will Dr. Cynthia Char, a seasoned evaluator, and other experts. Dr. Robert Brennan, a renowned expert in multilevel modeling and other sophisticated statistical techniques, will continue as statistical consultant.
GRG is conducting formative evaluation and longitudinal outcomes evaluation of It Takes a Village, the start-up phase of the larger four-year Art for Change (A4C) project, at the Massachusetts Museum of Contemporary Art (MASS MoCA). We are assessing the program’s effect on children's understanding and demonstration of positive habits of mind; specifically, empathy, optimism, courage, and problem solving, using quantitative and qualitative methods, including a pre-post survey of students, writing prompts, and coding of their discussions during museum visits. GRG is also helping MASS MoCA build internal evaluation capacity via staff trainings on survey administration, systems for planning and tracking data collection, and data management and storage.
GRG is conducting a formative evaluation of the museum's Cabinet of Art and Curiosity exhibit, which introduces visitors to a unique concept of collecting that flourished in 16th- and 17th-century Europe and encourages them, through traditional and digital interpretive materials, to make connections to the present time. GRG will be conducting Walkalongs, wherein visitors walk through the gallery for the first time, accompanied by a field researcher, and discuss their initial reactions and experiences. GRG will also be conducting intercept surveys, with exhibit visitors, with the aim of assessing their experience with the exhibit, as well as how well they comprehend the themes and concepts being presented.