header
header


            The following
is a selection of our completed projects at Goodman Research Group, Inc

Association For Interactive Media Education, St. Louis, MO
GRG conducted formative and summative evaluation for the Association for Interactive Media Education’s (AIME) ATETV.org project. ATETV aimed to broaden the reach and impact of the existing Advanced Technology Education (ATE) community by producing interactive web-based video to promote awareness and expand dissemination of ATE efforts. Project goals were to reach, recruit, and retain more ATE students, educate policy-makers, and encourage industry involvement in ATE programs. GRG’s evaluation goals included monitoring the link between program activities and intended outcomes during video development and production, and determining effectiveness of the project in meeting its goals once the website was accessible by the public.

Blue Cross Blue Shield of MA, Boston, MA
Trustee Advantage was a grant program designed and sponsored by Blue Cross Blue Shield of Massachusetts to help the Boards of Trustees at five Massachusetts hospitals advance the governance of quality and safety improvement at their institutions. This one-year program provided funding for three components: a coaching engagement, an in-depth experiential practicum, and participation in an ongoing peer learning community with other grant recipients. As part of our process evaluation, GRG collected data from Board members, coaches, and learning community facilitators to understand how the program is unfolding. GRG’s outcome evaluation focused on intermediate outcomes such as Board members’ commitment to improved quality and safety and understanding of their role in quality and safety governance, the extent to which Boards implement or plan to implement changes in service of these goals, and the facilitators of and barriers to such efforts going forward.

Boston Children's Museum, Boston, MA

GRG is conducting evaluation activities for the Race to the Top grant received by the Boston Children’s Museum for their Museums and Libraries Project. GRG conducted simple statistical analyses on the existing surveys for which BCM has data from museums and libraries, conducted a simple analysis of past a evaluation form, and developed and launched an online survey to recipients of each of the four kits being developed by BCM to assess the extent to which museum and libraries found them effective.

Brown University, Providence, RI
GRG served as the external evaluator of the IGERT, An Integrated Program on Development and Inequality in the Global South, for the Watson Institute for International Studies at Brown University.  The evaluation included formative, process, and summative components and focused on the implementation of the project as well as the success of the project in meeting its goal of having IGERT trainees incorporate inter-disciplinary and innovative techniques in their doctoral work and beyond.  Evaluation activities included the following: an annual survey with students each year, including those who have completed their traineeships; survey with faculty during years one, three, and five; site visits to the program to talk with groups of students during selected years; and interviews with program staff.  Data collected in the first couple years was used formatively to help the program make improvements, while ongoing data collected over the five years was used to assess program effectiveness.

Center for Aquatic Sciences at Adventure Aquarium, Camden, NJ

In the Communities of Learning for Urban Environments and Science (CLUES) project, funded by NSF’s Informal Science Education (ISE) program, four museums are collaborating to build community capacity for STEM education and increase STEM learning opportunities for underserved families. CLUES provides apprentices from the community with intensive professional development in informal science education; in turn, apprentices train and mentor presenters in providing accessible family education programs on neighborhood environmental issues. GRG’s process evaluation focuses on the implementation of the apprentice and presenter professional development activities and the family STEM education programs. GRG is also evaluating outcomes, including community STEM education capacity; apprentices’ and presenters’ knowledge, skill, confidence, and interest in informal science education; and families’ knowledge of and connections to their urban environments.

Colorado State University, Fort Collins, CO

Housed at Colorado State University and supported by the NOAA and the NSF, the Community Collaborative Rain, Hail & Snow Network (CoCoRaHS) is a non-profit, community-based network of volunteers working to measure and map precipitation across the United States. GRG helped CoCoRaHS explore opportunities to expand their educational and citizen science initiatives by conducting a library needs assessment, a formative assessment of the “CoCoSchools” curriculum, and an evaluation of the effectiveness of short animated videos aimed at teaching adults about climate and weather-related topics. Data collection activities included online surveys of library staff and teachers and a two-group randomized comparison study of the impact of animated videos versus traditional webinars using 100 adult participants recruited from the general public.


Columbia University, New York, NY
GRG served as the evaluator for the PoLAR Learning And Responding: PoLAR Climate Partnership, led by the Columbia University. The partnership engaged adult learners through novel polar educational approaches to advance their understanding of climate change and then to stimulate engagement in meaningful individual and collective responses.. GRG’s evaluation activities during aimed at gauging the potential capacity of the Partnership to achieve its long term goals. To this end, GRG conducted interviews, surveys, and feedback sessions with the key players from each organization involved in the project. GRG is currently conducting evaluation for Phase II of the partnership, click here to read more about the ongoing project.     

Curved Light Productions, East Hampton, NY
GRG is conducting summative evaluation of StarTalk Radio, a highly innovative new genre of science radio hosted by Dr. Neil deGrasse Tyson that bridges the intersection between popular culture and science education. The radio programs combine comedy, references to pop culture, and public fascination with space science to reach an untapped audience for the informal science field. Goals and intended outcomes of the project include increasing knowledge and interest of space science topics, and motivating audiences to pursue additional learning activities as the result of listening to the programs. GRG’s evaluation is examining how well the joint presentation of science and humor reaches a public audience, including an audience new to informal science content, and how well it engages and informs them, and piques their interest such that they are motivated to continue learning more.
                  


Dartmouth College, Hanover, NH
“Pushing the Limits: Making Sense of Science” (PTL) is an NSF-funded partnership to build rural and small libraries’ capacity to enhance public understanding of science and math. A team including Dartmouth College, Dawson Media Group, Oregon State University, the Association of Rural and Small Libraries, and the Califa Library Group created specially produced brief video segments and professional development materials for librarians to conduct four science café-style guided public discussions for an adult audience. Each topical unit is linked to a popular work of literature; discussions are co-facilitated with a local science partner and center around the notion that we all use science in our everyday lives to “push limits.” Twenty libraries piloted the program before it was rolled out to 75 additional rural and small libraries throughout the nation.
                                      

Discovery Channel, Silver Spring, MD
GRG conducted a summative evaluation of Frozen Planet, a seven-episode prime time natural history series co-produced by the Discovery Channel and BBC. The Frozen Planet project, including series and website, aimed to introduce viewers to the earth’s polar regions and to show the impact of the year-long seasonal changes on the landscape and wildlife. GRG’s summative evaluation assessed the overall influence of the series on viewers. In particular, evaluation examined the extent to which the series visuals and the portrayal of the natural world led viewers to understand, appreciate, and want to learn more about the environment and the global importance of the poles to the functioning of the entire Earth climate system. GRG conducted a national viewer study along with a survey of visitors to Discovery Channel/Frozen Planet website.

Dudley Street Neighborhood Initiative, Roxbury, MA

GRG is conducting a survey of the Roxbury and North Dorchester neighborhoods for the Dudley Street Neighborhood Initiative as part of the organization’s grant to coordinate the Boston Promise Initiative. GRG is designing a survey to be piloted in the Dudley Street neighborhood, leading training sessions for interviewers, and analyzing the collected data. GRG will be collecting and reporting on several Government Performance and Results Act (GPRA) indicators.


EDC, Waltham, MA

For EDC’s Center for Science Education (CSE), GRG conducted process and summative evaluations of the National Infrastructure of Support for Extended Explorations in Science and Engineering in After-School Programs (NPASS).  The NPASS project was designed to train and support educators at community-based organizations to conduct more effective hands-on science and engineering projects with children. Through a variety of methods, GRG examined the extent to which the train-the-trainer model supported CBOs and informal educators in promoting children’s science inquiry skills.

When EDC received three more years of NSF funding for Taking NPASS to Scale (NPASS2): Creating State-based Professional Development Networks for Out-of-School-Time Science, GRG continued its role as external process and summative evaluator. This follow-on project provided professional development training to science trainers, who then trained Out-of-School Time (OST) providers to increase OST science programming. The program focused on underserved children in eight geographically and demographically diverse U.S. states. GRG’s evaluation assessed how the project built the states’ infrastructure and capacity to support this work into the future.  


EDC, Waltham, MA
GRG conducted summative evaluation for EDC of their NSDL Youth Resources (NYR) project. The evaluation was guided by questions about the implementation, scope, and impact of the NYR study and assessed the extent to which the NYR study upheld commonly accepted research standards, principles, and ethics. GRG also tracked the project’s effective and timely progress in meeting its overall goals. GRG's consultation activities included reviewing research designs, methodologies and instruments, as well as participating in phone/web conference calls with the research team and advisors. GRG researchers provided expertise in finalizing the data collection instruments, data analyses, and activities for dissemination of the findings.

Emory University, Atlanta, Georgia

GRG conducted an external formative evaluation for the Center for Chemical Evolution's Small Matters project, funded by the National Science Foundation with additional support from NASA. The centerpiece of the evaluation was a cross-sectional study featuring an online survey of 100 members of the project’s target audience. In addition, GRG developed a survey for the project team to administer at live literary event that featured Small Matters content. The primary purpose of the evaluation was to collect feedback about the potential for this new model, integrating multimedia and live storytelling to convey new chemistry-related research, to engage the public in science learning.


Ford Family Foundation, Eugene, OR
The Ford Family Foundation contracted GRG to evaluate its Ford Scholar Program, which offers college scholarships to graduating high school seniors and community college students in Oregon and Northern California. In addition to the scholarship funds, the Ford Scholars Program provided support and encouragement to Scholars through conferences, counseling, and on-campus resources. GRG assessed the impact of the program through an online survey and focus groups with current scholars and their parents, with the goal of capturing the benefits of the program, exploring potential improvements, and contributing to the literature on college access.

The Ford Family Foundation, Eugene, OR

GRG conducted an online survey with alumni and current Scholars in both the Ford Opportunity Program (FOP) and the Ford ReStart (RST) Program. While both scholarship programs are need-based and for residents of Oregon or Siskiyou County, CA, FOP is designed for single, head-of-household parents seeking a bachelor's degree, while RST was created to encourage adults, age 25 years of older, to begin or return to full-time post-secondary education to earn a college certificate or degree. GRG assessed outcomes of the two programs in relation to scholarship and college retention, bachelor's degree attainment, indebtedness, employment, and community service, and likelihood of college attendance without the scholarship. We also explored which scholarship enhancements have been most meaningful and have enriched their educational experience, and analyzed similarities and differences in outcomes for FOP and RST alumni with those of the Ford Scholars program, which GRG previously evaluated for the Foundation.


Franklin Institute, Philadelphia, PA
The Franklin Institute partnered with four other organizations to develop a climate change education partnership project for urban centers, called the Urban Climate Education Partnership (UCEP). These four organizations are The Center for Climate Systems Research, a unit of the Earth Institute, Columbia University, The University of Pittsburgh Center for Learning in Out of School Environments (UPCLOSE), Carnegie Museum of Natural History, New York Hall of Science, and Marian Koshland Science Museum of the National Academy of Science. GRG served as the external evaluator for the project and conducted the front-end and summative evaluation. GRG’s front-end evaluation of the CCEP program involved a needs-assessment related to the target audiences’ concern and engagement with environmental issues. To that end, GRG conducted one focus group with each of the CCEP’s target audience groups. The summative evaluation was two-fold: evaluation of UCEP network and the evaluation of inventories of climate science resources. GRG is currently evaluating Phase II of this partnership, Climate and Urban Systems Partnership (CUSP). Read more on the current project here.

Franklin Institute, Philadelphia, PA
GRG conducted formative and summative evaluation of Community Ambassadors in Science Exploration (CASE), a three-year NSF-funded project designed to develop and establish a model museum-community partnership that provides hands-on science experiences for underserved families in the Philadelphia/Camden area.

Friends of Laurel Hill Cemetery (FLHC), Philadelphia, PA
The Friends of Laurel Hill Cemetery’s Urban Mourning Project (UMP) seeked to address youth violence in Philadelphia. Specifically, the UMP combined “art, education and psychological support to help youth: 1) better understand their personal grief and loss so that they might channel this pain into productive, creative and healing outlets of self and community expression; 2) raise awareness of the mourning process as a conscious demonstration of cultural, historical and humanistic necessity.”  For this project, GRG conducted an evaluation that focused on both processes and outcomes and provided feedback to the program developers to enable adjustments to be made and to help them refine the program for future years.

Gardner Museum, Boston, MA

GRG is conducting formative and summative evaluation of the Gardner's IMLS-funded project to create a series of 12 short videos looking at “behind-the-scenes” activities at the museum. The videos will be available via on-site kiosks and on the museum's web site. After the first three videos are made, GRG will gather feedback from a range of individuals to inform development of the remaining films. At the end of the project, we will conduct an online survey to assess the videos' influence on people's understanding of and appreciation for what goes on behind the scenes in the museum, and how this impacts their personal connection with the museum.

George Mason University, Washington, DC

With funding from the U.S. Department of Education, the Roy Rosenzweig Center for History and New Media created Teachinghistory.org with the goal of making U.S. history content, teaching strategies, resources, and research accessible to all K-12 teachers. GRG evaluated the site for its overall accessibility and utility. In addition to visitors’ self-reported use and feedback, GRG monitored the website’s visitors using Google Analytics and AWStats website tracking tools.                                                                     

Harvard Graduate School of Education, Cambridge, MA

Goodman Research Group, Inc. (GRG) was contracted by Harvard Graduate School of Education (HGSE) to conduct an end-of-project evaluation of the HGSE Noyce Scholarship, funded by the National Science Foundation. The Noyce Scholarship encourages science, technology, engineering, and math (STEM) majors and professionals to pursue a career in K-12 math or science teaching. The purpose of this evaluation is to determine benefits and challenges of participating in the scholarship program and to assess scholars outcomes. GRG’s evaluation includes secondary data analysis of existing data, a survey of all former and current scholars, and interviews of program administrators and participants.

Harvard Smithsonian Center for Astrophysics, Cambridge, MA
GRG conducted a summative evaluation of The Black Hole Experiment Gallery (BHEG), a traveling exhibit and accompanying materials produced by the Smithsonian Astrophysical Observatory (SAO) of the Harvard-Smithsonian Center for Astrophysics. Two innovations of this project that were explored through evaluation activities were: 1) the inclusion of significant input from youth collaborators in the design and development phase, and 2) the use of networked exhibit technologies to personalize and enhance the visitor experience of science inquiry both within and beyond the exhibit gallery. The adult and youth collaboration was evaluated through student focus groups and adult interviews. The summative exhibit evaluation included exit interviews, surveys, observations, and analyses of data gathered through the networked technology.

Harvard Smithsonian Center for Astrophysics, Cambridge, MA

GRG conducted an evaluation of the Education and Public Outreach (EPO) activities associated with Chandra X-ray Observatory, launched by NASA in 1999 as its flagship mission for X-ray astronomy. The Chandra EPO is based at the Smithsonian Astrophysical Observatory at Harvard University and funded by NASA. Chandra EPO activities include a public web site; written and other media materials on astronomy such as posters, handouts, and DVDs; and workshops for teachers on using Chandra materials in the classroom. GRG collected data from the audiences for these programs and materials to gather information on public awareness of the Chandra mission, X-ray astronomy, and space-based telescopes as well as the efficacy of EPO programming in fostering awareness and engagement with these topics.


The HistoryMakers, Chicago, IL
GRG conducted summative evaluation of the NSF-funded ScienceMakers: African Americans and Scientific Innovation. ScienceMakers consists of a collection of interviews with 180 of the nation’s top African American scientists, as well as the associated curriculum, public programs, and website that includes a comprehensive digital archive of all interviews. In correspondence with the overall project goal to create positive role models and use scientists’ life stories to encourage others to enter scientific professions, GRG’s multi-method evaluation, in the project’s final year, focused on the effectiveness of key project deliverables for STEM professionals, adults, and youth.

Institute of International Education, Washington, DC

GRG completed an evaluation of the Fulbright Foreign Language Teaching Assistant (FLTA) program for Institute of International Education. The Fulbright Foreign Language Teaching Assistant (FLTA) program brings aspiring teachers of English from around the world to the United States to teach their native language to American college students and to develop their own professional skills. FLTAs are embedded in 2- and 4-year colleges and universities throughout the U.S. and teach foreign language classes, lead cultural events, and enroll in classes on education theory and American history and culture. Through both quantitative and qualitative data from FLTA alumni, the supervisory staff at American colleges and universities, and the American students taught by FLTAs, we assessed participants’ levels of satisfaction with the FLTA program. We also looked at personal and professional outcomes for the FLTAs, in particular measuring to what extent they increased their English language and intercultural skills.

International Rescue Committee, New York, NY

The IRC in New York’s Education & Learning (E&L) unit supports the personal growth, cultural adjustment and educational development of refugee youth, empowering them to attain high school graduation, enroll in post-secondary education, and become active participants in their new communities. We are evaluating the IRC’s E&L youth programming: Refugee Youth Summer Academy (RYSA), Out-of-School Time programs (3 after school and aSaturday program), Academic Coaching, and Leaders in Training (LIT). The evaluation is designed to provide information to be used by the New York E&L staff to make strategic decisions regarding program improvements and program funding. The evaluation results will also inform the development of a 3-5 year vision and plan for youth programming.


Jack Kent Cooke Foundation, Landsdowne, VA
GRG conducted an evaluation of the outcomes for students who received a grant from the Jack Kent Cooke Undergraduate Transfer (JKC-UT) program or were highly competitive nominees for the grant.  The JKC-UT program annually awarded grants designed to help high-achieving students transfer from community colleges to selective four-year schools.  Our study examined the outcomes for students who received the grant as well as those who did not by tracing their paths and achievements in order to identify the factors that facilitated or hindered their academic and professional success.   

Jim Henson Company, Hollywood, CA

GRG conducted evaluation of the weeklong pilot of Sid the Science Kid (STSK) Head Start camp curriculum program. STSK is a show for pre-school children with a focus on early science exploration. Head Start teachers in two cities, Los Angeles and New York, participated in the online training and implemented the weeklong curriculum program in their classrooms. GRG assessed short term teacher and student outcomes from participation in the program through pre-and post-surveys, classroom observations, and comparison with a control group of Head Start classrooms.

Jim Joseph Foundation, San Francisco, CA
GRG conducted evaluation of the Jim Joseph Foundation’s Bar Ilan University Fellows Program. The Fellows Program was in its first cohort, which consisted of 14 Jewish educational leaders from around the U.S. who were selected from a pool of 220 applicants. The program is a two-year professional development experience that provides participants with leadership development, enriched Jewish learning, and in-depth training on how to build online communities of practice (CoPs). Each Fellow was expected to facilitate at least one online CoP with educators in their respective fields. Evaluation activities included observations, focus groups, one-on-one interviews, and online surveys. GRG researchers conducted evaluation activities at the Boston Retreat in early October and also at the 10-day seminar which took place in Israel in December.

Jim Joseph Foundation, San Francisco, CA
GRG also evaluated B’yadenu (In Our Hands) Demonstration Project, Meeting the Needs of Diverse Learners in Jewish Day Schools: A Whole School Approach funded by the Jim Joseph Foundation and the Ruderman Family Foundation. The B’yadenu model is an initiative designed to establish a comprehensive, sustainable professional development model for Jewish day school educators, building long-term capacity of teachers and leaders to better serve students with a range of learning needs. GRG’s evaluation was aimed at determining the effectiveness of the hybrid model at implementing “whole school change” and provided recommendations on implementing the program in other communities. Evaluation methods included online surveys, site visits, focus groups, classroom observations, and observations of professional development sessions.


KCET, Los Angeles, CA
GRG conducted a national summative evaluation for KCET of their children’s science program, Sid the Science Kid. The daily series and associated national outreach aimed to engage children ages 3 to 6 years old and their caregivers. For children, goals included supporting and extending their natural science-related curiosity by increasing science process skills they can use to seek answers as they explore the world around them. For caregivers, goals included increased interest, comfort, and confidence to support their children’s curiosity. Primary goals of GRG’s evaluation included: 1) Examine the effectiveness of the series and website through a viewer study to compare the science-related interests, attitudes, and activities among those who are or not exposed the series and website, and 2) Assess implementation and effectiveness of the community outreach including reach, use, and satisfaction with the outreach trainings and materials. 

Massachusetts Energy Consumers Alliance, Boston, MA
The Renew Boston Residential Energy Efficiency Program, launched in 2010 as a partnership among the City of Boston, NSTAR, National Grid, Mass Energy Consumer’s Alliance, and Next Step Living, was designed to demonstrate how to achieve city-wide energy reduction goals. With a focus on Boston renters and homeowners living in 1-4 unit buildings, Renew Boston conducted home energy assessments in approximately 6,500 median income households. GRG’s evaluation activities were aimed at gauging the impact of the approaches used to overcome specific barriers in implementing energy efficiency upgrades identified through the home energy assessments. GRG conducted surveys and interviews with a stratified random sample of households that received a home energy assessment through Renew Boston. GRG also conducted statistical analyses of the extensive data already collected by Renew Boston in order to identify factors that explain a household’s full or partial participation in this program.                                                                        

Massachusetts Housing Investment Corporation (MHIC), Boston, MA
The Massachusetts Housing Investment Corporation (MHIC) is a community development entity in Boston, Massachusetts. In February 2009, the MHIC contracted GRG to conduct an analysis of the impacts that four of MHIC’s projects were having on the City of Boston through semi-structured interviews of project stakeholders. These interviews were used to write a report about the MHIC projects’ impact and to revise the data collection instruments that the MHIC used to monitor the successes of each project.


Massachusetts Linking Experiences and Pathways Study, Cambridge, MA
In March 2010, GRG received its own grant from the National Science Foundation, Division of Gender in Science and Engineering. Our research project, the Massachusetts Linking Experiences and Pathways (M-LEAP) study, was designed to answer the question: How do students’ in-school and out-of-school experiences shape their choices related to their future careers? M-LEAP was a longitudinal study of over 1,000 students who have been followed for three years, starting when they were in 3rd, 4th, 5th, or 6th grade, and who were in 5th, 6th, 7th, or 8th grade at the study's completion. Aside from student surveys (and interviews with a subsample), we collected survey data from their parents, teachers, district subject matter specialists, and community program representatives. One school in each of 8 districts across Massachusetts participated in the study. These schools were a mix of urban, suburban, and rural in different parts of the state, and the sample provided diversity in terms of socioeconomic status and of race/ethnicity.  The study's findings were released in 2014.

The research team consisted of GRG staff members Irene F. Goodman, Ed.D., Principal Investigator; Karen Gareis, Ph.D., and Laura Houseman, B.A., Senior Research Assistant. The two other co-PIs are Jacque Eccles, Ph.D., University of Michigan, and Robert T. Brennan, Ed.D., independent consultant. Dr. Eccles is the creator of the Eccles et al. Expectancy Value Model of Achievement-Related Choices.  Dr. Brennan is a statistical methodologist who specializes in the complex statistical modeling techniques that will be necessary to analyze the data from the M-LEAP study. The research team was also assisted by Victor van den Bergh, M.Ed., independent consultant.

MetroWest Health Foundation, Framingham, MA

GRG is conducting process evaluation of a nine-town tobacco education collaboration in west suburban Boston, funded by the MetroWest Health Foundation. The purpose of our evaluation is to examine the extent to which the coalition is able to accomplish its objectives of having local public health departments working together on this public health issue; the challenges and barriers to this work; level of effort across the coalition; and best practices that can be applied to similar work on other public health issues. Data collection methods include document review and semi-structured interviews with key stakeholders and program staff.

Metropolitan Opera, New York, NY
GRG conducted three one-year evaluations of HD Live in Schools, the Metropolitan Opera’s initiative to bring the Met’s live HD performances to high school students across the country. During the first two years, GRG’s evaluation activities included an online survey for all teachers who had been trained and used the curriculum materials, and paper and pencil surveys with a sample of students after their attendance at the operas and their participation in the follow-up activities. GRG’s evaluation during the third year focused on qualitative data from the teachers – anecdotal evidence and real experiences during implementation of the program. GRG also conducted phone focus groups with 23 teachers participating in the program. GRG also completed a one-year evaluation of the 2013-2014 season.

Michigan Family Independence Agency (FIA), Lansing, MI
GRG collaborated with research partner Mills Consulting Group to conduct a market rate study for Michigan Family Independence Agency of the Michigan child care work force. The survey of child care centers, group day care homes, family day care homes, day care aides, and relative care providers was designed to gather information about rates charged for care for children ages zero to 2½ and children 2½ to 13; hours for each age group; market rate at the 75th percentile by county, by age break out and by provider type; and establish a county rate, a state rate, and a shelter area rate.

MIT, Cambridge, MA

GRG is serving as the external evaluator of the Science Festival Alliance (SFA), a professional community dedicated to more and better science and technology festivals. We are conducting formative evaluation that can be used to improve the Alliance and summative evaluation to determine the vibrancy, connectivity, and effects of the Alliance, including science festival sector growth/expansion and strengthened science festival organizational capacity. This is a cross-site evaluation of the population of science festivals and SFA constituents. Data is being collected at regular intervals throughout the three years of the project using a variety of methods including program records review; surveys and interviews with festival organizers and champions; surveys with STEM professionals; observations of SFA offerings; focus groups with SFA hubs; and a survey with stakeholders.


MIT Terrascope Youth Radio, Cambridge, MA
GRG conducted a multi-year process and outcome evaluation of Terrascope Youth Radio, an NSF-funded project in which urban teens develop, report, write, produce, and host a regularly broadcast radio program on environmental and Earth-system science. The process evaluation component focused on what happens during the initiative, documenting how it unfolds and operates. The outcome component evaluated consequences for the beneficiaries of the initiative, including local youth participants, youth radio participants nationwide, and youth radio listeners. Participating teens reported on science topics that are engaging and relevant to their lives at the same time as they learn radio production skills. The initiative aimed to serve as an impetus to increase coverage of environmental science and other STEM topics and to establish a model for university researchers and students to engage and work with youth in their communities.

Museum of Science, Boston, MA
GRG conducted an evaluation of The Amazing Nano Brothers Juggling Show, created and produced at Boston's Museum of Science, to introduce nanoscience and nanotechnology to museum visitors of all ages.  The evaluation activities included surveys being taken by audience members either before or after they viewed the performances.  We also interviewed teachers who attended the program on their class field trip to the Museum.

National Academy of Engineering, Washington, DC

GRG carried out an evaluation of the Engineering Equity Extension Services (EEES), a comprehensive research-based consultative and peer mentoring infrastructure that supported enhanced gender equity in engineering education in the U.S. The evaluation included both formative and summative components. The purpose of the evaluation was to: gather formative feedback from the Experts and Extension Agents involved in the initial professional development workshops, gather formative feedback from the Extension Agents and Clients involved in professional development through the extension services, assess Clients’ perceptions of program impacts on their gender equity knowledge base and on their capacity to implement activities in support of women in engineering, and assess the influence of the extension services on the recruitment and retention of women in engineering.

National Association of Charter School Authorizers, Chicago, IL
GRG worked with the National Association of Charter School Authorizers (NACSA) to evaluate their School Portfolio Network program, a partnership whereby authorizing agencies partner with NACSA to receive comprehensive and sustained support for up to three years, in an effort to improve authoring practices. We provided a comprehensive analysis of current policies and practices, including an internal audit and a survey of stakeholders. The data gathered through this survey was used in conjunction with other data gathered by the NACSA team to determine the authorizing practices that will be addressed as part of the partnership. A similar survey was also used at the end of the partnership to determine whether stakeholders noticed changes in key areas. 

National Geographic, Pittsburgh, PA
GRG provided formative evaluation consultation as part of a planning grant that National Geographic received from the Richard King Mellon Foundation. National Geographic and a consortium of informal and formal education institutions in the Pittsburgh area came together to plan how they can best educate Pittsburgh students about climate change. The partners were seeking to leverage their various resources in order to mount a project with three large components involving professional development for teachers, programs for middle school students, and family/public outreach. Consortium members were: Carnegie Museum of Natural History, Carnegie Science Center, Phipps Conservancy, Western Pennsylvania Conservancy, Pittsburgh Public Schools, Pittsburgh Zoo, Pittsburgh Parks Conservancy, National Aviary, Allegany County Schools, Carnegie Mellon University, and University of Pittsburgh.                                              

National Safety Council, Itasca, IL
GRG conducted a comprehensive, multi-pronged evaluation of the National Safety Council Standard First Aid, CPR, and AEDtraining program — a U.S. Department of Labor, OSHA-mandated program used in business, industry, school, and community setting across the U.S. and internationally.  The evaluation team employed diverse techniques for collecting and analyzing data, including both quantitative and qualitative methods. These included a web-based survey of instructors, in-class observations of instructors and course participants, CPR skill assessments of course participants immediately following training, end-of-class written survey of course participants, and a web-based follow up survey of course participants. The information gained from the evaluation served to inform the development of priorities and strategies for the revision of the course and course materials, as well as development and refinement of education and evaluation methodologies.                    

Ohio State University at Lima, Lima, OH
It’s About Discovery was a STEM curriculum for students in grades 8–10 developed by OSU Lima and funded by NSF’s ITEST program. The program, founded on new science modules from the Ford Partnership for Advanced Studies (PAS) curriculum, aimed to extend student readiness to engage in STEM careers and equip teachers to use a challenging curriculum that brings STEM careers and content to life; technology was integrated throughout the program. GRG’s external summative evaluation focused on changes in student knowledge of and attitudes about STEM subjects, skills, and careers and teacher knowledge of how to educate students on these topics; we also examined student and teacher attitudes toward technology.

Paul G. Allen Family Foundation & Vulcan Productions, Seattle, WA
Success at the Core (SaC) is a web-based professional development toolkit – with videos as the centerpiece – which builds on research that shows how instruction, school leadership, and learning culture and community impact student outcomes. Launched in 2009 as a partnership between EDC and Vulcan Productions, with additional funding from the Stuart Foundation and the Paul G. Allen Family Foundation, SaC was designed to allow schools and teachers to create their own professional development experience customized to meet their own needs. GRG’s evaluation activities are aimed at gauging the impact of toolkit use on the quality of instruction and, subsequently, on student achievement. To this end, GRG is conducting teacher and student surveys and observations and statistical analyses of existing student achievement data.

Purdue University, West Lafayette, IN
GRG conducted formative and summative evaluation of the Louis Stokes Alliance for Minority Participation (LSAMP) program in Indiana. The LSAMP Indiana program was established in 2003, uniting Purdue West Lafayette, Purdue Calumet, IUPUI, Ball State, and Indiana University Bloomington in their goals to increase the number of underrepresented minority students earning baccalaureate degrees in the fields of Science, Technology, Engineering, and Math (STEM). This program, funded by the National Science Foundation, completed a five-year Phase I implementation. LSAMP Indiana received a second five-year award to conduct a Phase II implementation of the program. As a Phase II program, LSAMP Indiana’s goal was to triple the number of baccalaureate degrees earned by LSAMP Indiana participants within five years. GRG’s evaluation gathered process and outcome data to demonstrate the successes of the program. Site visits, observation of program components, and student focus groups were conducted to help describe the program in action. To gather information on the program’s overall goals, students and faculty mentors were surveyed each year about the role that LSAMP Indiana played in supporting students’ educational and career pursuits in the STEM fields. These data served the formative purpose of helping the alliance continue to improve the management and implementation of its program, while at the same time providing ongoing information about the extent to which the program is meeting its overall goal.

Purdue University, West Lafayette, IN
In September 2011, GRG began a study focused on the Midwest Crossroads Alliance for Graduate Education and the Professoriate (AGEP), a consortium of three universities in the Midwest region. This short-term study collected data from graduate students, faculty mentors, program staff, and administrators. Collecting data through interviews, focus groups, and a web-based survey, we will attempt to uncover institutional practices and policies at the university, school, and/or department level that students and faculty report that encourage or deter underrepresented minority persistence in graduate education in the STEM fields.

RESOLVE, McLean, VA
GRG conducted an evaluation of RESOLVE educational modules on embryo donation/adoption for potential embryo donors and those interested in embryo adoption. In this project, RESOLVE sought to increase awareness of embryo donation and/or adoption as well as to increase knowledge about the medical, legal, mental health, and practical aspects of embryo donation and adoption. GRG evaluated whether the educational modules were successful in increasing awareness and knowledge of embryo donation and adoption. Data collection methods incorporated both qualitative and quantitative data. RESOLVE will use data from the evaluation of conference presentations as a basis for adjustments and revisions to the educational modules as well as development of subsequent materials. 

SAE International (Society of Automotive Engineers), Warrendale, PA
GRG recently completed the fifth and final year of a longitudinal outcome evaluation for SAE International of A World In Motion (AWIM). With over 18 years of distribution to elementary and middle school (and now high school) teachers, the interdisciplinary curriculum supplemental materials are designed to promote science, engineering, and mathematics literacy among students by engaging them in challenging engineering and design activities.

The Schwartz Center for Compassionate Healthcare, Boston, MA
After completing an evaluation of their signature Schwartz Center Rounds® program, GRG conducted process and outcomes evaluation of a Schwartz Center pilot project. Funded by the patient safety and medical malpractice company CRICO/RMF, the Schwartz Center Connections pilot program was designed to build communication across healthcare professionals to mitigate adverse outcomes and malpractice risk. GRG’s outcomes evaluation was a pre-post study of program participants with a comparison group of providers who do not attend the program. 

Shelburne Museum, Shelburne, VT
GRG conducted an evaluation of the Passport to Learning program created and implemented at the Shelburne Museum. The program intended to introduce and engage K-8 students in art, language arts, math, science, and social studies through exploration of the Shelburne Museum. The evaluation activities included an educator survey implemented in the Spring and Fall of 2010 and the Spring of 2011. GRG also visited the Museum to observe program implementation.

Triangle Coalition for Science and Technology, Arlington, VA

The Albert Einstein Distinguished Educator Fellowship Program, administered by the Triangle Coalition and the Department of Energy–Office of Science, offers outstanding K–12 mathematics, technology, and science teachers the opportunity to serve in the public policy arena. Fellows bring their expertise to Congress and appropriate branches of federal government and gain insights into national educational issues that can then be brought back to the classroom. GRG’s evaluation examined the impact of the program not only on the Einstein Fellows themselves, but on the federal branches in which the Fellows serve and the schools to which the Fellows return. These outcomes included the Fellows’ familiarity with national STEM policy, legislation, and funding opportunities; the value added to the host branch of having a Fellow; and the benefits to the school and district to which the Fellow returns.

University of California, Jacobs School of Engineering, San Diego, CA
GRG has conducted both formative and summative evaluation of an NSF ITEST project (Information Technology Engineering and Environmental Education Tools -- IT-E3) that featured the use of environmental sensors, and includes: 1) a teacher training program with ongoing support; and 2) a curriculum that will be developed by teacher participants. Formative evaluation gathered feedback from teachers and students to improve the professional development, curriculum, and online game. Summative data gathered throughout the project was used to determine the influence the program is having on teachers and their teaching practices, as well as the influence of the program on students’ interest in and knowledge of STEM.

University of Massachusetts, Lowell, MA
GRG completed a three-year evaluation of iCODE, an NSF ITEST-funded project that used intensive, hands-on IT activities in a collaborative on-line environment. The primary aim of iCODE was to increase the likelihood that participating middle and high school students from racially diverse and economically disadvantaged communities in Boston and Lowell would pursue IT and STEM careers. The summative evaluation assessed the influence of the program on participating students' IT attitudes, career aspirations, and skills over the three years. Data collection each year included pre- and post-surveys, focus groups, and content quizzes for students, interviews and surveys of educators, and site observations conducted by GRG researchers. Evaluation results indicated that the iCODE program was highly successful and can serve as a model for other ITEST initiatives that feature programmatic efforts to make hands-on, inquiry-based engineering and programming experiences available to underrepresented groups in the middle and high schools.

University of California, Los Angeles, CA
A Systems Response to Improving Education on Aging in California (SAGE), a gerontology core curricula, was developed by UCLA for gerontology, social work, and nursing programs in both two- and four-year colleges. This U.S. Department of Education FIPSE-funded project aimed to address the need to prepare a skilled and representative workforce to provide quality services and care to older Californians. GRG conducted evaluation of many aspects of the project, including the management structure developed for the project, the Manual of Procedures, and the faculty development program.

University of California, Los Angeles, CA

As a follow up to the Systems Response to Improving Education on Aging in California (SAGE) project, we evaluated a FIPSE-funded project, the Evidence-based Health Promotion (EBHP) Educator Certificate Program. Our evaluation questions focused on the job market preparation and outcomes for community college students who completed the Career Technical Education (CTE) and the professional development outcomes for faculty who completed the faculty development program and taught the new certificate courses.

University of California, San Diego, San Diego, CA

GRG served as the external evaluator of the three-year NSF-funded Science Festival Alliance (SFA), a collaborative started by the University of California San Diego, the MIT Museum (Cambridge, MA), the University of California San Francisco, and The Franklin Institute (Philadelphia, PA). The focus of the SFA over its first three years was helping establish and sustain science festivals in each of these four cities. GRG conducted a multi-method process and summative evaluation to assess the success of the SFA project at meeting its intended impacts. Evidence of success was gained largely through surveys of public and professional audiences as well as SFA document review, interviews with SFA science festival team members, and participatory observations at SFA meetings. Our evaluation answered questions about who participated in science festivals, the benefits of science festivals (for attendees as well as participating STEM practitioners), the most important characteristics of science festivals (that were related to increased benefits for attendees), and the support that is needed to initiate and sustain science festivals.

University of California, San Diego, CA
The BioBridge professional development model, developed by educators and scientists at the University of California, San Diego, consists of a four-phase training model for high school science teachers. The four-phase model was developed over three years by using an iterative process. In 2008 the BioBridge team and GRG were awarded a research grant form the United States Department of Education to conduct a study of the BioBridge model. The purpose of the study was to gather data that can be used to systematize and strengthen the existing BioBridge training. Evaluation activities included surveys, observations, and a secondary analysis of student test scores. The final deliverable for the project was an Implementation Manual that can be used to replicate the BioBridge model and GRG’s evaluation in other sites.

University of Massachusetts, Lowell, Lowell, MA

Led by researchers at UMass Lowell’s Graduate School of Education, an NSF-funded public education project will roll out in Fall 2013 to provide information about climate science on Boston’s Mass Transit Authority (MBTA) subway placards and platforms, and on commuters’ smartphones, in an effort to teach the public about the issue. GRG’s summative evaluation will focus on the impact of Science Express on riders’ awareness and understanding of climate change as well as their motivation to take action.

USS Constitution, Boston, MA

The USS Constitution Museum's new “Forest to Frigate” exhibition focuses on the people and processes behind the building of one of Boston’s most iconic landmarks, the USS Constitution. GRG conducted a visitor study to measure the impact of viewing the exhibition on visitors’ understanding of the construction of this ship. Using intercept surveys and interviews with museum patrons, we also explored the extent to which “Forest to Frigate” promotes engagement and personal connections with the story of the USS Constitution.

Vermont Community Foundation, Montpelier, VT
In 2009, GRG completed a study of Early Care and Education in Vermont that had the goal of exploring Vermont’s distinct needs and best practices in other parts of the U.S. An informal affiliation of charitable foundations in Vermont, led by the A.D. Henderson Foundation and the Permanent Fund for the Well-being of Vermont’s Children, commissioned GRG to conduct this first-of-its-kind study of early care and education in Vermont. Through stakeholder interviews, literature review, and secondary data analysis, the study provides comprehensive, up-to-date information on the most important aspects of Vermont’s early care and education landscape, including supply and demand, early childhood spending, universal and targeted approaches to healthy child development, quality in early care and education, and education and professional development of providers.

WETA, Arlington, VA

GRG conduced its 10th annual evaluation of Reading Rockets, a national multimedia project that provides a variety of resources, strategies, and research-based information to help young children learn to read. Reading Rockets offers four websites, professional development for teachers, several TV programs in conjunction with PBS, and a variety of social media outlets to help build childhood literacy. In addition to annual benchmark questions, the 10th evaluation focused on four specific Reading Rockets resources: new Classroom Strategies videos, Literacy in the Sciences tip sheets, Family Literacy Bags, and the Start with a Book microsite. A group of active Reading Rockets users -- teachers, parents, and other education professionals -- provided feedback on each of these resources.

WETA, Arlington, VA
In 2008, with funding from the Defense and Veterans Brain Injury Center, GRG conducted an external evaluation of the website, BrainLine.org. The BrainLine website is part of a national multimedia project to disseminate authoritative information about preventing, treating, and living with traumatic brain injury (TBI). BrainLine’s target audiences includes people living with TBI, family members and friends of people living with TBI, healthcare professionals, active duty service members, reservists, veterans, and anyone else with an interest in TBI. As such, the purpose of this evaluation was to assess these population’s opinions regarding quality, ease of use, effectiveness of graphics/interactive features, and perceived usefulness of this new website. WETA sought feedback about the strengths of the site, what needed to be improved, and areas for potential growth. To answer these questions, GRG conducted a two-part evaluation comprised of both a web-based survey for site visitors and a paper-and-pencil comparative survey in which survey participants (from BrainLine’s target audience; recruited by GRG) compared aspects of BrainLine.org to those of two other websites about TBI.

WGBH, Boston, MA
Since 1992, GRG has conducted numerous evaluations of public TV programs, web sites, educational print materials, and outreach initiatives developed by WGBH. Most recently, GRG evaluated NOVA ScienceNOW season 6 (having previously evaluated the first five seasons), and the fourth season of the children's series PEEP and the Big Wide World. Additionally, in June 2011, GRG completed evaluation of the Vietnam Collection, which is part of WGBH's Open Vault project. Following is a partial list of past projects that GRG evaluated for various WGBH departments (e.g., Educational Print and Outreach, Children’s Programming): Africans in America, Arthur, Between the Lions, Building Big, Design Squad, Einstein’s Big Idea, Fetch, Lives in Science, Oliver Twist, Peep and the Big Wide World, Saved by the Sun, and ZOOM.

WNET / Thirteen, Boston, MA
GRG conducted a formative evaluation of The Music Instinct – an NSF-funded 2-hour PBS television program that aimed to convey the strong evidence of the connections between music and science as well as a deeper understanding of these two fields. The overall project consisted of the PBS program, a web site, and ancillary educational materials. The formative evaluation obtained timely information to support and guide producers as they make decisions regarding the design, content, and format of project components. Pre-production evaluation activities included an online survey of PBS viewers and music aficionados that explores the likelihood of watching such a show. Production evaluation activities included four focus groups of target audience members, an online survey of formal and informal educators and PBS viewers about the website, and a written survey of educators (middle school, after-school, and museum) about the ancillary materials, which included a tool kit and a museum guide.

Return to Top




© 2016 Goodman Research Group, Inc. All Rights Reserved.
site design:www.reddoormedia.com