At ETI, we design each of our evaluations according to the unique needs of our clients. We carefully consider the evaluation’s “intended use by intended users” and work closely with clients to define the purpose and priorities of the evaluation before recommending specific evaluation methods. Our approach is based on the utilization-focused theory of evaluation planning (U-FE; Patton, 2002), which emphasizes democratic processes and strong organizational ownership of the evaluation plan.
Planning & Assessment
Evaluation Planning. We guide our clients through the processes to prepare them for effective program development and use of evaluation, including developing clear and measurable program objectives and creating a logic model to ensure that the evaluation design is aligned with the stated program goals.
Needs Assessments. We help our clients determine if a program/project is really needed through comprehensive assessments of the target audiences’ needs. We design a process that provides appropriate input to inform decision-making and guide the development of a program that will be successful.
Formative Evaluations. We work with the client in the development of a program, reviewing the program design, processes, structure and implementation to help identify areas that are working well and areas that need to be improved. Formative evaluations can answer the questions related to program implementation, such as: “What activities worked the best?” or “How did the program participants experience the workshop?”
Summative Evaluations. With the groundwork established in the implementation phase, we measure the results of a program, comparing the program outcomes against the stated mission/objectives/standards. Summative evaluations can help to answer the key questions: “How much did the participants change as a result of the program? or “How long did changes to the participants last?”
To help our clients gain access to valid and reliable information about their programs, we provide a full spectrum of applied research methods related to formative and summative program evaluations. We have experience working with sensitive populations, such as minors, K-12, and special education, and work with different linguistic groups, such as Spanish speakers and others. In addition, we are experienced using school-based student data systems, analyzing the relationships among academic achievement scores, grades, attendance, English language proficiency, advanced placement enrollment, graduation rates, and other school outcomes.
Our expertise includes the following applied research skills:
Small and large-scale surveys, using paper, e-mail and online data collection strategies
Site observations, including classrooms
Single and Mixed Methods research designs: case studies, quasi-experimental, and experimental (including randomized control and treatment groups)
Analysis of statistical data, ranging from simple descriptive statistics to complex statistical modeling (i.e. latent variable modeling, multi-level modeling, hierarchical regression, and others)
Technical Support & Training
Literature Review of Best Practices. We dig into scientific databases and other sources of information to select a sample of the most successful programs. We synthesize the literature and give the client concise and user-friendly findings about why they are successful.
Organizational Capacity Building. We work with organizations to help them establish accountability systems through the organization, identifying evaluation goals and metrics for every division and subgroup within the organization. We provide technical assistance in the development of forms for evaluating specific events and work with grantees of organizations to design and implement their evaluation plans.