Evaluating Your Proposal

Evaluating what you propose to do is often the weakest link in a proposal, because few grantseekers actually develop a plan for measuring and reporting project results. Some grantwriters ignore evaluation altogether. Remember, when someone gives you money to solve a problem, the question in the funder's mind at the end of the project is going to be: "Did it work?"

Evaluation of your program is really a lot easier than you might think. Just think of it in terms of the following question: how will you measure your success? When you write your evaluation plan, you must match each objective with a corresponding evaluation criteria. Proposals that include evaluation components have clearly thought through the program development process and have determined ways to identify and document what worked and what didn't.

Be sure to read the Request for Proposal (RFP) carefully for specific evaluation procedures. In general terms, you will:

• describe your evaluation criteria and identify how you will indicate success.

• detail all procedures that will be used, including instruments, surveys, checklists, etc.

• create a REALISTIC plan (not complex; keep it simple).

When I write an evaluation plan, I like to clearly identify:

• the methods and procedures to be used to evaluate all of the project's components.

• the instruments to be used in the evaluation of the program.

• the persons responsible for the program evaluation (this might be a project employee, a school or district employee, and/or an independent contractor).

• how the evaluation feedback information will be used to improve the program. In a project timeline I like to indicate when and how we will review our progress and formally mark our milestone events along the way.

A Ten Step Plan

Whereas your needs assessment identifies the problem or the current situation, your program suggests specific activities intended to "bridge the gap" from what is to what is wanted. In a sense, as you plan your program, you are also looking for "expected outcomes." Your evaluation of your project will document these expected outcomes and may even find a few surprises along the way.

I offer a ten-step process for creating your evaluation plan.

1. Decide on the scope of the evaluation.

What will you attempt to capture?

2. Decide which tasks to evaluate.

Which parts of your program will you evaluate: student outcomes, teacher staff development gains, implementation strategies, etc.?

3. Decide evaluation questions to answer.

It helps if you phrase your evaluation plan as questions.

4. Decide which instruments or procedures you will use.

Will you count on standardized testing to show gains in student achievement? Will you create an attitudinal questionnaire for project participants and look at pre- and post-testing? Will you look for levels of concern as you implement your program? Will you look at case studies?

5. Decide on the respondents for each instrument.

Which of your program participants will take which identified instrument — for example, teachers may take a different attitudinal questionnaire than parents or students take.

6. Decide when to gather the information.

Will you do so pre and post? Quarterly? Every other week?

7. List what to do and who will do it.

8. Estimate the cost of the evaluation.

9. Make a timeline for the evaluation.

10. Decide how you will interpret, communicate, and act on the findings.

Here is a brief example. Using the scenario from our previous lessons, in this evaluation plan I try to capture what has taken place and how we will look at improving the program as we continue to implement the project. As you remember, this project was to help English Language Learners access the core curriculum, especially in science. Through the use of graphic organizers and HyperStudio projects, English language learners gain access to core curriculum content and experience in reading, writing, and communicating. Here was my evalution plan:

"Periodic surveys will be used for formative and summative evaluation at the school-wide level. Teachers will report on and evaluate the effectiveness of staff development, trained parent assistants, access to multimedia equipment, student peer aides, Project Coordinator support, and their peer coaches by observing their students' progress as they produce their projects. Students will complete a written evaluation of their training experience as well as evaluate their classmates' projects. Teachers and parents involved in the project will attend a monthly project evaluation and troubleshooting meeting. Dr. Garcia will write the final report with the assistance of the Instructional Technology Specialist. The report will be delivered to the State. We also plan to share our student projects at our local and statewide Computer Using Educator conferences."

Formative evaluation looks at process. This type of evaluation concerns your procedural design and program implementation.

Summative evaluation is keyed to your program objectives and determines the degree to which your objectives have been achieved.

Your Evaluation Plan

Your homework is to create your proposal's evaluation plan. Begin by creating an evaluation component for each objective. Determine which instruments you will use. Keep it simple. If you have developed a product or if your process is the product, determine how you will share this with others (dissemination).

Just as a side note, often educators want to "increase student self-esteem" or create a "greater interest in school." These intangibles are often difficult to document. In these cases you could design original questionnaires or attitude surveys. You could also interview and/or use case studies. These supplemental instruments should be placed in an Appendix, if allowed by the RFP. Remember to "test" your instrument on a small sample for clarity. Is your instrument giving you the kinds of results you need to determine your program's effectiveness?