The Retirement Research Foundation has three objectives in promoting the use of evaluation as part of every grant the Foundation makes. These are:

• to encourage applicants and grantees to be more effective learning organizations, gathering and systematically analyzing important client and program information about the nature, reach, quality and efficiency of the services they provide;
• to understand, as a Foundation, the value of investments we make and the lessons that funded projects can teach us about how to invest our grant dollars more effectively in the future; and
• to add to knowledge in the field about best practices in services for older adults by supporting, when appropriate, rigorous experimental or quasi-experimental outcome studies.

As this list suggests, evaluation is not exclusively about whether a grant “makes a difference” for seniors. Many other types of evaluation questions are equally important. Listed below are the three categories or types of evaluation RRF uses. The blue link that begins each section can be “clicked” to provide the reader with a detailed list of the types of questions each covers. We ask applicants to consider this information in light of what they seek funding to do, and to incorporate evaluation plans most appropriate to their goals. Funding to cover evaluation activities can be included in your request. You may call us at (773/714-8080) or e-mail Nancy R. Zweibel, Ph.D. for more direction.

Implementation Evaluation asks about the practical lessons that emerge from putting a new project into action. Rarely does a project go off without a hitch. Lessons learned in implementation help grantees identify where their approach may need modifying and what critical next steps are needed. In turn, these lessons can help others wishing to replicate grantees’ work avoid the pitfalls grantees experienced. Finally, they teach the Foundation Trustees and staff important lessons that can help us improve our grantmaking capacity. Implementation evaluations are relevant for such activities as (1) planning; (2) taking a proven model and adapting it or offering it to a different audience; (3) testing the feasibility of an innovative program for the first time; and (4) community organizing or other advocacy-related work. Direct service grantees looking to expand services into new communities, or add to service capacity, also should attend to lessons they can learn as they implement their efforts.

Process Evaluation is designed to generate a blueprint of a program in action. There are three purposes for process evaluation: (1) to describe for the Foundation how you use funds provided; (2) to give others wishing to replicate your work a guide to follow; and/or (3) so that when the outcomes of a model program are to be studied, it will be possible in the end to describe what the “intervention” consisted of in reality, not just as designed. Process evaluations document how a program operates by describing characteristics of clients and staff, the nature of services offered and methods of delivery, and patterns of service use. Most applicants will find this form of evaluation relevant for their proposed work.

Outcome Evaluation is what most people think of when they hear the term evaluation. It focuses on determining whether a novel program improves one or more targeted results (e.g. health, mental health, quality of life, risk of falling, rehospitalization rates, etc.) for those served. To be done correctly, outcome evaluation requires that clients are compared to a control group of older adults who are similar to clients in every way except for the fact that they are not exposed to the program being studied. It is important to note that this type of evaluation is appropriate for only a minority of our grantees. One reason for this is that many projects submitted for our consideration are for services that we already know work! Another is that doing outcome evaluation is expensive, time consuming and requires the involvement of experts with a track record documenting knowledge of and experience with evaluation research and statistics. Finally, the Foundation feels that outcome studies are worth funding only when the proposed project is likely to be replicable and already has been pilot-tested to document that it is feasible to implement.

In addition to reading the content available from the links above, you can learn more about program evaluation at these Web-based Evaluation Sites.

Back to top