you are here:

NIDRR Grantee Perceptions About Logic Models

The concept of using a logic model is a relatively new concept to some grantees, but others have varying degrees of experience applying the model to their projects. In order to obtain feedback regarding experiences in logic model development, the NIDRR-funded National Center for the Dissemination of Disability Research (NCDDR) surveyed the principal investigators, project directors, and designees from several diverse NIDRR projects for comments, both positive and negative, about the logic modeling process and their NIDRR-funded project. The comments below were selected as representative example responses to the NCDDR's survey, and they originated from newly-funded projects, some who used the logic model in their proposal development process, and some experienced grantees who applied the model to ongoing projects.

• Why Use a Logic Model
• Positive Perceptions
• Negative Perceptions
• Challenges Perceived by Grantees
• Grantee Suggestions for Improvement

Why Use a Logic Model:

• "There are two purposes; first is to help guide the investigator in producing a translational outcome that meets needs of the community the project is working with. A second purpose is to help show what NIDRR is doing overall."
• "The logic model is very useful as an evaluation tool. At first, backward chaining was awkward and difficult, but it was helpful to determine the outcomes first and then figure out what steps were needed to get there. There have been some gaps from the model to actual implementation."
• "The Logic Model gives a wider scope or aperture on the camera, rather than focusing on just one area. Don't just stay in the first half of the Model on resources, inputs, and outputs. The important part is the outcomes or impact on the user or target groups, which usually involves more than one cycle of funding given the complexity of designing products and getting them to market."
• "Until using the Logic Model, we've been doing the best we can with D&U, but it has not felt like a structured activity. We are researchers, not dissemination specialists. But dissemination is important and the Logic Model provides a structured approach that lends itself to apply to the big picture, a way to look at what we have, what we are doing, and where we can make the biggest impact. We look at our dissemination plan….does it make sense to reach this population with our limited resources, or, apply more resources to another group that might provide a better return?"
• "The Logic Model shows the BIG PICTURE….not the smaller pieces or the day to day events. A lot of the “heavy lifting” of planning, partnering, and so on does not appear in the Logic Model. It is an iterative process that may not have the desired effects in five year period. The Model is about making progress and ways to show it. Will you be held to the fire if these outcomes do not come about in five years? You try to project and to be realistic; science does not always work as you expect it to work."

Positive Perceptions:

• "As a planning tool, the Logic Model is very good. It creates a shared vision of what the DBTACs are trying to accomplish, and how. A side benefit was to identify common procedures and increase collaboration among the DBTAC projects."
• "I appreciate the effort to establish consistency across programs through use of the logic model. . . using the logic model give a clearer focus on outcomes, and clearer outcomes make it easier to clarify what should be included and to develop better criteria."
• "There are a number of “upsides.” It lets you identify your groups, figure out what activities are appropriate to reach them, and then, even before you start, what your chances are of being successful."
• "NIDRR staff gave training that was clear. Time was given as needed for orientation along the way, to understand what was wanted to help build consistency. Investment of NIDRR staff confirms they think it is a useful tool from their perspective. We should trust the judgment of NIDRR staff in implementing such an activity in common. . . it allows NIDRR to begin to pull things together across Centers."
• "There is value at looking at outcomes, but learning the new terms is like converting everything to a language….. Perhaps if had been incorporated into the initial proposal, along with data collection techniques, it would have been better than after the fact."
• "Staff have been able to harness potential and put the Logic Model to constructive use within the Center. A most notable impact has been on how progress is assessed. Use of the Logic Model has extended assessment and performance measurement for specific projects into new areas that were not previously tracked. It has been a positive activity that led to developing performance measures for projects that solidify accountability."
• "The only logic model work that we have done was in preparation for our formative review. Our instructions for the formative review were to consolidate the results of all of our project activities into about three anticipated overall outcomes. Since that time, I have referred to our "overall outcomes" on several occasions when I needed to summarize the purpose of our project to outside parties. That logic model has also reinforced the idea among our investigators that R&D and training/dissemination projects must be linked with a common goal in mind. This idea is too often lost when we are asked to report just R&D activities, or just training/dissemination activities."
• "In reality, when putting the latest application together, thinking ahead about impacts and outcomes--thinking strategically--was very constructive. Led staff to develop some diagrams to be able to effectively share information with others. The Logic Model helps develop a commonality of purpose; facilitates getting on same wavelength with others; helps all know where they need to go."
• "In general, since dissemination is so important, anything that can help carry it out more systematically is a good thing. Looking at a Logic Model can help us determine what activities and audiences we should be focusing on. Going through the process helps us focus so that we are investing out effort where it will best pay off."

Negative Perceptions:

• "Very time and effort-intensive. Lots of time was required outside of scheduled meetings. However, this first step learning experience will be paid back by benefits. For example, DBTACs are now ready to implement the Model and use it for data gathering for reporting and evaluation purposes."
• "It may not be ambitious enough. Projects are asked to identify just a few outcomes, holding the bar rather low. Long-term outcomes may sound like short-range process goals to some of the project staff."
• "We were asked to consolidate outcomes to just three, but then to be very specific about those three. This did not make sense to project staff. In order to track the effectiveness of our project activities, we need to examine each activity individually -- not combine them. While a fully detailed logic model may provide this information, the logic model we've developed to date (the version developed for the formative review) does not transparently reflect that information. When we combined related projects together that had a common goal, the result was that different types of activities for very different target audiences also became grouped together."

Challenges Perceived by Grantees:

• "The biggest challenge overall was working as a large group; there was difficulty getting consensus from ten projects. But a 13-year history of working together and past experience in breaking into small task groups helped make it happen."
• "We've been struggling with the language of the Logic Model over the past two years and in writing the dissemination plan for our newest proposal. Logic Model is a terrible name; it's not descriptive, not helpful. It can be a useful evaluation tool for your project, prospectively or retrospectively--you need to plan for evaluation."
• "It was initially tedious to think in the terms of the Logic Model process….It was challenging to help all staff to understand the system. There were higher requirements for communication and helping others to “get on board.” Costs associated with time, communication, demands, and planning energies were not required previously. Those costs have been reasonable comparing with benefits realized."

Grantee Suggestions for Improvement:

• "Mentoring is needed for implementing the Logic Model. NIDRR provided help in preparing for this project's formative review. This requires a new language…..you need time to learn how the words are used."
• "Hiring a Logic Model-experienced consultant was a big plus to help facilitate original brainstorming and planning meeting. Having an experienced outside person was helpful in gearing the group to work on Logic Model."
• "Hiring a Logic Model-experienced consultant was a big plus to help facilitate original brainstorming and planning meeting. Having an experienced outside person was helpful in gearing the group to work on Logic Model."
• "Our project started adapting logic models before NIDRR did and we will continue to use the process. It fits with what we've been doing and we've developed a framework and procedures to use it. Be sure that the model allows for incorporating history and evidence of experience from previous work."
• "Although I believe that the logic model approach has potential for project planning and tracking outcomes, the reduced version created for our formative review has not been particularly useful. I would suggest that future groups be asked instead to develop a full logic model that represents individual activities and deliverables as a cohesive plan."