Examining the Evaluation Capacity, Evaluation Behaviors, and the Culture of Evaluation in Cooperative Extension

Abstract

Evaluation is a burgeoning field and remains fairly young by most standards. Within Cooperative Extension, evaluation practices have been implemented at a variety of levels given that evaluation is mandatory for much of the funding Cooperative Extension receives. With evaluation in high demand, it is expected that most Extension educators are performing some levels of evaluation as a routine part of their jobs. In order to perform the required evaluations, an Extension educator must exhibit some level of knowledge and skill regarding evaluation. While much research to date has been done on the level of evaluation within the organization, there is a lack of understanding regarding the evaluation competencies that Extension educators must possess and the culture of evaluation within the organization. This study set out to examine the evaluation competencies, culture, and evaluation behaviors within Cooperative Extension. Utilizing an online survey format and quantitative methodology, a widely accepted set of evaluation competencies were examined for their importance within Cooperative Extension. A panel of 13 experts was selected to examine the competencies and it was determined than all competencies in the list were necessary for Extension educators to exhibit in their jobs. The list of competencies was then combined with a subscale regarding culture and a subscale based on the Theory of Planned Behavior (Ajzen, 1991). A total of 419 Extension educators in four Extension systems participated in the study, with 222 generating usable data for a response rate of 13%. The highest and lowest skill level for the competencies were determined by Extension educators self-reporting. Perception of importance of each competency was examined and the highest and lowest importance rankings were determined. These were compared to the rankings of importance by the expert panel. A path analysis was conducted by modifying the Theory of Planned Behavior model and multiple regression analysis. Mean weighted discrepancy scores were calculated to determine the differences in skill level and perception for each of the competencies. The subscale of culture was examined for potential areas of Evaluation Capacity Building (ECB) within the organization. Results show that while there was much agreement between the expert panel and Extension educators regarding the importance of competencies, experts ranked all competencies as important while Extension educators did not. The results of the path analysis determined intention and perceived behavioral control explained 3.9% of the variance in the evaluation behavior exhibited by skill. Subjective norm and attitude explained 11.8% of the variance within intention. Perceived behavioral control, attitude and culture accounted for 13.1% of the variance in subjective norm. Culture and perception accounted for 7.1% of the variance in attitude. Perception, program area, college major, location, training in evaluation, degree level and years of experience explained 28% of the variance within evaluation culture. Finally, recommendations for practice and future research were made based on these findings.