As every Project, CYBELE has to find a way to evidence its impact, especially in relation to the Project: Precision Agriculture (PA) and Precision Livestock Farming (PLF) Demonstrators. Evaluation is a process that critically examines an implemented solution, in our case PA/PLF Demonstrators. It involves periodic collecting and analyzing information about characteristics and outcomes of the innovative solutions, in such a way that the resulting information can be used to determine whether all planned activities are effectively carrying out, and the extent to which stated objectives and anticipated results are achieved. Evaluation can and should be used as an ongoing management and learning tool to improve solution’s effectiveness.
What type of evaluation can be conducted and when?
Evaluations fall into one of two broad categories: formative and summative. Formative evaluations are conducted during the development and implementation of a new solution and are useful in the case of determining direction on how to best achieve pre-established goals or improve some aspects of the solution. Summative evaluations should be completed once the solution is well established and its purpose is to show to what extent the solution is achieving its goals.
Within the categories of formative and summative, there are different types of evaluation.
Adapted from: Pancer, s. M., and Westhues, A. (1989) “A developmental stage approach to program planning and evaluation.” Evaluation Review (13): 56-77.
In terms of end-users, evaluation can be conducted internally and externally. Internal evaluation refers to a review of the quality of the work done within the participants of the solution development process. This may also be called as self-evaluation. Internal evaluation is done to support the solution’s improvement efforts and as part of a performance appraisal process. It is commonly done before an external evaluation.
External evaluation involves entities or individuals concerned outside the solution development process. Unlike internal evaluation, it reviews the solution in the perspective of those who are not linked with it. This kind of evaluation, most of the time, has the goals and rationale of the solution as reference. It is used as a tool of standardization for the assessment.
Although internal evaluation contradicts the external evaluation in meaning, and vice versa, these two actually go hand in hand in order to achieve the goal of a project.
In the scope of CYBELE project all previously explained types of evaluation will be conducted in different phases of the project implementation. Evaluation of the CYBELE Demonstrators and the CYBELE platform has already started in parallel with the kick-start of the demonstrators and particular attention is paid on constantly monitoring the exploitation and potential business impact of the collected data and of the offered tools and services by the demonstrators. The provision of generated insights, data and intelligence reports back to the PA/PLF data value chain.
All participants (scientists, technical providers and future beneficiaries) are involved in the evaluation process in appropriate ways. This collaboration helps ensure that the evaluation is fully participatory and builds commitment on the part of all involved to use the results to make critical demo improvements.
So far, use-case specific business and technical Key Performance Indicators have been specified and will be used as a benchmark in the future evaluation activities. In order to determine whether these KPIs are achieved, structured and unstructured communication with stakeholders will be established to capture their opinion and experience. The lessons learnt from the project (with regard to the implementation, operation and execution of the demonstrators) will be gathered and formulated as methodological adoption guidelines for the further exploitation and utilization of the CYBELE platform, tools and services.
Pancer, s. M., and Westhues, A. (1989). A developmental stage approach to program planning and evaluation. Evaluation Review (13): 56-77.
Rossi, P., Lipsey, M.W., & Freeman, H.E. (2004). Evaluation: a systematic approach (7th ed.). Thousand Oaks: Sage.
Love, A.J. (1991). Internal Evaluation: Building Organizations from Within. California: SAGE Publication.