Practitioners from different countries have been commenting on the new guidelines on the evaluation of LEADER/CLLD. These guidelines, published by the European Evaluation Helpdesk for Rural Development complement the legal requirements relating to LEADER/CLLD evaluations and aim to clarify the common elements related to LEADER/CLLD evaluations and to offer a wider set of recommendations on how to formulate and manage LEADER/CLLD evaluations, both at the RDP and LAG levels. We publish below comments from local practitioners from Germany and the UK. Contributions from other LAGs are welcome.
Implementation of EAFRD and LEADER has in general become a very complicated highly regulated administrative act and a major challenge for the national authorities as well as for the LAGs and even more for the beneficiaries. This high bureaucratic effort is due to ensure that EU money is used to fulfill the EU objectives only. Monitoring and evaluation shall show how much the funded measures contribute to the targets of EU policy.
With the guideline the evaluation at all levels shall be supported to comply with the challenges of reporting the effects of LEADER. Target groups of the guideline are Managing Authorities, National Rural Networks, evaluators, as well as LAGs and officials within DG AGRI.
The guideline seeks to plausibly present the legal basis as well as the logic and objective of the evaluation for all levels and derives possible evaluation methods from it. This is a very high requirement and very well done for many aspects. However, from my point of view, it is almost impossible to do justice to all target groups. Perspectively, therefore, individual handouts could be more targeted, which are customised to the specific target groups. The guideline provides a very solid basis for this and extraction and processing of the individual parts could be a future task of the help desk. To increase the usability at national and local level, it is furthermore inevitable to translate the guideline. [The evaluation guidelines have now been translated into 22 languages].
As a general remark about evaluation in LEADER I want to point out a danger of a one-dimensional view of the evaluation from the viewpoint of the evaluators: the impression arises that the evaluation could provide a very accurate, detailed and objective picture of the effect of all imaginable measures. However, the assessment always remains an attempt to approach reality. Evaluation is therefore not suitable as a means to justify political decisions, but to expose undesirable developments that can be avoided by an appropriate response. Just as the illusion of almost complete flawlessness in EAFRD or LEADER funding has led to an unmanageable level of regulation, there is the danger that the illusion of complete controllability and accountability will escalate the methods and effort of the evaluation.
Especially with the mostly very small LEADER projects, it is not possible to reliably measure direct effects, for example on the number of employees, tourists or biodiversity indicators in meaningful size units. They are usually additionally heavily overlaid by large-scale development tendencies. With this the proportion between subsidies and evaluation effort can be completely out of balance. But even with very high efforts there is not only the danger that the impact indicators cannot be realistically recorded, but also that wrong conclusions are drawn because overlay factors are not recognized.
The effect of LEADER measures is often very long-term, e.g. the development of a culture of cooperation or the strengthening of regional identification. The long-term impact on jobs can be much more effective than promoting an investment in a single enterprise, which, however, directly contributes to the evidence of a new job. The isolated consideration of these supposedly easily measurable indicators can lead to a misinterpretation of the effect of measures.
In principle, the evaluation approach at the LAG level should focus on recording the implementation of the LEADER method:
- Adequate participation of regional actors in the formulation of the strategy and its implementation.
- LAG animation and capacity building
- Multi-sectoral approach and cooperation
This focus has already been implemented in the guideline in essential approaches and makes it to a useful tool. Overall, however, the proportionality between evaluation effort and informative value must be an even clearer maxim.
Dr. Hartmut Berndt
German LEADER Association, BAG LAG e.V.
I have a few general comments:
- The guidelines are an excellent guide for both managing authorities and local action groups, and it’s good to have all the guidance in one place so each can see what the other should be doing
- The guide might be a bit daunting for an inexperienced LAG member to tackle alone, but as a guide for accountable bodies and as a basis for LAG development and self assessment within a group session I can see it being invaluable. It might be useful to have a very short summary version or even just a flowchart showing the evaluation process to circulate to LAG members in advance of such a session or as an aide memoire
- I do find it a bit of a shame that evaluating the added value and increased social capital are not mandatory – for me these are the absolute bedrock of CLLD/LEADER, and managing authorities should not be able to avoid these evaluations. I can see that the current England LEADER programme would do quite well in an evaluation of its contribution to wider RDPE objectives because it is so structured to ensure this, but its added value is somewhat debatable and the LAGs’ ability to innovate and cooperate are severely impacted by the lack of flexibility – under current requirements the Managing Authority won’t need to consider this within their evaluation, although armed with this guidance we will certainly be encouraging them to do so.
External Funding & Policy Manager, East Riding of Yorkshire Council*
*Accountable Body for the Coast, Wolds, Wetlands and Waterways LEADER programme, the Holderness Coast FLAG programme and the Yorkshire Coast CLLD programme