Evaluation of CLLD: concerns raised by local practitioners and a practical response

Practitioners from different countries have been commenting on the new guidelines on the evaluation of LEADER/CLLD. These guidelines, published by the European Evaluation Helpdesk for Rural Development complement the legal requirements relating to LEADER/CLLD evaluations and aim to clarify the common elements related to LEADER/CLLD evaluations and to offer a wider set of recommendations on how to formulate and manage LEADER/CLLD evaluations, both at the RDP and LAG levels.

We have recently published below comments from local practitioners from Germany, Greece and the UK.

FARNET and FAME have recently developed a practical evaluation guide addressed specifically to LAGs: Evaluating CLLD: Handbook for LAGs and FLAGs – Urban, Rural Coastal.  It is hoped that the guide will go some way towards addressing the issues raised by the local practitioners.

 

Implementation of EAFRD and LEADER has in general become a very complicated highly regulated administrative act and a major challenge for the national authorities as well as for the LAGs and even more for the beneficiaries. This high bureaucratic effort is due to ensure that EU money is used to fulfill the EU objectives only. Monitoring and evaluation shall show how much the funded measures contribute to the targets of EU policy.

With the guideline the evaluation at all levels shall be supported to comply with the challenges of reporting the effects of LEADER. Target groups of the guideline are Managing Authorities, National Rural Networks, evaluators, as well as LAGs and officials within DG AGRI.

The guideline seeks to plausibly present the legal basis as well as the logic and objective of the evaluation for all levels and derives possible evaluation methods from it. This is a very high requirement and very well done for many aspects. However, from my point of view, it is almost impossible to do justice to all target groups. Perspectively, therefore, individual handouts could be more targeted, which are customised to the specific target groups. The guideline provides a very solid basis for this and extraction and processing of the individual parts could be a future task of the help desk. To increase the usability at national and local level, it is furthermore inevitable to translate the guideline. [The evaluation guidelines have now been translated into 22 languages].

As a general remark about evaluation in LEADER I want to point out a danger of a one-dimensional view of the evaluation from the viewpoint of the evaluators: the impression arises that the evaluation could provide a very accurate, detailed and objective picture of the effect of all imaginable measures. However, the assessment always remains an attempt to approach reality. Evaluation is therefore not suitable as a means to justify political decisions, but to expose undesirable developments that can be avoided by an appropriate response. Just as the illusion of almost complete flawlessness in EAFRD or LEADER funding has led to an unmanageable level of regulation, there is the danger that the illusion of complete controllability and accountability will escalate the methods and effort of the evaluation.

Especially with the mostly very small LEADER projects, it is not possible to reliably measure direct effects, for example on the number of employees, tourists or biodiversity indicators in meaningful size units. They are usually additionally heavily overlaid by large-scale development tendencies. With this the proportion between subsidies and evaluation effort can be completely out of balance. But even with very high efforts there is not only the danger that the impact indicators cannot be realistically recorded, but also that wrong conclusions are drawn because overlay factors are not recognized.

The effect of LEADER measures is often very long-term, e.g. the development of a culture of cooperation or the strengthening of regional identification. The long-term impact on jobs can be much more effective than promoting an investment in a single enterprise, which, however, directly contributes to the evidence of a new job. The isolated consideration of these supposedly easily measurable indicators can lead to a misinterpretation of the effect of measures.

In principle, the evaluation approach at the LAG level should focus on recording the implementation of the LEADER method:

1. Adequate participation of regional actors in the formulation of the strategy and its implementation.
2. LAG animation and capacity building
3. Multi-sectoral approach and cooperation

This focus has already been implemented in the guideline in essential approaches and makes it to a useful tool. Overall, however, the proportionality between evaluation effort and informative value must be an even clearer maxim.

Dr. Hartmut Berndt

German LEADER Association, BAG LAG e.V.

 

I have a few general comments:

• The guidelines are an excellent guide for both managing authorities and local action groups, and it’s good to have all the guidance in one place so each can see what the other should be doing
• The guide might be a bit daunting for an inexperienced LAG member to tackle alone, but as a guide for accountable bodies and as a basis for LAG development and self assessment within a group session I can see it being invaluable. It might be useful to have a very short summary version or even just a flowchart showing the evaluation process to circulate to LAG members in advance of such a session or as an aide memoire
• I do find it a bit of a shame that evaluating the added value and increased social capital are not mandatory – for me these are the absolute bedrock of CLLD/LEADER, and managing authorities should not be able to avoid these evaluations. I can see that the current England LEADER programme would do quite well in an evaluation of its contribution to wider RDPE objectives because it is so structured to ensure this, but its added value is somewhat debatable and the LAGs’ ability to innovate and cooperate are severely impacted by the lack of flexibility – under current requirements the Managing Authority won’t need to consider this within their evaluation, although armed with this guidance we will certainly be encouraging them to do so.

Claire Watts

External Funding & Policy Manager, East Riding of Yorkshire Council*

*Accountable Body for the Coast,  Wolds, Wetlands and Waterways LEADER programme, the Holderness Coast FLAG programme and the Yorkshire Coast CLLD programme

 

Implementing Leader in European Rural areas has proved to be a successful approach in order to achieve Local Development Results. This is probably the reason why in the past few years the principles of LEADER methodology have become a widespread theme of discussion in various EU funds and processes.

LEADER is in its fifth implementation period and has gone from the EU Initiative-pilot action stage to a mainstreaming stage and now back again, there were many voices debate on going back to its roots.

There has been significant experience earned in implementing LEADER in the past 2 decades but still the current period poses challenges in evaluating LEADER. There has been early on criticism on the heavily administrative burden and at the same time on the scope of Evaluation of LEADER.

Especially as LAGs and FLAGs that operate on the field there is often a question and even criticism on whether we evaluate in order to make sure we see how we can get results or we evaluate just for the sake of administrative reporting.

A good evaluation system is a challenge for a LAG, and LAGs with long experience have come to realize its importance in documenting the impact of their work.

The LAG/FLAG in order to be able to achieve the important and high-stakes role it has needs to apply quite a lot of essential resources and skills and in one entity incorporate many different concepts and tasks.

• Availability & commitment, Broad range of relevant experience, skills & knowledge
• Experience & recognition of the area and the economic sectors and social variances, knowledge of the territory
• Representativeness and influence in different sectors of the local community
• Ability to communicate the LDS objectives and opportunities in the community
• Be an agent of participative dialogue and exchange of ideas employing ability to cooperate with others
• Create the sense of a team – be a team player, Balance between men & women; young & veterans
• Political neutrality, Open minded, curious and holistic vision
• Experience in sustainable territorial development
• Technical experience to support project development and knowledge of relevant regulations and legislation, administrative capacity to manage (EU) public funds
• Strong communication skills: capacity to listen & understand; patience and objectiveness; negotiation skills and leadership.
• Ability to work in a bottom-up approach: team working; networking; imagination for new solutions; ability to motivate others.
• Ambition and committed to making a difference be accessible, be available be flexible.

The above mix of skills applied by the LAG through its dedicated resources, its Staff members and Partnership representatives, lead to the necessity to create an environment of Trust, Credibility, Legitimacy and Decency in the community.

All of the above is in essence the way a LAG works and affects the Social Capital of a rural, insular or mountainous area. The LAG is the carrier of the Local Development Strategy which should be the product of free local deliberation, supported by wide partnerships, the vision for an area that aligns people and local forces. All of the above is what we locally see as the added value of LEADER in our areas and it is important to evaluate.

So how can we evaluate LEADER added value and impact on an area? The “Evaluation guidelines of LEADER/CLLD”, provide a significant methodology and insight. But is that enough? Can the LAGs understand it -especially new LAGS exposed in the methodology for the first time- or navigate through the complexity of LEADER evaluation?

There is quite some debate on the issue; there is a lot of interplay between two main groups of interest, the Organizational and Administrative and the Local Community and stakeholders.

It becomes more and more apparent that the conventional methods that measure results and impact, the conventional indicators do not capture the underlying tendencies and effects on a local community and as a result the conclusions we derive in various cases, they could even be misleading or missing the true effect of an intervention in an area.

The LAG comes in the equation as an intermediate; the LAG becomes the facilitator that has to translate the local needs to a Local Strategy, to measures and results that fit the objectives of Rural Development Programs and EU priorities, therefore it is important to have good advance planning and preparatory work by Managing Authorities, quite a lot of training and awareness raising of all the actors that participate in the delivery system, and good communication in all levels in order to establish common understating.

Finally in my view it is important to be free to plan and implement in an area, of course not without accountability, therefore a good evaluation system in my view validates the demand to be able to act freely.

Anastasios M. Perimenis

LAG & FLAG Manager in Lesvos

General Manager, Lesvos Local Development Company S.A. – ETAL S.A.