 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
It is important for policy makers to have insight if their carried out policies are functioning as expected.
One can distinguish ex-ante and ex-post evaluations, respectively looking forward and backward on the effects of
policies and instruments. All members of the consortium of the AID-EE project have excellent expertise in evaluating
energy and environmental policies and measures. The background of evaluation and the used method in the AID-EE project,
called theory-based policy evaluation, is shortly described below. For detailed information and related evaluation methods
see Documents and Useful links.
Theoretical background
Theory-based policy evaluation
In a perfect policy cycle first policies are formulated, in the next step policies are implemented and ultimately policies are evaluated to determine their effect(s). After policies have been implemented they should be monitored and evaluated, and results of the monitoring and evaluation process might lead to a reformulation or even an abolishment of policies. Ex-post policy evaluation is an essential element in the policy cycle (see figure 1).
Ex-post policy evaluation in principle boils down to answering the following two questions:
· What was the contribution of policy instruments in the realisation of policy targets (effectiveness of policy instruments)?
· What was the cost effectiveness of policy instruments, and could targets have been reached against lower costs?
These questions can be answered at two levels (i) at the programme level (in case a package of policy instruments
aimed at reaching a specific target is evaluated) and (ii) at the instrument level (in case one specific policy instruments is evaluated).
Figure 1 Outline of the policy cycle and the role of the program theory in the policy cycle
The theory-based approach is not new and has been used numerous times to evaluate policies. The method of theory-based policy evaluation is extensively described and illustrated (Rossi et al, 2004) . Rossi, P.H., M.W. Lipsey and H.E. Freeman, 2004, Evaluation, A systematic Approach, 7th ed., SAGE Publication, Thousand Oaks, CA, USA
In the energy sector the method was used to design, evaluate and adapt market transformation programs in the field of energy efficiency in California (Blumstein et al, 2000). Though overall, the application in evaluating energy efficiency policies has been limited and the method was so far not applied in a systematic way in the policy implementation process.
In practice, theory-based policy evaluation boils down to establishing a plausible theory on how a policy instrument (or a package of instruments) is expected to lead to energy efficiency improvements. Application of the theory-based approach in ex-post policy evaluation means that the whole policy implementation process is unravelled to evaluate the effectiveness and cost-efficiency of the different steps in the implementation process. Through this unravelling insight is gained on where something went wrong in the process of policy design and implementation. This findings leads to the keys for improving the effectiveness and cost efficiency.
Most methods used in ex-post policy evaluation of policy instruments focus on final effects i.e. energy savings and CO2 emission reductions.
Characteristics of the theory-based policy evaluation method compared to other ex-post evaluation methods are:
· Evaluates the whole policy implementation process and not just focuses on the final effects (i.e. efficiency improvement
and CO2 reduction).
· Quantifies, to the extent possible, the successes and failures of policy instruments through the development of indicators for
each step in the implementation process.
· Provides insight on why policies succeeded or failed and how they can be improved.
