Multi-criteria analysis aims to compare different actions or solutions according to a variety of criteria and policies. The method is based on the evaluation of actions by means of a weighted average. Multi-criteria analysis is not a method specific to Foresight exercises.
This method can be used to select or hierarchize solutions. Thus in a Foresight exercise it is mainly used in the strategy phase to choose, for example, which policy option fits the main features (criteria for the tool) of the project best.
It can also be used to find out which of the exploratory scenarios best matches decision makers’ expectations. But this means that decision makers have to agree or reach a consensus on a weighted set of criteria with which to judge the performance of the project. Using only part of the method, by only reaching an agreement about the weighed set of criteria to define a good project, might also be a way to help build a desired future.
The method can also be used in a Foresight study, to hierarchize the different technological solutions for example to produce energy as cheaply as possible or with the least greenhouse gas emissions.
The method can be used in the foresight working group to reach a consensus about the best project or scenario. It might be used by a group of decision makers to define which policy is the best to achieve a goal defined by several criteria. But generally an expert prepares the weighed set of criteria for them and the decision makers just grade each policy option or action according to these criteria at a meeting. A group of experts generally prepares the set of weighed criteria for a group of decision makers. But for important decisions the group itself might prepare the list of criteria and give to each one a weighting according to its importance.
When building the weighted set of criteria or grading the actions according to the criteria, it is easier to work with a small group because a consensus is needed if it has to be done in a short time. For a more participatory process, the expert group prepares the set of weighted criteria and questionnaires are used to grade each action or policy according to these criteria.
Approach
1. Decide the list of criteria that matter for the choice. These can be criteria such as the cost, the implementation delay, expected benefits, etc.
2. Give a weight to each of the different criteria. If this is done by a group, you can allow 10 to 20 points per person (depending of the number of criteria) and ask each one of them to distribute their points to the different criteria according to their importance. Generally giving more than 5 points to a single criterion is forbidden. But sometimes a group can reach an agreement without voting. The weight for each criterion generally ranges from 1 to 5 or from 1 to 3.
3. Draw up the matrix
Criterion 1 Criterion 2 Criterion 3 Criterion 4
Option 1
Option 2
Option 3
1. On a scale from 0 to 20, grade each option for each criterion. If option 1 is better than option 2 for criterion 1, its grade should be higher; if option 3 is better than option 2 and lower than option 1 for criterion 1, its grade for criterion 1 should be in between the grade of option 1 and 2. The grading step can be individual (by questionnaire) or done during a meeting group. If grading is done during a meeting, according to the size of the matrix, extra time should be allowed to discuss and reach consensus on each grade.
2. Compute the average score for each option. In each column the grade obtained for each option is multiplied by the weight of each criterion. The score of each option is found by summing up each line by adding the grades multiplied by the weight of each criterion. The option with the highest score fits the multiple-criteria analysis best.
Resources
The time required to use this tool depends on the size of the matrix (the number of criteria x the number of options to choose from) and if the grading is done by means of a questionnaire or during a meeting.
The most difficult part is certainly to decide the weight for each criterion. But for an expert group it can generally be done in a half day.
If grading the different options is done during a meeting, a chairman will need to be appointed to manage the time each option is discussed and help reach a decision.
Outputs
The output is a hierarchy of the different options that can be argued for on the basis of the scores obtained by each option for each criterion.
Pros and cons
Multiple criteria analysis is a simple method suitable for testing the robustness of the results of different policies or options. It can also allow users to incorporate new criteria, weightings or actions easily, not only during the survey but after it as well, with a view to enriching the analysis. The simplicity of the aggregation criterion (weighted average) is certainly an advantage.
On the downside, if the goal is to draw up a plan based on several actions, difficulties might appear because of any synergies, incompatibilities and redundancies among the actions retained that need to be taken into account. Other and more sophisticated methods should then be used.
Another obvious requirement is that all the options tested must be judged on the basis of the whole set of criteria. If a criterion is not relevant for an option, either the criterion or the option has to be changed.
Variations, Complementary methods
To help use this tool and avoid drawing up the matrix and computing the scores, a software called ‘Multipol’ can be downloaded from the LIPSOR (prospective and strategy of organisations investigation laboratory) website [www.cnam.fr/lipsor/].
Conclusion
The checklist of questions to use this method could include:
• Is there a choice to be made?
• Do I have comparable options or scenarios to choose from?
• Is there a decision maker or a group that feels allowed to set a list of judgement criteria?
• Is the comparison of options so complicated that I need a structured method?