Methodology

Choosing and applying the right method for a specific exercise is a crucial step in any Foresight process. In choosing among the wealth of methods, it is important to understand some key differences:

• Qualitative vs quantitative methods
• Predictive vs non-predictive methods
• Exploratory vs normative methods

The table below classifies selected method according to the attributes defined above and to the exercise phase they are most suited to.

Does the method meet my objectives?

When deciding on the methods to be used at a certain point in your exercise you will usually start from the objectives of each phase and the outcomes you expect. You will probably consider a range of methods that might be able to fulfil this particular function (as outlined in “Assigning sets of methods to functions“) by carefully looking on the description of various methods as they are described in the table of methods in this guide e.g. in the “Is it for me?” section of the description. From that you will have to determine which method is best suited to achieve your targeted outcomes (formal and informal!).

Important criteria to be considered when deciding whether a method is suitable for your exercise are the Level of participation on the one hand and the degree of expertise you need to achieve your objectives on the other. You should keep in mind that this is not naturally the same for each step of the exercise. A largely participatory exercise is indeed not necessarily participatory at each phase. It might well be that only a few experts are taking part in a specific phase while large-scale participation is envisaged in other steps. (See also: The use of expertise in Foresight).

In defining the methodology we distinguish between the methodological framework (i.e. the backbone of the exercise) and the actual methods that populate it, which can be drawn from the wider family of Future-Oriented Technology Analysis (FTA). In other words the framework can be defined as a set and sequence of methods to generate, elicit, structure, synthesise and capture the information at different stages of the process.

• Description of methodological framework
• Description of (FTA) methods used in foresight exercises

The overall methodology should be defined for the exercise early within the design phase. It should be content and results-driven and should also take into account the resources constrains (stakeholders availability, exercise duration, working resources, etc.). Nonetheless, the methodology is evolutionary and will be re-defined throughout the process, subject to discussion with the team and stakeholders.

To be guided in crafting the appropriate methodology it is critical to think of methods as serving different functions within the broader foresight exercises, as explained in the page Assigning methods to functions. Furthermore, once needs to evaluate whether the method is appropriate with respect to the objectives and the resources of the study.

Methodological Framework

The design of the methodological framework is the backbone of any Foresight exercise as it allows a structured collection of anticipatory intelligence, which differentiates Foresight from loose reflection on the future.

As a Foresight exercise may make use of different (sets of) methods at different stages, finding the appropriate sequence of methods is one of the most delicate design steps. Indeed, it is an evolutionary process that needs continuous adaptation and in depth discussion with the sponsors, the team and other stakeholders. In this respect it might be useful to distinguish the different functions that need to be performed in different phases of your exercise:

• Diagnosis: Understanding where we are
• Prognosis: Foresighting what could happen
• Prescription: Deciding what should be done

These functions might relate to specific phases of the process or they might come up at more than one point in time during the exercise. One way to define the methodological framework is to assign specific methods to fulfil specific functions (See the table in the methods page).

Another thing to take into account is the potential tension between the various orientations of Foresight. This ambivalence is problematic as there is a danger that exercises are started with misguided expectations as to their impact and outcomes. It seems crucial that within Foresight design there is a clear understanding about what type of impact is being targeted within each particular Foresight activity.

Assigning methods to functions

Three basic functions of Foresight methods can be distinguished:

• Diagnosis: Understanding where we are…
• Prognosis: Foreseeing what could happen…
• Prescription: Deciding what should be done…

These functions may be confined within particular phases of the exercise so e.g. diagnosis will often be carried out in the beginning while prescription will be done towards the end of an exercise so an exercise might involve the following phases:

1. a phase to understand the current situation (Diagnosis)
2. to continue with a phase to explore what can happen (Prognosis)
3. and to end up the exercise with a phase to define recommendations about what can be done (Prescription)

However, particular functions could also take place during several phases.

For each of the functions you may have to choose among a variety of available methods such as those outlined below. In each phase there are several possible ways of combining the methods. For example, the choice between the rigour of quantitative methods and the qualitative insight into social processes and structures can be avoided by a judicious application of each, most commonly the latter providing the framework, and the former generating relevant empirical data.

Approach

There are different ways of approaching a Foresight process once the focus and scope have been decided upon. Types of approach to designing a Foresight exercise:

• Top-down approaches work from a fixed procedure. Usually small panels of experts drawn from different stakeholder-groups work on information gathered from a wide range of sources. Formal methods are used to structure the process.

• Bottom-up approaches put more stress on interaction. The process of information gathering as well as the dissemination and implementation of the results are itself subject to the discussions.
There is more information on the approach to design decisions in some of the example cases:

• Vision 2023: Turkish National Technology Foresight Exercise

• Futur – the German Research Dialogue

• Eforesee Malta – Exchange of Foresight Relevant Experiences among Small Enlargement Economies

• Product-oriented approaches produce tangible outcomes like reports, priority lists, scenario descriptions, action plans, checklists etc. These products might be aimed at narrow circles of decision makers who commissioned the project but can also address a wider audience or the general public.

• Process-oriented approaches emphasise intangible outcomes such as network building and learning processes as the main outcome of the exercise. The goal is to achieve increased receptivity to signals of change and an enhanced understanding of how and where to access critical resources thereby improving the preparedness for action. Furthermore the building of a Foresight culture in organisations and constituencies is an important aim of process oriented Foresight.

Choosing the best mix of approaches

The particular balance of approaches to Foresight chosen should depend, of course, on the specific circumstances. This may seem to be an obvious point. Nevertheless, there have been several cases of an approach simply being copied from one context to another, without adequate appraisal of how the approach might need to be modified or restructured to deal with the new circumstances. In some of these cases this has led to major failures in the Foresight process, and a loss of political support for Foresight.

What can be chosen, in practice, is going to depend upon various political circumstances. It may be that senior policymakers have an entrenched idea of what Foresight should be; they may even be seeking to emulate high-profile programmes that have been undertaken elsewhere. There may be high-level doubt that a broad public could have anything of value to say about important topics, or it may be feared that a process of broad consultation could exacerbate existing ethnic, political or civil strife. Results may be required to inform extremely pressing policy decisions, or to convince international aid or loan agencies that serious strategic analyses have been conducted. This highlights, among other things, the need for scoping and feasibility studies prior to embarking on a major exercise. Such studies will often be required to arrive at an exercise ‘plan’, and they are also important for convincing actors of Foresight’s merits – as can other strategies, such as soliciting contributions from international experts to pre-Foresight meetings.

We may have to accept that something less than the ideal is all that can be achieved in current circumstances. To the extent that we do a good job now, explain as far as possible why particular choices have been made, and make our case for doing things differently in the future, there may be hope for continuous improvement in Foresight practice.

Time Horizon

Carefully defining the time horizon of the Foresight exercise is essential:

• Different time horizons are suitable for different types of subjects. When the exercise is operating on a time horizon that does not match the dynamics of the subject it will not be useful

• For many of the formal methods that can be applied to support the Foresight exercise it is extremely important to be clear about the time horizon that is to be addressed. Without this information the methods cannot be applied adequately.

Criteria for defining the time horizon

The time horizon of a Foresight exercise should lie beyond the normal planning horizons of the players involved but close enough that it could still be influenced by today’s decisions. In other words it should be far enough to allow changes to be possible but not so far away as to seem irrelevant.

This criterion depends very much on the sector. Within the public sector, the time horizon usually tends to vary from around 10 to 20 years. But in the case of infrastructures (like power stations, transport networks, etc.) the time horizon might be 30 to 50 years. Within the private sector, the normal planning is generally one generation of the product or service ahead, thus a good time horizon for corporate Foresight might be two generations of the products and services. Nobody is yet working on products and services two generations ahead, so whatever the outcome nobody’s current work will be disrupted.

An exercise which is more action-oriented will tend to have a shorter time horizon. A 5-year time horizon tends to be used in technology-oriented exercise in fields where technologies are developing very quickly (e.g. Information & Communication Technologies).

An exercise which is more vision- or creativity-oriented will have a longer time horizon. A 20-year time horizon is necessary to take into account longer-term changes such as demography, social values, the rise or decline of economic powers, environmental degradation, global warming. Other factors that should be taken into account when defining the time horizon:

• The inertia of the system and the need to blur the periodic effects that generate turbulence (which can lead to the system’s being misunderstood)

• The schedule of decisions to be made, the power to decide and the means to be used (note that drafting a strategy is useless if the means to implement it are unavailable: for example if the investment budget is allotted for the next ten years, a ten year Foresight exercise is useless since there is no room for manoeuvre)

• The degree of rigidity and motivation of the players

In the end, there is no secret recipe. Common sense and pragmatism are needed when choosing the optimal horizon.

Exploratory vs. Normative methods

Exploratory methods begin from the present, and see where events and trends might take us; normative methods begin from the future, asking what trends and events would take us there.

A fundamental distinction in futures and forecasting studies is commonly drawn between exploratory and normative methods. This terminology is well-established, but rather misleading (since both approaches involve exploration, of course, and both call into play questions about norms and values).

Exploratory methods

Exploratory methods are “outward bound”. They begin with the present as the starting point, and move forward to the future, either on the basis of extrapolating past trends or causal dynamics, or else by asking “what if?” questions about the implications of possible developments or events that may lie outside of these familiar trends. Trend, impact, and cross-impact analyses, conventional Delphi, and some applications of models are among the tools used here. The majority of forecasting studies are mainly exploratory, though when these result in alarming forecasts, there may well be an effort to locate turning points or policy actions that could create a more desirable future.

Normative methods

Normative methods are, by contrast, “inward bound”. They start with a preliminary view of a possible (often a desirable) future or set of futures that are of particular interest. They then work backwards to see if and how these might futures might or might not grow out of the present – how they might be achieved, or avoided, given the existing constraints, resources and technologies.

An common example of a desirable future that can sustain a normative approach is the Knowledge Society envisaged in the European Commission’s Lisbon strategy.

The tools used here include various techniques developed in planning and related activities, such as morphological analyses and relevance trees, together with some uses of models and some less conventional uses of Delphi such as “goals Delphi” methods. A fairly recent development is the use of “success scenarios” and “aspirational scenario workshops”, where participants try to establish a shared vision of a future that is both desirable and credible, and to identify the ways in which this might be achieved.

What is the right mix?

There is little evidence as to when each of these approaches is most valuable, and again in practice we often find Foresight involving a mixture of the two. It may be that more normative approaches are most likely to be effective where a widely shared goal already exists, and where Foresight can then help put flesh on the bones of this implicit vision of the future. For example, a common long-term regional goal may be for more rapid and equitable economic development in the region; or where S&T issues are at stake, it may be to achieve a secure grip on at least some niches of technology innovation, production and use. In such cases, normative approaches can be powerful inputs into priority-setting and other elements of decision-making (and help provide road-maps and indicators that can be used to monitor progress towards the desired future). In other cases, normative approaches may be considered insufficiently objective, or there may be a lack of consensus as to shared goals, at least in early stages of the Foresight process. Exploratory methods can then be expected to predominate.

Quantitative vs. Qualitative methods

Foresight usually draws on both quantitative and qualitative approaches as they provide distinctive inputs to the analysis of the problems being dealt with.

Quantitative methods

Quantitative methods place greatest reliance on representing developments numerically. Numerical data, of many types, are useful in thinking about longer-term developments, and to a certain extent they can be useful ways of presenting Foresight results, too.

As in in econometrics, quantitative methods implicitly or explicitly use simple models of some sort. For example, time series extrapolations of trends imply a model in that use time as the “independent variable” – really, as a proxy for unmeasured processes that take place in time. More complex models relate variables together so their mutual influences can be tracked. In so-called dynamic models this is tracked over time, whereas equilibrium models often employed by economists assume a move from a present state towards a (presumably more balanced) future state. Some quantitative approaches involve experts putting numerical values to developments, or creating such values on the basis of the numbers of people agreeing with particular statements or forecasts (as in Delphi).

Data may be generated in various ways. Secondary data is data that was generated for other purposes, but which we can re-use in our own work – often we can use secondary data from official statistics or academic sources. Sometimes we need to generate our own primary data. The most common sources of data are sample surveys (in which a proportion of a population is systematically sampled: a fairly small proportion can give results that are good estimates for the whole population), or censuses of the population. Many statistics are generated by means of questionnaires and other surveys, where the people concerned are requested to provide information for data collection purposes. Otherwise, data may be “captured” from various sources – as a by-product of people’s contact with tax, health or other authorities, and the records that these produce; or from other sources which in some way “capture” their behaviour. (For example, a new source of data is websites, and it is, for example, possible to track the growth of activity in a particular field in various regions by counting up and examining the websites addressing the topic.)

Once we have data in a numerical form, there are many quantitative techniques that can be employed in the course of the Foresight exercise. Many statistical tools are employed to determine the relationships that can be found between variables and most good basic textbooks of statistics and data analysis will discuss these techniques and more fundamental procedures such as how to represent averages, trends, etc.

There are considerable advantages to using quantitative methods, which account for the great interest in them. Being able to put information in numerical form means that:

• It is possible to manipulate the information in consistent and reproducible ways, combining figures, comparing data, examining rates of change, etc. This allows for much greater precision than simply talking about increases/decreases, etc. As an accounting tool, numerical data can help us to engage in basic accountancy-type testing of the consistency of different elements of the whole, so that, for instance, we do not plan to spend the same money twice over, or to work for more than 24 hours a day, etc.

• It is possible to process the relevant data in systematic ways to produce trend extrapolations and other forecasts.

• It allows comparison of the scale of developments in various circumstances (e.g. estimates of the numbers of people in different areas, who might be suffering a disease, are in need of housing, etc. Such comparisons can inform decision-making in significant ways – for instance, helping to validate or undermine claims from particular interest groups about how more serious their problems are than those of other people. (But remember that statistics can only inform, not substitute for political decisions.)

• Results can be represented in the form of tables, graphs and charts, which can often communicate very efficiently with people under severe time-shortage and information-overload.

They also have notable disadvantages:

• Some factors cannot be represented numerically, yet this does not mean that they are necessarily less tangible, less significant, or less amenable to serious analysis or appraisal within Foresight.

• The quantifiable elements of a phenomenon should not be taken as encompassing the entire phenomenon (or even all of the most important features of the phenomenon) – but often they are, and often, for example, attention of policy-makers or executives will be focused mostly on the graphical elements of a report and qualitative elements within the text will be disregarded.

• Good quality data are often not available or not sufficiently up-to-date to inform a Foresight exercise – and the production of new data may be costly or excessively time-consuming. For example, the production of structural analysis matrices alone may require the mobilisation of dozens of experts over several weeks or months.

• The spurious precision can give a misleading impression of the amount of knowledge really available on the issues in question.

• They can hinder the communication with less numerate audiences. Not everyone is comfortable with working with or even reading statistical information, and some people are extremely suspicious of “lies, damned lies and statistics”, knowing that often so-called hard facts are actually misleading – based, for example, on inappropriate samples, using inadequate indicators, or being misinterpreted in various ways. Certainly it is important to use reliable sources (e.g. official statistics) and to seek the advice of independent experts as to the use and presentation of such data.

• The excessive formalisation can lead to decreasing levels of involvement by the participants, notably when complex group techniques are used. Some advanced statistical methods and modelling techniques are highly complex, and relatively few people are able to scrutinise or challenge the assumptions that are being made in using them. Experts are also wedded to one or another type of method, and discount other experts’ reservations as to their uses and limitations.

Qualitative methods

Qualitative methods are often employed where the key trends or developments are hard to capture using simplified indicators, or where such data are not available. Methods for working systematically with qualitative data are becoming more widely available with the development of Information Technology – tools for “mind mapping” and “conversation analysis”, etc. – which can also be helpful devices for facilitating meetings and workshops. For many years the development of qualitative methodologies in social science, as well as in forecasting and Foresight, has lagged behind that of quantitative approaches, and there has often been an explicit or implicit reliance on experts to pull together the strands of qualitative analyses and come up with a synthesis by more or less intuitive means. In the last decade or so this situation has improved considerably, and many tools for capturing and analysing qualitative data, and processing and representing results of such analyses, have become available.

Mixing Methods

The methods above are just components of the larger foresight exercise. The exercise framework will include several of them and an appropriate choice of methods depends on a careful evaluation of the strengths and weaknesses of each tool in light of the exercise objectives. Particularly important in the debate of mixing methods is the combination of qualitative and quantitative approaches, discussed here.

Mixing qualitative and quantitative methods

Foresight and FTA bring together different disciplines that try to understand and shape the future. As this online guide shows, these include both qualitative and quantitative approaches.

Whilst both are acknowledged as important and useful in preparing us to address the future, only in few cases both approaches are combined and the divide between quantitative and qualitative practices in the field remains to a large extent unsolved. Although, current trends in the discipline, combined with the increasing policy demand for robust evidence for decision making, indicate that there may be a momentum for pushing forward such methodological integration, many barriers persist.

Epistemologically, there is a long-standing debate, not confined to the FTA, on the type of knowledge that qualitative and quantitative methodologies can produce and on the value of combining them. A layer of complexity is added to this debate when the future is the object of the analysis. The different communities involved in FTA (foresight / forecasting / technology assessment / futures communities) are still perceived as independent, competing rather than collaborating. For instance, the idea that one can forecast future seems to be contradicting with the idea that it can be (jointly) shaped using foresight.

These difficulties reflect deeply ingrained cultural differences that hinder good communication at the outset. Efforts to address this cultural clash have often focused on the adaptation of the methods and tools commonly used by the two communities, in the hope that adequate interfaces between the two could be found and thus facilitate operational collaboration. However, such efforts have not been able to promote a new culture that inherently mixes qualitative and quantitative approaches. Rather, one could paradoxically observe that they have led to the consolidation of the historical practice that sees the two communities working in isolation: each can safely remain in their own cultural realm, so long as a mechanism has been identified to ensure some level of communication.

The aforementioned epistemological and cultural obstacles are reflected in and reinforced by the lack of skills: researchers, practitioners and evaluators acquainted in both quantitative and qualitative FTA approaches are not common. All in all, these barriers undermine trust among practitioners from different schools and towards the community as a whole.
To overcome such barriers one could consider the following short-term research agenda:

• Look at areas where convergence of quantitative and qualitative methods might bring potential benefits, such as scenario development combined with ex-ante impact assessment, planning future studies, designing advanced models of macroeconomic forecasting, anticipation of disruptive technologies, use of quantitative methods for creation of a background for expert judgments, forecasting in a technologically complex environment, future market assessment and the estimation of forecasting bounds of confidence and uncertainty.

• Use new methods and tools that are entering FTA to increase synergies between qualitative and quantitative approaches (see new tools)

• Explore what is done in other fields on methodological combination (see sectoral studies, or other disciplines).

EFMN Brief No. 137_ Manufacturing in Europe Manufacturing in Europe is facing challenges that may impact on its performance in the near future: the emergence of international competitors, new technologies allowing the emergence of new business models, increased off-shore and relocated activities. The aim of this study was to provide policy-makers with a long-term vision of European manufacturing, its characteristics, its place in the EU economy, in the world and the main challenges it will be facing. Its purpose was to identify, on the basis of current demographic, environmental, technological, economic and social trends, and possible scenarios, the likely bottlenecks, unsustainable trends and major challenges that European manufacturing will have to face over the coming 30 years. From this, implications for various microeconomic policies, notably for industrial policy, were explored, contributing to the mid-term review of industrial policy in 2007 by the European Commission’s Directorate-General for Enterprise and Industry.

Predictive vs. Open methods

Predictive methods

Predictive methods are primarily those that envisage a single future considered most likely, whether this future is desirable or not. To this extend, a deterministic method may or may not be normative. The methods stemming from the forecasting and from the planning schools of thought are mostly predictive.

Open methods

Open methods envisage different possible alternative futures. Open methods are usually exploratory but this is not necessarily the case if alternative ways of reaching a desirable future are considered. Core Foresight methods are essential open?

While many people know about the specific topics addressed in Foresight activities, relatively few people have a well-informed view of the longer-term developments that are likely or possible in these topics. Experts should be able to address more fundamental questions, and to know of what problems, innovations, and opportunities are arising in their areas. Sometimes these will be practitioners, sometimes researchers. Some experts know a great deal about their subject area but are relatively narrow, with little knowledge of developments even in adjacent fields; some find it difficult to communicate with non-experts, and some are convinced that their sort of expertise alone is sufficient to address all of the problems posed by Foresight (so that there is no need to encourage wider participation – the only issue is seen as being a matter of disseminating their own views).

Often it will be necessary to sample a broad range of expert opinion, to inform the Foresight activity. There may be various reasons for this:

• Critical knowledge is widely dispersed
• Someone may possibly have knowledge of relevant material that is not yet common knowledge, even among experts
• It is useful for identifying recruits for networking activities
• It can contribute to reinforcing the legitimacy of the exercise

Consultation – through questionnaires, workshops, Internet, etc. – may be carried out at a number of points in the Foresight process. On other occasions it will be necessary to work more intensively with smaller groups of experts, to stimulate dialogue, to deepen the analysis and produce reflective conclusions.
More information: See Selection of participants

Experts may be:

• Used as a “passive” source of data, so that their views are elicited and collected, but they have little say in these processes

• Involved much more interactively, so that they play a more creative role in determining what knowledge is relevant and how it could be used.

Putting these two dimensions together, we can locate some of the main techniques as follows:
Experts are: Remotely sampled, Physically present
Mainly passive: Conventional postal surveys (e.g. most Delphi studies)
Interviews: Attendance mainly as observers at workshops, conferences
Delphi and similar surveys as group events at conferences, etc.
Highly interactive Participation in computer conferences, remote group working.
Expert Panels
Scenario building
Brainstorming

Specific methods that can be used to capture expert knowledge include:

• Delphi survey
• Expert panels
• Brainstorming & Mind mapping
• Scenario building

Below you can find some examples of foresight exercises that made use of experts.

EFP Brief 188: Improving Foresight through Methodological Innovation We present insights into the design and execution of an international large-scale project on the future of logistics by the year 2025. The basis of our research was an innovative real-time Delphi application. We applied a multi-methodology framework including a real-time Delphi, a futures conference and participatory expert workshops. This allowed for cross-validation and a strong participatory inclusion of policy makers. An example shows how a multi-stakeholder environment can be approached using innovative foresight tools. We illustrate a research case study that aligns foresight activities with a rigorous scientific procedure.