|
Evaluation
and validation of DIS4ME
Authors:
Jorge García Gómez <jorgegg@um.es>
and Nichola Geeson <desertlinks@medalus.demon.co.uk>
g
Introduction
Throughout the DESERTLINKS project,
workshops with stakeholders in the Target Areas have been used to discuss
requirements of an indicator system and the merits of individual indicators.
The process of interaction between scientists and stakeholders continues,
as more people have the opportunity to find out about DIS4ME and contribute
to improving it.
Stakeholders include:
- Farmers and farming organisations.
- Local and regional administrative
authorities, representative technicians in city councils, regional environmental,
land management, agriculture. Technicians from cooperatives and trade
unions.
- Civil society, NGOs, environmental
associations, private foundations, local associations; teachers and
students at universities, colleges and schools.
- Scientists from disciplines related
to desertification.
The evaluation process has
included:
- Workshops in the local language
in the Target Areas, workshops at specific project meetings and other
international meetings, and a workshop with the GHEA Masters students
at the Università della Basilicata.
- Questionnaires, circulated to stakeholders
in the Target Areas; representatives from national Focal Points; representatives
from the UNCCD and Committee for Science and Technology; other scientists
in parallel projects; and everyone else who has asked for the password
to view DIS4ME.
5
top
g Basic
questionnaire design
The questions included:
- Is DIS4ME simple to use?
- Is the structure of DIS4ME logical?
- Is DIS4ME comprehensive?
- Is the level of detail sufficient?
Guidelines were proposed to develop
a suitable questionnaire:
- Questions should be kept simple,
addressing one idea at a time.
- Closed questions, with a specific
range of response, allow responses in the different Target Areas to
be compared. Some open questions are valuable too of course.
- Allowing people the easy option
of giving non-committal answers is best avoided, i.e. it is better to
have four classes for responses than three.
- Details of the persons answering
the questionnaire must be recorded, e.g. male/female, their age range,
the group or organisation they belong to, scientist/non scientist.
Basic questionnaire
Closed Questions |
Good |
Fairly good |
Not good |
Poor |
Is it clear what the purpose
of DIS4ME is and how it can be used? |
|
|
|
|
We have tried to present the
information in a popular scientific style. How clear did you find
the language used? |
|
|
|
|
How easy was it for you to navigate
around DIS4ME? |
|
|
|
|
In general, how well does DIS4ME
address your particular requirements as a scientist, policy maker,
farmer, or student, etc.? |
|
|
|
|
Did you find that DIS4ME helped
you to better understand the processes of desertification? |
|
|
|
|
If you have ticked the "not
good" or "poor" box for any question, please explain
your reasons and suggest how improvements can be made. |
|
|
|
|
Open questions |
1. Do you have any
suggestions for improving the structure or content of DIS4ME? |
2. Can you suggest
any areas where the information (e.g. descriptions of indicators or
processes) should be expanded or clarified? |
3. Are
there any specific desertification problems that we are not addressing? |
4. Have you used
any of the tools (i.e. ESI tool, tool to calculate desertification
risk from salinisation or erosion under various land uses) to assess
desertification in your own area? If so, how accurate or appropriate
were the results the tools gave? |
5. Have you assessed
the availability of data in your area for any of the indicators? If
so, for which indicators were data available/unavailable? |
As DIS4ME has evolved, the evaluation
questionnaire has become increasingly specific and targeted at individual
pages. The current version can be downloaded from Desertification
and DIS4ME: Take part in the DIS4ME evaluation.
5
top
g
Results from evaluation questionnaires
Although the rate of return of questionnaires
from people issued with the password to access DIS4ME on the DESERTLINKS
website has not been good, some very useful feedback has been received.
By November 2004, most people rated DIS4ME as at least fairly good. The
aspects that some people found to be not good or poor included:
- It was not clear that DIS4ME could
be used for a wide range of purposes.
- Some of the options for using DIS4ME
did not live up to what they promised
- Some of the options for using DIS4ME
are not relevant to stakeholders
- Operational objectives of some
of the indicators are not clear
- Some work is still needed to bridge
the gap between the needs of decision-makers and scientific results,
especially to make them available and useful to management
- DIS4ME is only good for a preliminary
evaluation of desertification
All these comments have been discussed
and improvements have been made where possible, between November 2004
and March 2005. Some of the specific comments are addressed below:
Comment |
Response |
The English language is not ideal
for use by stakeholders. |
Almost all the pages in DIS4ME
are being translated into Portuguese, Spanish, Italian and Greek. |
Navigation of DIS4ME
could be improved. |
Improvements were
made to the left and top menus and a sitemap was added. |
It is not easy to see which indicators
to use. |
The section "Choosing and
using indicators" has been written to help. |
There are too many indicators
and it is not clear how to rank the different indicators according
to relevance to the issue.
|
Tables of related indicators
have been added to the individual Issues pages, and all indicator
descriptions list related indicators. DIS Database provides a comprehensive
tool for sorting and selecting indicators according to many criteria. |
Reference to the DPSIR indicator
classification was missing. |
DPSIR classification was already
mentioned in all indicator descriptions, and is now described in "Choosing
and using indicators", and indicators may be sorted according
to their place in the classification in DIS DataBase. |
Better descriptions of indicators
and processes of desertification including typical examples are required. |
All indicator descriptions and
descriptive texts have been reviewed and improved where possible. |
Some indicators are not assigned
benchmarks. |
Benchmarks are suggested only
where appropriate, and they may need calibration for different situations.
|
There is no distinction between
parameters and indicators. |
Since indicators should be Specific,
Measurable, Achievable, Relevant, and Time-bound, many indicators
are parameters, but not all parameters are indicators. See "Choosing
and using indicators" for more details |
The basis for ranking desertification
risk is not well justified or documented. |
The methodology for the ESI tool
and for calculating desertification under various land uses has been
described in further detail. |
The use of photographs helps
understanding of the Issues. |
The number of photographs has
been greatly increased. |
Some models, e.g. the ESI tool
are not precisely accurate. |
The ESI tool is used to indicate
desertification risk using available data, and there are inevitably
situations not explained by the model. |
Typical examples of best management
practices for combating desertification would be useful.
|
The ManPras tool and ManData
include this information and are now part of DIS4ME. |
Use of DIS4ME requires considerable
knowledge and data. |
Indicators with unreasonable
data requirements have been excluded, and where possible, a choice
of similar indicators is provided to suit different users with different
data availability situations. |
ESA methodology and desertification
risk are not defined. |
Documentation has been improved,
including a downloadable version of the MEDALUS ESA Manual. |
There are too many indicators,
a basic set of indicators is needed. |
A long list of indicators is
useful as it may provide choice according to data availability. The
basic set of indicators is that used in the Environmental Sensitivity
Index. |
Analysis of other processes of
desertification (besides erosion and salinisation), such as toxicity
from heavy metal contamination and soil acidification is not included. |
These processes will be addressed
if time permits. |
The part of DIS4ME describing
desertification risk needs further expansion for other land uses and
calibration to other environmental and socio-economic conditions. |
The methodology used to quantify
desertification risk under different land uses is now fully described.
Calibration to other environmental and socio-economic conditions is
beyond the scope of this particular project. |
5
top
g Evaluation
of the choice of indicators used in relation to some desertification issues
Students from the GHEA Masters course
at the Università della Basilicata in Italy were invited to discuss
the indicators offered by DIS4ME in relation to desertification issues.
Some of their recommendations are summarised below:
- Land abandonment. Although
the process of land degradation is related most to physical indicators
such as slope gradient and slope aspect there are some additional considerations.
Land is most likely to be abandoned if the resistance of the soil makes
it difficult to till, if there are few services to maintain a quality
of life, and if there is little provision for developing and disseminating
new technologies that could make life easier. Some of these ideas could
be developed into new indicators.
- Intensive irrigation. The
most important indicators were considered to be land use, irrigation
potential realised, and whether income obtained justified using irrigation.
It is also necessary to include ideas about whether an area is attractive
for investment. This might be dependent on existing infrastructures,
to establish whether there is sufficient potential income to be derived
from investing in irrigation.
- Deforestation. Although
current vegetation cover is a useful indicator, it would also be helpful
to evaluate the history of the vegetation cover, as abrupt changes could
have contributed to the degree of land degradation or desertification.
In the same way, climate records may provide information relevant to
drought resistance.
- Littoralisation. The demands
made by tourism, affecting infrastructure and services, are very important.
There may be a threshold relating to existing tourism intensity, balancing
positive effects of improving the local economy with negative effects
such as increased pollution, depletion of water resources and increased
salinisation potential.
- Agricultural practices.
The chosen indicators focus on tillage operations and should consider
fertiliser application more.
- Economic activity. The influence
of changes in available or potential technology is important. Economic
activity is a primary factor of desertification but it is difficult
to define applicable indicators. It is important to understand what
is happening, why it is happening and what the consequences will be.
- Land degradation. The most
important indicators related to this issue are: soil erosion (state
indicator); fire frequency ( driving force); salinisation potential
(state); vegetation cover (state); sustainable farming and biodiversity
conservation (impact).
- Water resources. It may
be necessary to use different indicators for the different sectors:
agricultural, industrial and domestic (water consumption by sector),
and distinguish between water quality, water availability and water
management (e.g. irrigated area).
- Social structure. It would
be very useful to know about mobility patterns of the population: how
many people travel to work or for other purposes, the timescales of
their mobility, and whether mobility is linked to social factors such
as income, occupation or education level.
5
top
g
Validation
Validation is the process, by which
the DIS4ME indicator system has been tested, to see if any changes have
needed to be made when it is used within and outside the Target Areas
of the DESERTLINKS Project. There have been three stages:
- Validation of DIS4ME by local land
managers in local stakeholder groups in the DESERTLINKS Target Areas
of Portugal, Spain, Italy and Greece.
- Trial use of DIS4ME by the Focal
Point representatives of Portugal, Spain, Italy and Greece.
- Consolidation of evaluation and
validation results to provide guidelines for the CST and UNCCD on the
use of desertification indicators.
In Portugal, Spain, Italy and Greece
many ideas originating from DIS4ME or from the preceding MEDALUS Projects
have been integrated into the National Action Programmes, and these are
described in: Using
and combining indicators: Indicators and National Action Programmes.
Also see Using
and combining indicators: Indicators that are relevant at national and
Mediterranean-wide scales.
One of the most comprehensive examples
of validation has been in association with the OLIVERO Project on
the island of Crete. This is fully described in Using
DIS4ME: Crete (OLIVERO).
The south western part of the Chania
prefecture in western Crete is characterised by a variety of landscapes,
lithological units and climatic conditions, and it is primarily covered
by olive groves. Vines, citrus, annual crops, nut trees, and natural vegetation
(shrubs, pine forest, oak forest, etc.) are also found. Based on the stage
of land degradation and the sensitivity to desertification four categories
of environmentally sensitive areas (ESAs) were identified. The most widely
extended ESAs areas are fragile (78% of the total area) followed by critical
(14%), potential (4%) and non-threatened areas (4%).
Olive groves or individual olive trees
cover much of the study area. Three major types of cultivation practices
are applied: (a) tillage once or twice during spring, (b) no tillage and
no pesticides, and (c) no tillage and application of pesticides. Based
on the DESERTLINKS Environmental Sensitivity Index methodology for defining
desertification risk, the following possible scenarios were applied in
order to assesses the sensitivity of the area to desertification:
- replacing olive groves with cereals
- keeping olive groves and ploughing
the soil
- replacing olive groves with natural
vegetation
- retaining olive groves with no
tillage
Analysis of these scenarios of land
management suggested that the best protection for the land from desertification
is achieved if land use remains unchanged but ploughing of the soils should
be avoided.
In Italy the DesertNet Project
(at a national scale) has explored the use of ESA methodology in regions
such as Sardinia, Sicily, and Apulia, as well as Basilicata, where the
Agri basin is the DESERTLINKS Target Area (see Using
DIS4ME: Italy (DesertNet))
At an international level the DISMED
Project has used and adapted ESA methodology in Algeria, Egypt, Morocco,
Libya, Tunisia, France and Turkey, as well as in Portugal, Spain, Italy
and Greece. They are using a Desertification Sensitivity Index based on
just three quality indexes, of vegetation, soils and climate, because
it was difficult to agree on the definition of a land management quality
index (see Using DIS4ME:
DISMED).
5
top
g
Conclusions
DIS4ME can answer many questions about
desertification indicators. The Focal Points of Portugal, Spain, Italy
and Greece have already applied and mapped many of the headline indicators
in their National Action Programmes using MEDALUS/DESERTLINKS methods,
such as the Environmentally Sensitive Area methodology.
Suggestions for new indicators from
evaluators have been added to DIS4ME if it has been possible to write
an indicator description. However, there are still some important factors,
e.g. economic relationships, or social mobility, for which a definition
or method of assessment is not agreed, and an indicator description cannot
yet be written. Also, many good ideas for indicators cannot be supported
by currently available data.
The indicators have not yet been used
over a long enough period by a sufficient number of users to suggest which
are being found most useful in a wide range of situations. If DIS4ME can
be developed as a training tool in a subsequent project it will be possible
to catalogue the locations and precise use of indicators. If this information
is gathered it will be easier to demonstrate the difference between actual
contemporary desertification and desertification risk. The final DESERTLINKS
workshops with stakeholders in the Target Areas are being held in March
2005. Although there are still possible improvements for DIS4ME this version
will be presented and explained to the stakeholders as the final version,
in their own languages.
5
top
|