DIS4ME DIS4ME βασική σελίδα | DESERTLINKS βασική σελίδα | © DESERTLINKS 2004
English-EN | Español-ES | Italiano-I | Ελληνικά-GR | Portuguese-PT
Σύστημα Δεικτών Ερημοποίησης για την Μεσογειακή Ευρώπη
leftmenu_gr
Evaluation and validation of DIS4ME

Authors: Jorge García Gómez <jorgegg@um.es> and Nichola Geeson <desertlinks@medalus.demon.co.uk>


g Introduction
g Basic questionnaire design
g Results from evaluation questionnaires
g Evaluation of the choice of indicators used in relation to some desertification issues
g Validation
g Conclusions

g Introduction

Throughout the DESERTLINKS project, workshops with stakeholders in the Target Areas have been used to discuss requirements of an indicator system and the merits of individual indicators. The process of interaction between scientists and stakeholders continues, as more people have the opportunity to find out about DIS4ME and contribute to improving it.

Stakeholders include:

  • Farmers and farming organisations.
  • Local and regional administrative authorities, representative technicians in city councils, regional environmental, land management, agriculture. Technicians from cooperatives and trade unions.
  • Civil society, NGOs, environmental associations, private foundations, local associations; teachers and students at universities, colleges and schools.
  • Scientists from disciplines related to desertification.

The evaluation process has included:

  • Workshops in the local language in the Target Areas, workshops at specific project meetings and other international meetings, and a workshop with the GHEA Masters students at the Università della Basilicata.
  • Questionnaires, circulated to stakeholders in the Target Areas; representatives from national Focal Points; representatives from the UNCCD and Committee for Science and Technology; other scientists in parallel projects; and everyone else who has asked for the password to view DIS4ME.

5 top

g Basic questionnaire design

The questions included:

  • Is DIS4ME simple to use?
  • Is the structure of DIS4ME logical?
  • Is DIS4ME comprehensive?
  • Is the level of detail sufficient?

Guidelines were proposed to develop a suitable questionnaire:

  • Questions should be kept simple, addressing one idea at a time.
  • Closed questions, with a specific range of response, allow responses in the different Target Areas to be compared. Some open questions are valuable too of course.
  • Allowing people the easy option of giving non-committal answers is best avoided, i.e. it is better to have four classes for responses than three.
  • Details of the persons answering the questionnaire must be recorded, e.g. male/female, their age range, the group or organisation they belong to, scientist/non scientist.

Basic questionnaire

Closed Questions Good Fairly good Not good Poor
Is it clear what the purpose of DIS4ME is and how it can be used?        
We have tried to present the information in a popular scientific style. How clear did you find the language used?        
How easy was it for you to navigate around DIS4ME?        
In general, how well does DIS4ME address your particular requirements as a scientist, policy maker, farmer, or student, etc.?        
Did you find that DIS4ME helped you to better understand the processes of desertification?        
If you have ticked the "not good" or "poor" box for any question, please explain your reasons and suggest how improvements can be made.        
Open questions
1. Do you have any suggestions for improving the structure or content of DIS4ME?
2. Can you suggest any areas where the information (e.g. descriptions of indicators or processes) should be expanded or clarified?
3. Are there any specific desertification problems that we are not addressing?
4. Have you used any of the tools (i.e. ESI tool, tool to calculate desertification risk from salinisation or erosion under various land uses) to assess desertification in your own area? If so, how accurate or appropriate were the results the tools gave?
5. Have you assessed the availability of data in your area for any of the indicators? If so, for which indicators were data available/unavailable?

As DIS4ME has evolved, the evaluation questionnaire has become increasingly specific and targeted at individual pages. The current version can be downloaded from Desertification and DIS4ME: Take part in the DIS4ME evaluation.

5 top

g Results from evaluation questionnaires

Although the rate of return of questionnaires from people issued with the password to access DIS4ME on the DESERTLINKS website has not been good, some very useful feedback has been received. By November 2004, most people rated DIS4ME as at least fairly good. The aspects that some people found to be not good or poor included:

  • It was not clear that DIS4ME could be used for a wide range of purposes.
  • Some of the options for using DIS4ME did not live up to what they promised
  • Some of the options for using DIS4ME are not relevant to stakeholders
  • Operational objectives of some of the indicators are not clear
  • Some work is still needed to bridge the gap between the needs of decision-makers and scientific results, especially to make them available and useful to management
  • DIS4ME is only good for a preliminary evaluation of desertification

All these comments have been discussed and improvements have been made where possible, between November 2004 and March 2005. Some of the specific comments are addressed below:

Comment Response
The English language is not ideal for use by stakeholders. Almost all the pages in DIS4ME are being translated into Portuguese, Spanish, Italian and Greek.
Navigation of DIS4ME could be improved. Improvements were made to the left and top menus and a sitemap was added.
It is not easy to see which indicators to use. The section "Choosing and using indicators" has been written to help.
There are too many indicators and it is not clear how to rank the different indicators according to relevance to the issue.
Tables of related indicators have been added to the individual Issues pages, and all indicator descriptions list related indicators. DIS Database provides a comprehensive tool for sorting and selecting indicators according to many criteria.
Reference to the DPSIR indicator classification was missing. DPSIR classification was already mentioned in all indicator descriptions, and is now described in "Choosing and using indicators", and indicators may be sorted according to their place in the classification in DIS DataBase.
Better descriptions of indicators and processes of desertification including typical examples are required. All indicator descriptions and descriptive texts have been reviewed and improved where possible.
Some indicators are not assigned benchmarks. Benchmarks are suggested only where appropriate, and they may need calibration for different situations.
There is no distinction between parameters and indicators. Since indicators should be Specific, Measurable, Achievable, Relevant, and Time-bound, many indicators are parameters, but not all parameters are indicators. See "Choosing and using indicators" for more details
The basis for ranking desertification risk is not well justified or documented. The methodology for the ESI tool and for calculating desertification under various land uses has been described in further detail.
The use of photographs helps understanding of the Issues. The number of photographs has been greatly increased.
Some models, e.g. the ESI tool are not precisely accurate. The ESI tool is used to indicate desertification risk using available data, and there are inevitably situations not explained by the model.
Typical examples of best management practices for combating desertification would be useful.
The ManPras tool and ManData include this information and are now part of DIS4ME.
Use of DIS4ME requires considerable knowledge and data. Indicators with unreasonable data requirements have been excluded, and where possible, a choice of similar indicators is provided to suit different users with different data availability situations.
ESA methodology and desertification risk are not defined. Documentation has been improved, including a downloadable version of the MEDALUS ESA Manual.
There are too many indicators, a basic set of indicators is needed. A long list of indicators is useful as it may provide choice according to data availability. The basic set of indicators is that used in the Environmental Sensitivity Index.
Analysis of other processes of desertification (besides erosion and salinisation), such as toxicity from heavy metal contamination and soil acidification is not included. These processes will be addressed if time permits.
The part of DIS4ME describing desertification risk needs further expansion for other land uses and calibration to other environmental and socio-economic conditions. The methodology used to quantify desertification risk under different land uses is now fully described. Calibration to other environmental and socio-economic conditions is beyond the scope of this particular project.

5 top

g Evaluation of the choice of indicators used in relation to some desertification issues

Students from the GHEA Masters course at the Università della Basilicata in Italy were invited to discuss the indicators offered by DIS4ME in relation to desertification issues. Some of their recommendations are summarised below:

  • Land abandonment. Although the process of land degradation is related most to physical indicators such as slope gradient and slope aspect there are some additional considerations. Land is most likely to be abandoned if the resistance of the soil makes it difficult to till, if there are few services to maintain a quality of life, and if there is little provision for developing and disseminating new technologies that could make life easier. Some of these ideas could be developed into new indicators.
  • Intensive irrigation. The most important indicators were considered to be land use, irrigation potential realised, and whether income obtained justified using irrigation. It is also necessary to include ideas about whether an area is attractive for investment. This might be dependent on existing infrastructures, to establish whether there is sufficient potential income to be derived from investing in irrigation.
  • Deforestation. Although current vegetation cover is a useful indicator, it would also be helpful to evaluate the history of the vegetation cover, as abrupt changes could have contributed to the degree of land degradation or desertification. In the same way, climate records may provide information relevant to drought resistance.
  • Littoralisation. The demands made by tourism, affecting infrastructure and services, are very important. There may be a threshold relating to existing tourism intensity, balancing positive effects of improving the local economy with negative effects such as increased pollution, depletion of water resources and increased salinisation potential.
  • Agricultural practices. The chosen indicators focus on tillage operations and should consider fertiliser application more.
  • Economic activity. The influence of changes in available or potential technology is important. Economic activity is a primary factor of desertification but it is difficult to define applicable indicators. It is important to understand what is happening, why it is happening and what the consequences will be.
  • Land degradation. The most important indicators related to this issue are: soil erosion (state indicator); fire frequency ( driving force); salinisation potential (state); vegetation cover (state); sustainable farming and biodiversity conservation (impact).
  • Water resources. It may be necessary to use different indicators for the different sectors: agricultural, industrial and domestic (water consumption by sector), and distinguish between water quality, water availability and water management (e.g. irrigated area).
  • Social structure. It would be very useful to know about mobility patterns of the population: how many people travel to work or for other purposes, the timescales of their mobility, and whether mobility is linked to social factors such as income, occupation or education level.

5 top

g Validation

Validation is the process, by which the DIS4ME indicator system has been tested, to see if any changes have needed to be made when it is used within and outside the Target Areas of the DESERTLINKS Project. There have been three stages:

  1. Validation of DIS4ME by local land managers in local stakeholder groups in the DESERTLINKS Target Areas of Portugal, Spain, Italy and Greece.
  2. Trial use of DIS4ME by the Focal Point representatives of Portugal, Spain, Italy and Greece.
  3. Consolidation of evaluation and validation results to provide guidelines for the CST and UNCCD on the use of desertification indicators.

In Portugal, Spain, Italy and Greece many ideas originating from DIS4ME or from the preceding MEDALUS Projects have been integrated into the National Action Programmes, and these are described in: Using and combining indicators: Indicators and National Action Programmes. Also see Using and combining indicators: Indicators that are relevant at national and Mediterranean-wide scales.

One of the most comprehensive examples of validation has been in association with the OLIVERO Project on the island of Crete. This is fully described in Using DIS4ME: Crete (OLIVERO).

The south western part of the Chania prefecture in western Crete is characterised by a variety of landscapes, lithological units and climatic conditions, and it is primarily covered by olive groves. Vines, citrus, annual crops, nut trees, and natural vegetation (shrubs, pine forest, oak forest, etc.) are also found. Based on the stage of land degradation and the sensitivity to desertification four categories of environmentally sensitive areas (ESAs) were identified. The most widely extended ESAs areas are fragile (78% of the total area) followed by critical (14%), potential (4%) and non-threatened areas (4%).

Olive groves or individual olive trees cover much of the study area. Three major types of cultivation practices are applied: (a) tillage once or twice during spring, (b) no tillage and no pesticides, and (c) no tillage and application of pesticides. Based on the DESERTLINKS Environmental Sensitivity Index methodology for defining desertification risk, the following possible scenarios were applied in order to assesses the sensitivity of the area to desertification:

  • replacing olive groves with cereals
  • keeping olive groves and ploughing the soil
  • replacing olive groves with natural vegetation
  • retaining olive groves with no tillage

Analysis of these scenarios of land management suggested that the best protection for the land from desertification is achieved if land use remains unchanged but ploughing of the soils should be avoided.

In Italy the DesertNet Project (at a national scale) has explored the use of ESA methodology in regions such as Sardinia, Sicily, and Apulia, as well as Basilicata, where the Agri basin is the DESERTLINKS Target Area (see Using DIS4ME: Italy (DesertNet))

At an international level the DISMED Project has used and adapted ESA methodology in Algeria, Egypt, Morocco, Libya, Tunisia, France and Turkey, as well as in Portugal, Spain, Italy and Greece. They are using a Desertification Sensitivity Index based on just three quality indexes, of vegetation, soils and climate, because it was difficult to agree on the definition of a land management quality index (see Using DIS4ME: DISMED).

5 top

g Conclusions

DIS4ME can answer many questions about desertification indicators. The Focal Points of Portugal, Spain, Italy and Greece have already applied and mapped many of the headline indicators in their National Action Programmes using MEDALUS/DESERTLINKS methods, such as the Environmentally Sensitive Area methodology.

Suggestions for new indicators from evaluators have been added to DIS4ME if it has been possible to write an indicator description. However, there are still some important factors, e.g. economic relationships, or social mobility, for which a definition or method of assessment is not agreed, and an indicator description cannot yet be written. Also, many good ideas for indicators cannot be supported by currently available data.

The indicators have not yet been used over a long enough period by a sufficient number of users to suggest which are being found most useful in a wide range of situations. If DIS4ME can be developed as a training tool in a subsequent project it will be possible to catalogue the locations and precise use of indicators. If this information is gathered it will be easier to demonstrate the difference between actual contemporary desertification and desertification risk. The final DESERTLINKS workshops with stakeholders in the Target Areas are being held in March 2005. Although there are still possible improvements for DIS4ME this version will be presented and explained to the stakeholders as the final version, in their own languages.

5 top