Illustration 24. Implementation stage Implementation stage Implementation of the NISP Evaluation Illustration 25. Implementation phase components 2.5.1. Inputs for the implementation phase The implementation phase gathers all the aspects related to the implementation of the NISP as planned in the elaboration stage, through a set of instruments and actions. In this phase, the implementation does not depend so much on the civil servant or governmental bodies charged with the construction of the NISP, nor on the experts team, but on the government and other social actors.
Illustration 26. Implementation phase inputs Tip 4. The implementation actions differ in each policy or strategy Choosing or creating a body (an agency or organization) to carry out the policies and strategies proposed by the NISP. This organization is usually coordinated by the government, but it includes multisectoral stakeholders: enterprises, universities, NGOs, etc.
Establishing goals and beneficiaries: Goals are the reason for the policy to exist; the beneficiaries are the individuals, communities and organization that will benefit from the NISPs implementation.
Plannig actions and activities to achieve the goals, concrete programs and projects, as priority areas: e-government, e-health, cybersecurity, etc.
Legislation changes to make the NISP proposal feasible.
Illustration 27. Implementation phase processes 2.5.2. Fast-Track Initiatives Some procedures (Findlay, 2007) advise starting the implementation of the NISP with iniciatives or projects that can be developed in the short run and that will show the stakeholders, the financial sources, and the citizens, the efficacy of the NISP. However, every national environment requires different implementation steps. In the case of deciding on fast track iniciatives or projects, it is sensible to start with concrete, uncomplicated projects, which can be easily carried out.
• Implement early • Demonstrated momentum/results • Non complex projects • Have a visible impact with citizens • Support with promotion and awareness Fast-Track Projects often include:
e-Government Portal Community Access Centres Computers for Schools ICT related legislative amendments 2.5.3. Full Implementation After the actors responsible for the NISP implementation have proved their efficacity and involvement, it is time to proceed to the full implementation of the policies and strategies.
The requirements of full implementation are the following:
• Strengthened Governance model • Detailed Project Planning • Project Management/Integration • Project staffing • Streamlined procurement/contracting • Financial management 2.5.4. Implementation Phase Outcomes New or updated legislations on Information Society The solidification of parts of the NISP, through concrete initiatives and projects, or of the full policy, over a given period of time The nomination of control agencies for monitoring and assessment Communication of the NISP to the population, in order to obtain citizens’ involvement Illustration 28. Outputs of the implementation phase 2.6. Follow-up phase The assessment or control is the method through which governments and society may judge the real worthiness or credit of governmental (or multistakeholder) actions. Many countries are concerned about measuring the effective impacts of a NISP. The evaluation process implies a systematic examination of the NISP’s objectives and its results, that is to say, an analysis of the distance between the effective results and the expected results.
Illustration 29. NISP follow-up, monitoring, control and adaptation.
Follow up phase NISP follow-up, monitoring, control and adaptation Evaluation Illustration 30. Follow-up components Milestone This distance may result from the intervention of random elements and/or the governments or chosen organization’s handling of determined obstacles. In general, the monitoring and evaluation processes measure the distance between the orchestrated policy and the initial plan, and the economic effects generated by the executed policy.
Illustration 31. Inputs of the follow-up phase 2.6.1. Monitoring According to Phil Bartle (2007), monitoring is the regular observation and recording of activities taking place in a project or programme. It is a process of routinely gathering information on all aspects of the project. In this case, to monitor is to check on how NISP’s activities are progressing. Monitoring also involves giving feedback about the progress of the NISP to the stakeholders, implementers and beneficiaries of the project. Reporting enables the gathered information to be used in making decisions for improving the NISP’s performance.
It is important to consider that generally there is no data available to consider the long term effects of the NISP. Therefore, rather than the accurate evaluation of the NISP’s implementation results, a complete analysis or monitoring during several years is necessary.
Monitoring provides information that will be useful in:
Analysing the situation in the country or community;
Determining whether the inputs in the NISP are well utilized;
Identifying problems facing the NISP’s implementation and finding solutions;
Ensuring all activities are carried out properly by the right people and in time;
Using lessons from the experience to update the NISP, its strategies and tactics;
Determining whether the way the NISP implementation was planned is the most appropriate way of achieving the goals.
2.6.2. Evaluation Evaluation is a key phase, measuring and analyzing the impact of actions taken, to judge whether goals have been attained. In order to achieve effective evaluations, the departing situations or diagnostics have to be taken into account, in order to verify the changes that have been triggered by the NISP and its successive phases. Evaluation is not limited to the NISP’s application: it should take place in all the phases of the NISP. As a result of this process, it may prove necessary to establish corrective measures demanding the formulation of new policy guidelines and implementation of new strategic actions, taking situational shifts into account. The policy can thus be updated. It should also be updated after some years.
Evaluation is a process of placing value on what an NISP has achieved particularly in relation to activities planned and overall objectives. It involves value judgment and hence it is different from monitoring (which is observation and reporting of observations). It is important to identify the constraints or bottlenecks that hinder the NISP implementation in achieving its goals. Solutions to the constraints can then be identified and implemented.
Evaluation should provide a clear picture of the extent to which the intended objectives of the NISP’s actions and policies have been realized. Evaluation can and should be done during and after implementation.
Before implementing the NISP, evaluation is needed in order to:
Assess the possible consequences of the planned NISP to the country over a period of time;
Assist in making decisions on how the project will be implemented.
During the NISP’s implementation:
Evaluation should be a continuous process and should take place in all the implementation activities. This enables the organization in charge to progressively review the strategies according to the changing circumstances in order to attain the desired activity and objectives.
After projects’ implementation:
Evaluation should be used to retrace the NISP’s planning and implementation process and results after its implementation.
Due to the time inbetween the layout or planning and the effective instrumentation, the evaluation of technological and organizational policies becomes an additional tool to understand the faults in the process, from the elaboration of the NISP to its application.
Evaluating an NISP and studying its limitations can help formulate a new suitable policy which contemplates the real necessities of the country. In many cases, it is verified that the implementation difficulties are due to lack of coordination between the agents who act in the innovation system (companies, research centers, universities, NGOs) and financing institutions.
The second aspect of the evaluation is centered on the axis that links the policy with its economic effects. In this case, the evaluation aims to understand the way in which the implemented NISP directly and indirectly affected the performance of the participating agents, as well as other spheres of the economy. The first methods of evaluation were created decades ago in developed countries and were based, mainly, on quantitative analysis, using two tools: the “administrative information” of the companies to catch the policy’s impact on the sales, and the “cost-benefit” analysis to understand the relation between the financial gains and losses of the companies favored by the program.
However, it is considered that those two evaluation tools are limited because they summarize the impacts of the policy with a unique financial variable and do not grasp all dimensions of the process.
The difficulties in measuring the effects of innovation policies, such as NISPs, are due to the fact that the innovation factor is the result of a dynamic process that supposes both short and long-term articulations among diverse stakeholders. In addition this process deals with the establishment of an innovative institutional environment, as well as new regulatory policies; both effects that are not easily measurable by means of traditional cost-benefit analysis.
In addition to quantitative methods (surveys, questionnaires), it may be useful to employ qualitative evaluation methods, including interviews to key informers, questionnaires, surveys, and case studies.
Illustration 32. Processes of the follow-up phase Example 25. The Macedonian Strategy The Macedonian Strategy On September 21, 2005, the Parliament of the Republic of Macedonia adopted the National Information Society Development Strategy 1 (hereinafter “the Strategy”). The Strategy represents the result of numerous efforts and processes in which various entities took place from the domestic political scene, the civil sector, international organizations, as well as from the political processes. The National Information Society Policy of the Republic of Macedonia States the “Development of a process of permanent monitoring and evaluation of the achieved results in the development of the Information society, with an emphasis of mandatory usage of the feedback (indicators) to create the future policies, strategies and plans in the Republic of Macedonia”.
Source: Republic of Macedonia, 2.6.3. The use of indicators An indicator provides evidence that certain condition exists or certain results have or have not been achieved. Indicators enable decision-makers to assess progress towards the achievement of intended outputs, outcomes, goals, and objectives. As such, indicators are an integral part of a results-based accountability system19.
Indicators can measure inputs, processes, outputs, and outcomes. Input indicators measure resources, both human and financial, devoted to a particular program or intervention (i.e., number of case workers). Input indicators can also include measures of characteristics of target populations (i.e., number of clients eligible for a program). Process indicators measure ways in which program services and goods are provided (i.e., error rates). Output indicators measure the quantity of goods and services produced and the efficiency of production (i.e., number of people served, speed of response to reports of abuse). These indicators can be identified for programs, sub-programs, agencies, and multi-unit/agency initiatives. Outcome indicators measure the broader results achieved through the provision of goods and services. These indicators can exist at various levels:
population, agency, and program.
As for the criteria for selecting indicators, Horsch (2007) admits that choosing the most appropriate indicators can be difficult. Development of a successful accountability system requires that several people be involved in identifying indicators, including those who will collect the data, those who will use the data, and those who have the technical expertise to understand the strengths and limitations of specific measures.
Some questions that may guide the selection of indicators are:
Does this indicator enable one to know about the expected result or condition Is the indicator defined in the same way over time Are data for the indicator collected in the same way over time Will data be available for an indicator Are data currently being collected If not, can cost effective instruments for data collection be developed Is this indicator important to most people Will this indicator provide sufficient information about a condition or result to convince both supporters and skeptics Is the indicator quantitative Horsch, Karen: Indicators: Definition and Use in a Results-Based Accountability System, Harvard Family Research Project, 1997, http://www.hfrp.org/publications-resources/browse-our-publications/indicatorsdefinition-and-use-in-a-results-based-accountability-system As stated by Horsch, it is important to note that indicators serve as a red flag; good indicators merely provide a sense of whether expected results are being achieved. They do not answer questions about why results are or are not achieved, unintended results, the linkages existing between interventions and outcomes, or actions that should be taken to improve results. As such, data on indicators need to be interpreted with caution. They are best used to point to results that need further exploration, rather than as definitive assessments of program success or failure.
Some indicators systems developed by international organizations, and national and regional governments, are the following: OECDs Guide to Measuring the Information Society (OECD, 2009); the ICT Development Index (IDI) of the International Communication Union - ITU (ITU, 2009b) and UNCTADs "The Global Information Society: a Statistical View” (UNCTAD, 2008).
Материалы этого сайта размещены для ознакомления, все права принадлежат их авторам.
Если Вы не согласны с тем, что Ваш материал размещён на этом сайте, пожалуйста, напишите нам, мы в течении 1-2 рабочих дней удалим его.