Give Me a Tool and I shall Move the World: Digitalization Options for Social Impact Assessment Methods

Lia Sidlina, Natalia Gladkikh

DOI 10.55140/2782-5817-2022-2-1-10-15

Performance measurement is now a necessity rather than a luxury. Without constant monitoring, tracking task completion indicators, assessing the effort input versus the achieved result, evaluating the impact of such results on a project's target audience, and self-reflection there will be no success, funding or attracting new clients and/or beneficiaries in any sphere. Thorough assessment is especially important in social sphere, because in this case performance measurement reflects real changes in the lives of people and entire communities.

Lia Sidlina

Leading Expert, Centre for Innovative Social Ecosystems, Institute of Social and Economic Design at the Higher School of Economics, ESG consultant

Natalia Gladkikh

PhD in Psychology, Leading Expert, Centre for Technological Innovations, Institute of Social and Economic Design at the Higher School of Economics

Despite all the importance of assessing the impact of social projects, there is no visible progress in the related theory and practice. The number of social projects where performance measurement is an integral part of the design process is very scarce. There is a lack of convenient metrics for various project types and spheres of activity, no systematic description of practices and approaches that would take into account state-of-the-art trends and technologies. While large foundations and non-profitable organizations do sometimes find resources for expensive procedures such as external evaluation by a qualified consultant in the sphere of assessment of social projects and programs, smaller entities can only report about the number of events held and services rendered as part of their grant-based projects. They do not have the time, the knowledge, and the resources for anything beyond that.

One possible solution to this challenge may be the use of simple, clear, and above all accessible tools for assessing social projects regardless of their scale, special features, and areas of concern. This mainly refers to interactive IT solutions that can be accessed from anywhere around the globe.

A number of important initiatives have already been implemented in this area, and it seems that the number will increase each year. Let's take a closer look at some of the initiatives created over the past year.

* * *

THE UNIQUE EVALUATIVE PROFILE (UEP) is a free online tool for various tasks and stages of assessment of socio-economic impact of non-profitable projects launched by the GLADWAY foundation in partnership with the Institute of Social and Economic Design at the Higher School of Economics. It is a methodology for self-assessment of project characteristics in the format of a questionnaire. After answering the questions, the program generates an individual profile which is then used to create a unique algorithm for further evaluation of projects for this specific organization. Depending on the individual and specific characteristics of the project, the evaluation model will include the elements and procedures that are most helpful in improving the efficiency of the project. In other words, the UEP is a unique map of the tasks and directions for monitoring and evaluation for a particular project. It includes the so-called "hygiene factors” – the minimum required activities for each project, and factors specific to the project under evaluation, taking into account the specifics of its target audience, the requests for data from a "third party” (for example, an organization that acts as a grantmaker), the need to assess economic efficiency, etc. The profile includes an assessment of the project according to 13 criteria, including the ability to control results at longterm impact level (outcomes), the limitations in providing feedback to the target audience, the need to assess the economic performance of the project, the availability of a project monitoring system, and others. At least three people from each organization should participate in the development of the UEP: a project manager, a member of the project team working with beneficiaries, and a specialist responsible for external communications. It may also be helpful to include the head of the organization in the team tasked with the development of the UEP. An integral UEP of a project is obtained when team members individually fill out questionnaires, and then the results are compared, thus choosing the optimal result for each criterion. Which, by the way, is not always the result of simple averaging.

https://clck.ru/eBY6B

* * *

YADRO ("CORE") IT SERVICE BY THE NUZHNA POMOSH ("NEED HELP") CHARITABLE FUND is one of the largest projects in Russia. It is available only in Russian. The service has several functions. First, it serves as a verification procedure in terms of informational transparency and reputation. Once registered, an organization becomes part of a catalogue of reputable foundations and can launch fundraising using a special service, and apply for support for its project. Second, the service offers an in-build annual report constructor. Third, the service has a multi-functional information management panel where one can find essential information about the project's segments in brief form. It also enables setting notifications that will remind you of important deadlines. Fourth, the service provides access to detailed statistical data across the organization regarding donations, donors, and funding sources.

https://clck.ru/eBY7q

* * *

AN ONLINE SERVICE FOR EVALUATING THE IMPACT OF NON-PROFIT ORGANIZATIONS IN THE ACHIEVEMENT OF INDICATORS USED IN THE PERFORMANCE ASSESSMENT OF TOP OFFICIALS OF THE TERRITORIAL ENTITIES OF THE RUSSIAN FEDERATION AND THE WORK OF EXECUTIVE BODIES OF THE TERRITORIAL ENTITIES OF THE RUSSIAN FEDERATION is a service that allows you to speak the language of government institutions. How to convince officials to support the project, allocate funding, provide information support? How to assess that a certain project will help achieve the goals in a particular region? What metrics and indicators to use? What methodology to apply? Answers to these questions can be provided by an online service for evaluating the impact of non-profit organizations on the achievement of indicators for evaluating the effectiveness of top officials (heads of the crucial governing institutions) of the territorial entities of the Russian Federation and the work of executive bodies of the territorial entities of the Russian Federation. Available only in Russian. According to the Decree of the President of the Russian Federation dated February 4, 2021, there are only 20 such indicators. Several of them can be fulfilled through active efforts of non-profit organizations and, accordingly, through the support of NGOs by regional administrations, local governments, and regional businesses. The service is primarily informational in nature – it provides information about the availability of assessment criteria, a list of indicators, their components and calculation methods, and also helps to easily and quickly compile a list of points of influence that each NGO has on assessing the performance of governors and establish communication in a language understandable to NGOs and the administration.

https://clck.ru/eBY88

* * *

THE KPI LENS is an international service. It only has an English version. It allows organizations to enter and analyze data to prepare monitoring and evaluation of their projects in online dashboards mode. The KPI Lens data collection functionality allows you to enter the on-hand data or conduct surveys using built-in tools. The resulting data are automatically transferred to an online workspace where it can be managed and analyzed in real time without the need to switch between multiple software tools. In KPI Lens, you can track the impact of projects using the Theory of Change or a logical model. The service allows you to add your own indicators, immediate results (outputs), long-term results (outcomes), and provides several monitoring platforms, an indicator tracker table, an option of comparing targeted values with actual ones. The service also enables preparing reports that may include various data visualization formats. The service operates on a subscription basis, the monthly fee depends on functionality and ranges from $0 (free version with a decent range of features) to $399 (full access).

https://clck.ru/eBY8S

* * *

THE VOLUNTEER ECONOMIC CONTRIBUTION CALCULATOR is another service developed by the GLADWAY Foundation in partnership with the Institute of Social and Economic Design at the Higher School of Economics. Available only in Russian. The Calculator uses a replacement cost calculation to identify the value of volunteer labor for an organization. The calculation uses data from an API service developed by Headhunter.ru, the leading Russian-language job search portal, including the average salary in the specified region for the specified type of specialist across 100 vacancies posted in a sector during the current month. To perform the calculations, it is necessary to fill in the fields «Specialization», "Location", "Position title"; choose the industry that best fits the competencies of a volunteer involved in the project (the list of industries is provided by Headhunter's API). After you click the «Calculate» button, you get a figure that shows the average salary for a certain position in a specific region. The result is then exported to a PDF file, which can be used to assess the estimated organization's contribution when applying for project funding, as well as to conduct an economic assessment of the effectiveness of volunteer contributions to a social project.

https://clck.ru/Y83KY

* * *

SELECTION OF SUSTAINABLE DEVELOPMENT GOALS (SDGS) is another tool launched by the GLADWAY Foundation in partnership with the Institute of Social and Economic Design at the Higher School of Economics. It is aimed at tackling the important task of translating project outcomes into a language understandable to both representatives of different sectors and representatives of different countries – the language of the Sustainable Development Goals (SDGs). On the one hand, these goals are well-known and understandable, so translating the results of the project into this language seems a relatively simple task. On the other hand, the SDG list currently contains 17 goals, and each goal includes 7–10 tasks on average, and translating the areas and results of activities into SDG language can turn into a rather laborious process. In order to simplify it, an online service was created that allows you to visually correlate your activities with the SDGs. The service is available only in Russian. The end product is 1–2 sheets in PDF format with icons and descriptions of the goals and objectives of the organization in the SDG language. This visual presentation of goals and objectives makes the process of negotiations regarding cooperation in the social sphere more open and comprehensible, business partners can easily scan the provided information and almost instantly identify the areas of common interest and how a social project can fit into existing policies, procedures and an organization's strategy.

https://clck.ru/eBY9Q

* * *

These examples of digital tools cover only a few instruments created over the past year. The wider trend towards simplifying the tasks associated with the evaluation of social projects and programs using open and accessible online tools has been in place for several years. For example, there are about a dozen different software solutions that focus specifically on the Theory of Change including TOCO, Dylomo, Theory Maker, Changeroo and others.

Most of the tools are available in English, making them more user-friendly for the global audiences. At the same time, these tools require localization (at least at the linguistic level) in order to become more accessible and convenient for use in different countries.

In addition, it is important to systematize and understand the capacities, limitations and conditions for using each of the tools in order to eliminate the redundancy and chaos and in order to simplify the choice of the best tool for any specific task. Therefore, it is logical that the development of digital assessment tools should build upon the challenges faced during practical activities that require assessing the impact of social projects. In other words, the methodology and technology of convenient online services should correspond to the real needs of practitioners.