End-of-program evaluation – Ukraine program

Terms of Reference (ToR) End-of-program evaluation

“Towards an RSF Press Freedom Center in Lviv to fostering reliable media coverage of the crisis in Ukraine and the region”

1. Context of the program

The war in Ukraine started on 24 February 2022. The military situation evolved quickly with dire consequences for journalists who are being attacked while media infrastructures are being targeted. The threats are not only physical, but include cyberattacks, disinformation campaigns and censorship attempts that aim at preventing the public from accessing credible information. The Ukrainian diversified media landscape is suffering the pressure of the war and its capacity to provide independent and reliable information to the public is being limited by the increasing pressure and danger that journalists are working under. In addition, international journalists arrived in Ukraine in large numbers. Protective equipment is not available for everyone, although such equipment is mandatory for field work. The pressure on both international and Ukrainian journalists on the ground is high. There was an urgent need to provide for safety measures and an enabling environment for journalists to perform their duties. The role of independent media as a witness of the war is of paramount importance and journalists are to be offered assistance and protection.

In this context, Reporters sans frontières – Reporters Without Borders (RSF) has developed its action in Ukraine and, together with its long-standing local partner, the Institute for Mass Information (IMI), has started to provide support to journalists from the early days of the war.

Headed by RSF’s Director of operations and campaigns, a crisis unit has been monitoring the situation from Day 1. On 11 March 2022, the Secretary-general of RSF, Christophe Deloire, was in the city of Lviv, in western Ukraine, for the inauguration of RSF’s Press Freedom Center. The Center aims at providing international and Ukrainian journalists with the best possible assistance with covering the war. The Center is working under RSF’s coordination and in partnership with IMI staff for the provision of individual sets of protective equipment to journalists. However, the Center aims not only at organizing deliveries, handover and loans of equipment to journalists and local organizations ; but it also works as a capacity building resource center and a channel to identify needs for assistance. After the withdrawal of Russian troops from Kyiv, an office of the Center was opened in Kyiv on 17 May 2022, in order to reach more journalists, in particular Ukrainians.

2. Reporters sans frontières

Founded in 1985, Reporters sans frontières – Reporters Without Borders (RSF) ensures respect for freedom of information, as provided for in Article 19 of the Universal Declaration of Human Rights, which states that “Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers.”.

The organization defends and promotes freedom, pluralism and independence of journalism throughout the world. By fighting censorship and concretely defending journalists or media in difficulty, persecuted because of their activity, RSF supports one of the foundations of democracy.

RSF has an international secretariat in Paris, fourteen sections and offices around the world and more than 130 correspondents and representatives. It is a recognized public utility association in France and has consultative status with the UN, the UNESCO, the OIF, the African Commission on Human and Peoples’ Rights and the Council of Europe.

3. The program

3.1. Objective of the program

The objective of the program is based on the identified need to foster access to independent and reliable media coverage of the crisis in Ukraine and the region. The project aims to provide an enabling environment for journalists working in the coverage of the crisis in Ukraine and the region. At the beginning of the program, the following needs have been identified for the journalists and media:

• A safe place to refer to when in distress
• Protective equipment (bulletproof vests and helmets)
• Individual first aid trauma kits
• Technical equipment (solar power banks)
• Psychological support
• Capacity building (physical safety, first aid, psychological safety)
• Emergency financial support to journalists and media outlets

Founding the Center enabled RSF to address the above-mentioned needs. Indeed, the project aimed at reinforcing journalists’ safety, documenting and fighting against the impunity of crimes against journalists, while raising awareness among the general public on the importance of independent and reliable coverage of the war in Ukraine.

In the framework of this project, RSF works with two types of organizations:

• Local organizations to ensure the relevance and implementation of the project. Among them, the following can be distinguished:
  • IMI as RSF’s long-standing and main Ukrainian partner.
  • Other local implementing partners identified during the project such as the National Union of Journalists of Ukraine (NUJU)
• Other NGOs involved in the Global Forum for Media Development (GFMD)’s Ukraine coordination meetings such as FPU, IMS, Internews, The Fix. RSF is in contact with these organizations to ensure that support activities in the field are not duplicated and are consistent with each other.

RSF has developed essential and very effective synergies with its partner IMI and other local organizations, as part of the equipment was directly handed over by the Press Freedom Center to them to be distributed to Ukrainian journalists closer to the frontlines. In addition, the RSF Center ensures the daily distribution (loans) of equipment to Ukrainian and foreign journalists and other media professionals (fixers in particular) in Lviv and Kyiv, counting with 297 beneficiaries, as of 31 December 2022.

3.2. Target countries

The target countries are Ukraine and the region.

3.3. Objectives, planned activities, beneficiaries, budget, duration

Overall Objective: To contribute to fostering access to independent and reliable media coverage of the crisis in Ukraine and the region
Specific Objective: To provide an enabling environment for journalists working in the coverage of the crisis in Ukraine and the region

Output 1: A Freedom Press Center is established in Lviv, equipped, and resourced to provide safety and support to journalists

• Activity 1.1: Setting up the structure of the Lviv Press Freedom Center
  • Task 1.1.1 Setting up the Center and the procedures in particular for procurement, purchase and distribution of equipment
  • Task 1.1.2 Setting up outreach mechanisms to provide access to journalists to the Center’s services
  • Task 1.1.3 Possible relocation of the Press Freedom Center
• Activity 1.2: Providing journalists covering the crisis with the necessary protective equipment
  • Task 1.2.1 Ensure the purchase and delivery of protective and other equipment (IFAK kit, solar chargers, external batteries, journalistic materials)
  • Task 1.2.2 Ensure the distribution of protective and other equipment
• Activity 1.3: Provide direct financial support to journalists and media covering the crisis in Ukraine
  • Task 1.3.1 Individual support for journalists (assistance grants and administrative support) – Financial Support to Third Parties
  • Task 1.3.2 Media Support Fund – Financial Support to Third Parties
  • Task 1.3.3 Psychological support to journalists
  • Task 1.3.4 Capacity building of journalists in crisis situations

Output 2: Awareness is raised on the importance of media coverage and access to information by the general public and the international community

• Activity 2.1: Press freedom monitoring and research
  • Task 2.1.1 Daily monitoring of the situation
  • Task 2.1.2 Frequently updated map of attacks against journalists and media outlets
  • Task 2.1.3 Investigation and inquiry into the abuses committed
  • Task 2.1.4 Publication of regular press releases and reports on the evolution of the situation and the abuses committed against journalists
  • Task 2.1.5 Translations of publications
• Activity 2.2: Informing general public and raise public awareness on the crisis situation
  • Task 2.2.1 Define and implement specific communication campaigns
  • Task 2.2.2 Focus on the situation in Ukraine during events
• Activity 2.3: Advocating towards authorities, governments and international organisations
  • Task 2.3.1 Development of recommendations
  • Task 2.3.2 Bilateral advocacy with governments
  • Task 2.3.3 Multilateral advocacy with international organizations (including the UN Commission of Inquiry launched by the Human Rights Council)
  • Task 2.3.4 Other planned advocacy activities

Beneficiaries:
Target groups: Journalists and media outlets covering the war in Ukraine

• Accredited local and international journalists and their collaborators (fixers, drivers, translators, etc.)
• Media outlets (press, radio, TV, blogs)

Final beneficiaries: Media, news and information consumers, society at large

Budget:
The total budget for the project is 1,070,000.00€. 1,000,000.00€ are funded by the European Union and 70,000.00€ come from co-financing.

Duration:
The implementation of the project began on 10 March 2022 for a period of 12 months. The project will end on 10 March 2023.

4. Objectives of the evaluation

4.1. Objectives and expectations
The evaluation will be initiated at the end of the project and will be based on contributions from the project stakeholders. It will be carried out by an independent evaluator and will be supervised by the project monitoring teams. RSF’s partners will be consulted for data collection.
The objectives of this evaluation will be :

• to provide the donor with an assessment of the effectiveness of the action’s implementation
• to provide RSF with a critical analysis of the work carried out under the program
• to propose areas of improvement for the implementation of activities Ukraine, and in response to other crises
• to assess the relevance of the partnerships tied to meet the project’s objectives
• to measure the impact of the action.

Recommendations will be formulated for RSF to identify and build on the strengths of its project, to exploit its margins of progress and to take into account the points of vigilance identified by the end of the evaluation.

4.2. Evaluation criteria and questions

Relevance and coherence

• Are the program, its activities and objectives still relevant and coherent with the needs of the target groups/partners?
• Are the different means of action deployed adapted to the political and cultural contexts of the target countries?
• Are the activities carried out and their observed results compatible with the expected impact and effects?
• Is the project’s intervention logic relevant?
• Is the program consistent with the activities already carried out by other organizations and does it bring added value?

Efficiency and effectiveness

• Have the resources mobilized (human, material, financial) for the program activities been adequate?
• How were the obstacles encountered overcome / bypassed? Were some activities reinforced / reduced / modified?
• Did the intervention logic of the project allow to reach a greater diversity of actors (profession, location, etc.)? To what extent?
• To what extent were the project results achieved?
• What was RSF’s capacity to respond to emergencies?
• How could the project have better reached and mobilized the beneficiaries (especially on psychological issues)?

Impact and sustainability

• Is the impact of the project already observable ?
• Did the intervention reach media and journalists with high audience ratings? In particular, audiences reached by disinformation on the war?
• How sustainable is the project? Have the possible negative consequences been taken into account?

Analysis of the partnership offer

• Did the partners have the necessary means to achieve the project’s objectives?
• How good was the synergy with the partners?
• How could RSF’s support to partners be improved?

4.3. Request for recommendations

Beyond the evaluation of the program and the recommendations relating to it, it is requested that the consultants be able to provide general recommendations with regards to RSF’s working practices and methodology. These recommendations should be formulated in such a way that they can be easily implemented by the teams. Please refer to the deliverables subsection for more information on the expected recommendations.

They should address the following questions in particular:

• How to strengthen the impact of the program in the target countries?
• What recommendations could be drawn from the implementation of the program to strengthen the organization’s overall strategy?
• How could the monitoring and evaluation system be improved to ensure better capitalisation on the results of the program (good practices and lessons learned)?

5. Evaluation methodology

Documentation :
All documents necessary for the evaluation of the program will be made available to the evaluator, including: program documents, evaluation questionnaires, interim and final reports from partners.

The stakeholders to be involved in the evaluation will be :

• RSF’s team:
  • Project and International Network Director
  • Head of Projects
  • Head of International Network and Capacity Building
  • Ukraine Project Officer
  • MEAL Officer
  • Advocacy and Assistance Director
  • Head of Investigation Desk
  • Head of Eastern Europe and Central Asia Desk
  • Press Freedom Center staff
• Executive Director of the Institute of Mass Information (IMI)
• Other Partners of the Project if needed

The evaluation of the program should be carried out in the most participatory way possible, and all relevant actors should be involved, including local partners and beneficiaries.

Calendar:
Desired start of the assignment 27 March 2023. A kick-off meeting will be held at the beginning of the mission and the inception report should be submitted no later than 5 days after the kick-off meeting.
The preliminary report should be submitted to RSF 10 days before the final report is due in order to allow for exchanges and feedback from RSF. Lastly, the final report, including feedback from RSF, should be submitted no later than 21 May.

Mission:
Due to the budget of the evaluation, the security context in Ukraine and security-related
costs on site, RSF favors remote interviews (online), rather than an on-spot mission to
Ukraine. All project partners and stakeholders can be contacted by videoconference.

Expected deliverables:
An inception report which will include:

• An updated work plan based on the documentation review and the kick-off meeting: methodology, tools, techniques, evaluative questions, etc.
• An updated presentation of the timetable
• A list of people who will be contacted

A first draft of the final report (preliminary report) will be sent to RSF for review and comments.

The final report incorporating feedback from RSF will include:

• An executive summary of a maximum of 5 pages including the main conclusions and recommendations resulting from the evaluation, written in English. Recommendations should be ranked in order of priority, marked from 5 (highly recommended) to 1 (optional recommendation).
• A main report including the context, objectives and methodology of the evaluation, the detailed findings and results of the evaluation in relation to the objectives and methodology of the evaluation, and the evaluator’s conclusions and recommendations. The report will not exceed 30 pages and will be written in English.
• In the appendix, the brief notes of the various meetings, the model questionnaires used, and any other relevant documents (the appendices may be in French or English).

6. Means

Preference will be given to the consultants with relevant expertise and proposing a
participatory and original evaluation methodology.
The following skills will be sought:

• Expertise in partnership evaluation;
• Strong project evaluation skills (methodology, interviewing, analysis, report writing, etc.), especially in crisis/war and humanitarian contexts;
• Experience in evaluating projects implemented by INGOs and/or international organizations (e.g., UN agencies);
• Expertise in logistics evaluation, including international shipment;
• Experience of working on EU-funded projects is an asset;
• Fluency in spoken and written English;
• Knowledge of French and/or Ukrainian is an asset.

The budget that can be mobilized for this evaluation is a maximum of €20,000 including tax and VAT. This amount must include all the costs necessary for carrying out the evaluation mission.

Proposals will be evaluated according to the following evaluation grid:

Understanding of the terms of reference /5

• Understanding of the purpose of the study /3
• Analysis and comments on the terms of reference /2

Methodology used /15

• Proposed steps for evaluation /2
• Actors studied /2
• Participatory dimension /5
• Suggested tools /2
• Planned progress point and feedback /2
• Duration and suggested calendar /2

Expertise /15

• Experience of the proposed consultant or team in similar projects /10
• Number and budgets of contracts in the same field as this call for tender /5

Means /15

• Budget proposed /10
• Clarity of the criteria used to calculate the budget /5
• Added value : valorisation of tools, particular expertise, etc.

How to apply

Consultants interested in the evaluation assignment should include the following documents in their application:

• A technical proposal detailing the understanding of the evaluation issues, the proposed evaluation methodology, as well as the envisaged implementation schedule.
• A detailed CV of the applicant(s).
• Description of previous relevant experience in the same field. Please give details of similar contracts: their amounts, implementation period, main results and providers.
• A detailed quotation.

Proposals must be submitted in English.

Full applications should be sent by email to the following addresses by the 5 March at the latest :

• Charlie Troncy: [email protected]
• Chloé Vernet: [email protected]

The first interviews can be organized around the 13 March.
Incomplete applications will not be considered.