Tshikululu Social Investments:Mid-term evaluation of the Programme Partners’ Black Academics Advancement Programme

Tshikululu Social Investments
Opportunity closing date: 
Monday, 26 October, 2020
Opportunity type: 
Call for proposals
 

Introduction

One of our clients working in the Tertiary sector in SA wishes to appoint a service provider to conduct a mid-term evaluation of the Black Academics Advancement Programme (BAAP). The outcomes of the evaluation will be used to review the current approach that the partners are taking towards the implementation of the programme, account for the difference the approach is making, document the implementation of the BAAP approach by the partners, and recommend how the approach (concept) can be revised going forward.
 
These terms of reference describe the scope and basic requirements of the mid-term evaluation.

Background and rationale

 
The first partner, in line with its transformation agenda towards changing the profile of the research workforce, has embarked on a strategic partnership with the second partner and introduced the BAAP. This programme is designed to promote the career development of black African South African academics, particularly black African males and females and people with disabilities, employed at public universities in South Africa, to attain doctoral-level qualifications and postdoctoral research training. This intervention spans a five-year period and aims to make 25 awards per annum to candidates in the PhD track to enable them to undertake full-time research to complete their doctoral degree and undertake postdoctoral research training, and 25 awards per annum to candidates in the post-PhD track to undertake post-PhD research training. The table below shows the number of grantholders reached to date.
 
Table 1: Achievements of the BAAP grant to date

 

2018 (cohort 1)

2019 (cohorts 1 and 2)

 

Target

Actual

Target

Actual

PhD track

25

21

50

47

Post-PhD track

25

17

50

28

Total for continuing and new awards

50

38

100

75

To assist the Funders to enhance their efficiency and effectiveness, Tshikululu would like to appoint a service provider to undertake a mid-term evaluation of the BAAP. The purpose of the mid-term evaluation is to review progress over the past two years, identify lessons learned of what has worked and what has not worked, and put forward recommendations for continuous improvement of the model. The Funders have particular interest in achieving the following:

  • determine the impact of the  to date;
  • understand the opportunities for improving the current BAAP approach;
  • document beneficiaries’ experience on the BAAP, and use the findings to suggest recommendations for adaptive management;
  • explore the effectiveness of key processes – the application process, recruitment process, payment process, management of the intervention process, stakeholder meetings and decision-making process;
  • explore the efficiency of the BAAP;
  • compare the BAAP to other programmes of a similar nature, e.g. New Generation of Academics Programme (nGAP) and Thuthuka Funding Instrument (TFI);
  • explore the BAAP’s complementarity to the Department of Higher Education and Training’s (DHET) objectives and other key role-players in the tertiary sector;
  • compare to the aims and outcomes of the report of the ministerial task team on the recruitment, retention and progression of black South African academics;
  • explore why targets have not been met thus far;
  • determine if the reporting process used is effective;
  • explore whether the marketing platforms used for recruitment are effective;
  • establish what other strategies can be used to improve other institutions’ participation;
  • use the findings to document learnings; and
  • use these learnings for a revised strategy that is reflective of the Funders  systemic social investment (SSI) and approaches.

Evaluation aim
 
General
The overall purpose of the evaluation is to determine progress made thus far of the BAAP in terms of the fulfilment of objectives, efficiency, effectiveness, sustainability and impact. The evaluation will also offer a learning aspect for all stakeholders through capturing programme achievements, challenges and best practices to inform future programming.
 
Specific
The evaluation is expected to answer the following questions

Criteria General Specific
Impact What is the BAAP’s impact to date?
 
What strategic action can be taken that can extend the impact of the BAAP?
  • What are the elements of the BAAP that make the programme successful?
  • How does the BAAP enhance complementarity?
  • How does the BAAP match up to the aims and deliverables of the report of the ministerial task team on the recruitment, retention and progression of black South African academics?
  • Determine the efficiency and effectiveness of the BAAP?
  • What is the BAAP’s emerging theory of change?
  • What value has the BAAP added to the lives of grant holders?
  • How can we track the individual grant holders over 10/20 years?
  • How has the BAAP been able to address the grant holders’ needs?
  • How effective has the BAAP’s reach been to grant holders with disabilities, and what are the hurdles if not effectively reaching grant holders with disabilities?
  • How have the grant holders experienced the BAAP?
  • How did grant holders experience the time buy out?
  • To what extent have grant holders/graduates been integrated into their academic posts after completing their PhD or post-doctoral training?
  • Did the grant holders go back to the same institution they attended before going for their studies?
  • How have grant holders progressed since obtaining their qualifications?
  • What future do the grant holders have academically?
  • Have the grant holders’ NRF ratings progressed?
  • What are the common themes among the grant holders in terms of their career plans?
  • What components of the bursary programme have contributed to the grant holders’ employability?
Relevance What is the extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies?
  • Which other funders are funding similar programmes to the BAAP?
  • Compare and contrast the BAAP to the nGAP and the TFI?
  • Which elements of bursary funding does nGAP and TFI provide?
  • What relevance does the BAAP have in the current bursary funding landscape?
  • How does the BAAP match up to the aims and deliverables of the report of the ministerial task team on the recruitment, retention and progression of black South African academics?
Effectiveness What is the extent to which the BAAP’s objectives were achieved, or are expected to be achieved, taking into account their relative importance?
  • What has the programme achieved to date:
    1. graduation;
    2. funding;
    3. links – networking/leveraged efforts;
    4. publications;
    5. student supervision;
    6. employment/promotion; and
    7. NRF rating progression?
  • What difference has the mainstreaming of disability concept made in the lives of programme beneficiaries:
    1. What has been learnt in implementing the mainstreaming of disability strategy?
    2. What can be done to strengthen the process of mainstreaming disability within the BAAP?
Efficiency How did the BAAP economically use resources/inputs (funds, expertise, time, etc.) and converted them to results.
  • What is being learnt during the implementation of the BAAP?
  • What can be done to strengthen the BAAP in future?
  • What are the reasons for not meeting annual targets?
Sustainability Is there sustainability of the BAAP? Are there going to be benefits long after the programme has been completed?
  • How is the programme strategy being implemented within the Funders’ objectives?
  • What are the major benefits of the BAAP post intervention?

Evaluation methodology and activities

The service provider is expected to use both a quantitative and a qualitative approach. In addition to the below provided activities, the service provider may decide to add on to the evaluation methodology and activities.

Quantitative approach

The key activities of the service provider during this assignment will be to:

  • review project monitoring data and qualitative reports (to be provided by Tshikululu);
  • analyse quantitative monitoring data using Excel, PowerBI, SPSS, R or any quantitative software and produce trends to be included in the evaluation report; and
  • develop quantitative survey tools for the evaluation.

Qualitative approach

The key activities of the service provider during this assignment will be as follows:

  • a desktop review of relevant documentation, including strategic frameworks, research work and prior evaluation reports, narrative reports, landscape analysis, monitoring and evaluation (M&E) data, and other programme-related documentation;
  • interviews, surveys and/or focus group discussions with grant holders and implementing universities, as well as with other stakeholders as deemed necessary. Due to COVID-19, data can be collected virtually. However, there might be a need to conduct some physical site visits;
  • conduct an in-depth literature review of the tertiary education landscape in South Africa and compare this to other tertiary funding models regionally or internationally;
  • an in-depth review of the report of the ministerial task team on the recruitment, retention and progression of black South African academics;
  • compare the funding experience of the  grant holders with other funding programmes – for example the nGAP and the TFI;
  • propose a representative sample for interviews and a survey;
  • design the evaluation methodology data collection tools (both qualitative and quantitative);
  • conduct interviews with students, the Funders, the DHET and other key stakeholders at tertiary institutions, and subject matter experts in tertiary education funding;
  • administer survey questionnaires via email, SurveyMonkey and telephonically, or any other electronic platform as necessary;
  • capture and verify data;
  • analyse data using qualitative software and prepare the report findings; and
  • produce a draft report, an executive summary of the evaluation and a final report (taking into account comments and feedback from both Foundations via Tshikululu Social Investments), and an edited version of the final report that does not disclose the identity of key stakeholders involved in the evaluation.

Potential evaluation participants:

  1. Project beneficiaries;
  2. key informants from the Funders
  3. key informants from universities participating in the BAAP;
  4. key informants from the selection panel (which comprises first partner trustees, Tshikululu and representatives from the second partner); and
  5. key informants of duly performed universities that did not participate in BAAP.

Responsibilities:

Tshikululu will be responsible for the necessary introductions to stakeholders that the service provider may need to engage with during this assessment. Tshikululu will also be responsible for facilitating the provision of all necessary information/documentation to engender effective implementation of the assessment. In addition, Tshikululu will be responsible for the overall administration of this project.

Required expertise

The service provider will:

  • demonstrate its commitment to transformation via provision of a broad-based black economic empowerment (B-BBEE) certificate and, where applicable, transformation policies and plans;
  • have experience in conducting similar evaluations;
  • have an appropriately capacitated team to complete the work;
  • have a thorough understanding of the tertiary education landscape in South Africa;
  • conduct this assignment in a professional manner;
  • follow all steps necessary for this assignment as articulated; and
  • store and hand over all information generated through this assignment to Tshikululu.

 Documents required:

In addition, the service provider will provide the following documents as part of its proposal:

  • overview of the company, including its directors and management’s credentials;
  • tax compliance/exemption certificate/company registration;
  • proof of legal registration of the company;
  • B-BBEE certificate/affidavit;
  • business resumé;
  • organisational transformation plan;
  • team member CVs; and
  • project proposal, including the budget, sample size, credentials of similar work done and traceable references, data analysis software to be used, methodology and workplan.

Timeline of activities

Activity

Date

Release of RfP

12 October 2020

Submission of Proposals Due Date

26 October 2020

Appointment of service provider and contracting

9 November 2020

Beginning of review

16 November 2020

Submission of draft report to the Funders

8 March 2021

Submission of final report to the Funders

29 March 2021

Budget

The estimated cost for this assignment is between R550 000 and R650 000.
 
Completed proposals, together with supporting documents, should be sent to Nyaradzo Mutanha, M&E specialist, Tshikululu Social Investments, at nmutanha@tshikululu.org.za. Please submit all proposals no later than 26 October 2020 (by 17:00). Proposals received after the closing date will not be considered. 
 
The Foundation reserves the right not to award this contract, in whole or in part.

For any queries and submissions, please contact Nyaradzo Mutanha: nmutanha@tshikululu.org.za

For more about Tshikululu Social Investments, refer to http://tshikululu.org.za/

For other vacancies in the NGO sector, refer to http://www.ngopulse.org/opportunities/other

Location: 
South Africa

NGO Services

NGO Services

NGO Events

S M T W T F S
 
 
 
 
1
 
2
 
3
 
4
 
5
 
6
 
7
 
8
 
9
 
10
 
11
 
12
 
13
 
14
 
15
 
16
 
17
 
18
 
19
 
20
 
21
 
22
 
23
 
24
 
25
 
26
 
27
 
28
 
29
 
30
 
31