Building integrity in Kenya Citizens demanding better public buildings and services in Kenya: a story in data

1. Monitors found a lot of problems and got most of them fixed

One of the key pieces of data generated through DevelopmentCheck concerns “problems” and “fixes”.

A citizen monitor reports a problem when they see something which should be the case, but isn’t – like a construction project that isn’t using the correct quality of materials.

She would then report a fix if that problem is subsequently remedied to her satisfaction. Perhaps the building contractor secures the right materials after the issue was spotted.

In the VOICE programme, citizen monitors reported 2001 problems and 1661 fixes – that’s a “Fix Rate” of 83%.

Here’s what that looked like over the course of the project (this analysis is for November 2019 – January 2022):

This shows the volume of problems was highest at the beginning, and that the running Fix Rate – the % of problems solved, as judged by the monitors – rose quickly in the first 6 months.

The data presented here covers 168 projects monitored in the VOICE programme, of which 153 were infrastructure projects. (The remainder were public service points and other public projects. We use the term “projects” to describe all the things that were monitored.)

This means there were, on average 12 problems and 10 fixes per project monitored.

2. One of the biggest impacts was on delayed construction

During each visit to a project site, the citizen monitors used DevelopmentCheck to work through a “checklist” of possible problems. The checklist was created in consultation with the organisations we work with and other stakeholders. Looking at infrastructure projects only, the most frequent problems reported (by some distance) concerned project delays:

The effect on delayed projects was corroborated by other stakeholders. Hannah Ngala (pictured), a local government official in Kwale, said that “most projects which had stalled for a long period are at the completion stage through [the monitors’] efforts.”

The chart above also shows the Fix Rate for each type of problem. Intriguingly, the highest Fix Rate is for “signs of corruption”, although this category seems to have been used to report problems already logged in other categories. The lowest Fix Rate – and the only Fix Rate under 80% - is for a lack of access to project documents. This was clearly a challenge for many monitors and, as we shall see, the availability of these documents has an impact on the success of monitoring.

As well as the frequency of problems and fixes, we can also look at how long it took for fixes to be achieved. The following chart shows average “fix times” for the different types of infrastructure problem (with the problems ordered the same as in the above chart, for ease of comparison).

Among the “slowest” fixes are ones which might require the subcontractor to secure additional resources and/or do additional work. For example, an unsafe project site might require the addition of a perimeter fence. “Project does not consider disability” may relate to a lack of access ramps and other accessibility measures which, again, would need to be constructed in order for the problem to be solved.

3. Citizen monitors stayed engaged throughout the programme

Mwanatumu’s enthusiasm was shared by many of her fellow citizen monitors. When asked about their continued involvement, they cited reasons including gaining new skills, the opportunity to collaborate with others, and securing solutions to the problems they identify.

We can also see this continued engagement in the data – this chart shows the number of times monitors submitted data on the DevelopmentCheck app each quarter (specifically on problems and fixes), alongside the number of projects being monitored at any one time.

There was a significant drop in monitoring activity caused by the COVID-19 pandemic, around the second and third quarters of 2020. However, it is notable that activity levels didn’t drop further. We heard from monitors that they were keen to continue monitoring construction during the pandemic, where this was permitted, because they knew building contractors were continuing to work and were concerned those contractors would fail to reach the right standards if they were not observed.

The level of data submissions also drops significantly in the last quarter – this is because the data collection phase of the project was winding down at this stage. Keep reading for more on what’s happening now in the VOICE programme.

4. Female monitors became more confident to participate and lead

Of the 126 citizen monitors taking part in VOICE, 66 (52%) were women. Uchi Chidunga, one of those women, said, “Being a monitor has prompted me into improving my communication skills, negotiation skills, together with improving my self-esteem. Previously I was not sure of my capability but after the trainings my self-esteem grew and then the rest is history. Most women and girls in the community view me as their mentor towards women’s empowerment.”

One way to explore any change in the confidence of women is to study the data on how problems were solved. When monitors report a “fix” in DevelopmentCheck, they have the option to also report the most important thing they did to secure that fix. Monitors don’t always report a “solution type”, but in about a quarter of fixes they did. This chart shows how the four most common solution types changed from (roughly) the first half of the VOICE programme to the second half, for women and men.

There was a big increase in the proportion of women saying they organised a community meeting – a very public forum for problem solving. Jacob Wambua from KYGC, one of the organisations coordinating the monitoring in Kwale county, said they had also observed this: “The monitoring activities and fixing problems has improved the female monitors’ confidence as well as making them visible to the community. They are not only being consulted, but also speak up during community meetings, unlike earlier in the VOICE programme.”

It is notable that, for both women and men, there was a decrease in their use of a “Joint Working Group” or monitoring committee - this is a regular meeting with key people responsible for the project in question. This might bode well for sustainability because the Joint Working Group was a new platform set up as part of the monitoring approach. It is no bad thing – and a useful lesson learned – if it ends up not being important for problem solving in this context.

5. We saw improved relationships between citizens and duty-bearers

One of the results of the VOICE programme was improved trust and relationships between citizen monitors and the people responsible for the projects being monitored. These “duty-bearers” might include the building contractor and local government officials at different levels. This result was highlighted by citizen monitors, duty-bearers and an independent evaluation of the programme.

Hannah Ngala, a local official working at the village level, said: “The relationship between my office and the community at large has improved. I trust monitors because the issues they raise are genuine and whenever we visit the project sites to confirm, we realize that the claims are true, hence it becomes easier to arrive at fixes.”

When we look at this through the data, however, it doesn’t tell quite the same story. For the monitoring of infrastructure projects, DevelopmentCheck asked monitors to record how helpful or unhelpful different stakeholders were being within the monitoring process. This chart shows the aggregated responses over time, focussing on the proportion of responses within each quarter which said a stakeholder was being “unhelpful” or “very unhelpful”. (The other available responses were “helpful” or “very helpful”.)

“PMCs” are project management committees – a community-based committee that is often set up to oversee a local construction project. Monitors within the VOICE programme worked closely with PMCs and in some cases shared skills with them, such as how to read project documents.

The chart firstly shows that levels of unhelpfulness were low overall – never going above 20% for any stakeholder in any quarter. We also see that contractors were consistently rated as the most unhelpful. This isn’t surprising because, in order to fix problems, they were sometimes being asked to improve labourers’ pay, find better materials, or even redo parts of their construction.

Any trends over time are minor – perhaps the clearest is the trend towards more helpful communities and PMCs, from a very helpful starting point.

Chirunga Mwachondo (pictured), a citizen monitor in Kwale, told us, “Initially the community members never showed support in the monitoring activities but after seeing the good work we are doing they supported us fully and they even volunteered in the monitoring exercise.”

However the trend for local authorities is less clear. What’s happening here? Based on the independent evaluation’s findings, we think this may be because the term “local authority” is too broad, and while we did witness better relationships with village-level officials like Hannah Ngala, relations were more challenging with mid-level civil servants in the county administration.

6. There are barriers to public participation – and citizen monitors helped communities to overcome them

Citizen monitoring is primarily about scrutinising delivery of public projects and services, but it is equally important for citizens to have a say on what to deliver in the first place. This is challenging for many residents in Kwale county.

Elsheber Oketch of KYGC explained that public participation forums, where citizens can feed into budget decisions, “are normally held at the Ward level, [but] the Ward headquarters are not accessible to the majority of the communities from other village units. In addition, communication and advertisements regarding the forums are always made through print media, which does not reach the people in the rural areas.”

We saw this in the data as well. As well as reporting problems and fixes, citizen monitors also surveyed their fellow community members about their experiences of the project being monitored. 57% of those surveyed (n = 1865) said they had been involved in decision making relevant to the project in question. This figure did not vary significantly by gender; it was lower for respondents aged under 18 and over 60; and it was higher for the 11% of respondents with a self-reported disability.

This does raise the question of what the right level of engagement is; if 57% of respondents were involved in decision making, the situation could certainly have been worse. However when the 43% were asked why they were not involved, the responses revealed an appetite to be engaged, alongside barriers preventing that engagement.

This shows that, of those who were not involved in decision making, 87% either didn’t know about opportunities for participation, or found the location or timing inconvenient. Only 9% of those respondents said they were not interested, or that their involvement wouldn’t make a difference.

What have citizen monitors been doing about this? Knowing that few people from remote areas can attend the forums, they collect inputs in advance and then present them at the meeting. Elsheber Oketch said that they “agree on who will say what, and how the other [monitors] will back up the points raised. At the end of the forum, they present their written document containing their list of priorities to add more weight to what they raised during the discussions.”

Elsheber said this had brought results. “Community members confirmed that there are a number of projects that they had proposed, some of which are pending approval by the county assembly, while for some implementation is expected to start soon.”

7. When monitors have project information, it’s easier to solve problems

Within the VOICE programme, citizen monitors reported whether key documents were available for each project. These documents can include details like what the project is supposed to do, when, by whom, and at what cost. For construction projects, the most important document is the “bill of quantities” which contains the specifications for the construction.

The data tells us that the Fix Rate for projects where no information was available (36% of all projects monitored) was significantly lower than for other projects.

This should be considered alongside the earlier finding that a lack of information was the hardest type of problem to solve. We now see that it makes other problems harder to solve too. This aligns with Integrity Action’s research published in 2021, which found that informed citizen action was one of the key enablers of successful problem solving by citizen monitors, as well as our learning paper published in 2022 on what information helps citizens demand accountability and improvements to services. (It’s notable that the highest Fix Rate was for projects where only some necessary documents were seen. This is likely connected to the small number of projects in this category, however.)

This backs up the hypothesis that making project information available is an important measure for improving the accountability and quality of project delivery – though this needs to be accompanied by other measures, such as training citizens on how to interpret this information, and providing platforms for dialogue and problem-solving. Citizen monitors in VOICE received training on interpreting project documents and shared these skills with others in the community.

So how was this data used?

KYGC and KCNRN, the organisations implementing the monitoring programme in Kwale county together with Integrity Action, both shared data visualisations with relevant officials in the county government. This included information on the volume and nature of problems being identified, as well as where they are manifesting. They have also been sharing visualisations on the levels of monitoring activity, to demonstrate the commitment of citizens and boost the legitimacy of the overall process. Meanwhile some of this information has been useful for supporting citizen monitors in near-real-time, such as knowledge of which projects are reported to have the most unhelpful stakeholders.

We are keen to better understand what forms of data are most useful to public officials, so as well as hearing their feedback we have recently commissioned a piece of research looking into this. We hope to share results later in 2022.

Integrity Action and partners also use this data to learn, and to share what we learn. We are happy to share data for others to explore – if this sounds interesting, please get in touch (data is appropriately anonymised).

What’s happening with the VOICE programme now? At the time of writing (April-May 2022) the programme is still ongoing, but is shifting away from DevelopmentCheck data collection. The strategy is to use the results of the monitoring approach to advocate changes to existing public participation processes that could mirror some of the successes we have seen. We look forward to sharing more on this in the coming months.