This is a copy of the paper we submitted and had published at CHI 2016. I wanted there to be a HTML version. I converted from LaTeX src to Markdown for posting on this site but there’s a few errors I need to fix up before it becomes properly readable. Bear with me
Authors: Matthew Marshall, David S. Kirk, and John Vines
- Designing Accountable
- Study Design
Increasingly, governments and organisations publish data on expenditure and finance as ‘open’ data in order to be more transparent to the public in how funding is spent. Accountable is a web-based tool that visualises and relates open financial data provided by local government and non-profit organisations (NPOs) in the UK. A qualitative study was conducted where Accountable was treated as a technology probe, and used by representatives of NPOs and members of the public who invest their time or effort voluntarily into such organisations. The study highlighted how: current open data sets provided by public bodies are inadequate in their representation of funding structures; the focus on finance and fiscal expenditure in such data makes invisible the in-kind effort of volunteers and the wider beneficiaries of an organisation’s work; and problems arising from the interoperability of open data technologies. The paper concludes with implications for the design of future systems, considering the domains of transparency and accountability in relation to the findings.
This paper discusses the design, development and evaluation of “Accountable” – a web-based tool that visualises and relates financial data provided by non-profit organisations (NPOs) and local governmental authorities (LAs). Accountable was designed to allow the general public to explore and understand the fiscal practices of trusted organisations. In the UK, the financial data of NPOs and LAs are systematically uploaded and presented online as a requirement of financial transparency and auditing protocols set by the government @newcastle_city_council_payments_2015 [@northumberland_county_council_payments_2015; @gov.uk_register_2015]. Although its purpose is to allow for public scrutiny, this data is not rendered easily accessible and exists in unwieldy formats such as large spreadsheets of supplier payments or aggregated expenditure figures. Data placed online has the potential to feed into the practice of Open-Source Intelligence (OSINT), which seeks to solve problems and answer questions using publicly available sources of information @steele_open-source_2012 [@bazzell_open_2015; @glassman_intelligence_2012]. This suggests that data regarding NPO and LA finances could be used for citizen interrogation of their practices, or for demonstration of the work that they perform. We designed Accountable to explore and support such practices, and to understand the opportunities and limitations of existing ‘open’ financial data provided by these organisations.
In order to examine these issues, we conducted a qualitative study, where representatives and volunteers of NPOs discussed the work they do and issues around financial transparency, and were asked to interact with the Accountable system. Our interviews provide insight into the structure and nature of how NPOs interact with LAs as local funders, spend the funds they receive, and produce accounts for governing bodies. We also highlight tensions between the perceived and actual values of back-end work such as administration in organisations that operate on a not-for-profit, or charitable basis. Building on previous work in organisational transparency and non-profit performance measurement @henderson_performance_2002 [@fox_uncertain_2007; @hood_accountability_2010], this paper comments on the potential of future systems to support the accountability of NPOs. We suggest how interactive financial systems can better reflect the work of NPOs, and discuss the data standards and requirements necessary to facilitate this.
The contribution of this work to the field of HCI is twofold. First, the paper begins to explore the relationship between the fiscal practices of trusted organisations and their stakeholders. In doing so, we build upon previous work around finance and technology in HCI by exploring interactions in an organisational context. We also open a new avenue of financial HCI research concerned with interactive financial data supporting transparency, accountability, and the impact of the work these organisations do. Second, the paper presents recommendations for the design of systems which seek to engage with fiscal transparency in the non-profit sector, including designing for visibility, how visibility might be achieved through capturing qualitative financial data, and the use of standard data formats to allow systems to support interrogation. We thus outline how HCI research and the design of interactive systems could affect global dialogues on transparency.
In most democratic societies, fiscal transparency and accountability are increasingly desirable qualities in both governance and business practice @hood_accountability_2010 [@oliver_what_2004]. The popular narrative nods towards a causal relationship where increased transparency leads to increased accountability; the more information about an organisation that is openly available increases the ability of interested parties to call its decisions into question @fox_uncertain_2007 [@hood_accountability_2010].
NPOs and LAs in the UK have a legal obligation to be transparent. The Charity Commission collects and displays the financial data and reports that are provided by registered charities and NPOs as part of their legal requirement for charitable status @gov.uk_how_2013. Likewise, the Local Government Transparency Code @gov.uk_local_2015 stipulates that LAs publish a list of all their expenditures over a certain amount online @newcastle_city_council_payments_2015 [@northumberland_county_council_payments_2015]. Charities submit annual accounts and reports, as well as complete a Summary Information Return (SIR) reporting on their activities. The development of trust is a keystone in the relationship between an organisation such as an NPO or LA and its stakeholders; those who have an interest or claim in the organisation’s activities and performance @krashinsky_stakeholder_1997 [@macmillan_relationship_2005; @ortmann_trust_1997; @conroy_non-profit_2005; @eikenberry_marketization_2004; @jacobson_lifting_2005]. This is compounded by the fact that a stakeholder funding an NPO might not necessarily be the one in receipt of its services @macmillan_relationship_2005. Commercial organisations are commonly said to be performing well, or meeting their goals, if they make money. The nature of an NPO, however, doesn’t lend itself to traditional performance measurements such as ‘bottom-line’ profits when establishing the organisation’s effectiveness @henderson_performance_2002. Instead, measurements might focus on how much the organisation spends on its services or projects versus how much is spent on management or general costs. It is still difficult, however, to determine whether a program has met its goals; all that is fully known is that money has been spent on it, failing to capture value that is non-monetary @henderson_performance_2002. Non-financial outputs can be communicated (i.e. what the NPO actually does), but this information can be hard to capture systematically, and demonstrating the value of work is a global issue for NPOs @henderson_performance_2002.
The assumption that transparency equates to accountability has been questioned. Fox argues that transparency covers the dissemination and access to information, with accountability being the measure of how, and to what extent, the public can call on those in authority to justify their decisions and issue sanctions @fox_uncertain_2007. Fox concludes that there exists an overlap between transparency and accountability known as “answerability”, where information is freely available and can be used to produce answers about institutional behaviour – although this can only be achieved when the institution is appropriately transparent @fox_uncertain_2007. Schauer notes that “To be transparent is to have the capacity of being seen without distortion”, and that for information or processes to be transparent, they must be “open and available for scrutiny” @schauer_transparency_2011. This definition notably lacks commitment as to who may take advantage of this transparency, how they can be expected to do so, and how this generates accountability @schauer_transparency_2011 [@oliver_what_2004; @hood_accountability_2010]. An “open” data set about an organisation’s finances may be online, but whether this reveals anything about its behaviour or provides a mechanism for scrutiny is questionable @schauer_transparency_2011 [@fox_uncertain_2007]. Furthermore, both Schauer and Oliver do not equate transparency with knowledge, describing transparency as a facilitator of knowledge “at best” @schauer_transparency_2011, and a journey instead of a destination @oliver_what_2004.
The open data based on the accounts of local governments and charities in the UK is an example of what Heald calls input transparency, which fundamentally lacks context @heald_fiscal_2003 [@heald_varieties_2006]. It is data that describes resource allocation (as can be seen in Figure [screen-inputdata1]), and is easily measurable; however, it does not address links between the input and the outcome @heald_fiscal_2003 [@heald_varieties_2006]. Simply exposing spending data does not qualify as transparency since reporting the amount spent on a service does not indicate how well funded it is; nor does it hold the organisation accountable to its decisions @fox_uncertain_2007. Schauer and Oliver respectively describe notions of passive transparency and old transparency, which is the view that simply making data available in this fashion is enough – and that an organisation does not need to concern itself with the promotion or interpretation of the data published @schauer_transparency_2011 [@oliver_what_2004]. In contrast, they each promote alternative notions of active transparency (Schauer) or new transparency (Oliver). Both of these notions regard transparency as an act of communication that concerns itself with the organisation’s responsibilities @schauer_transparency_2011 [@oliver_what_2004]. Fox similarly describes a clear transparency that describes policies, information access, and data about the behaviour of institutions – which can promote answerability @fox_uncertain_2007.
HCI, Money, and Transparency
In recent years, HCI began to study the relationships that individuals have with money and finance. Work around the design of technology to support budgeting and the management of a range of different incomes has highlighted the ways in which technology typically acts to obscure finances, rather than to make them more transparent, and easily accounted for @vines_pay_2014 [@kaye_money_2014; @snow_fostering_2015]. The values placed on transactions have also been explored both in an individual context (such as the banking practices for the older old) @vines_eighty_2011 [@vines_joy_2012], and at a community scale (such as the Bristol Pound or at emergency relief centres) @ferreira_spending_2015 [@vyas_more_2015]. However, research in HCI thus far has primarily focused on issues of personal finance or how members of communities respond to new financial systems and services; there has thus far been no work venturing into understanding what role technology may play in supporting fiscal transparency and the accountability of organisations and institutions.
Recently, tools and platforms have been developed that take open data (input or passive transparency) and attempt to present it in a manner that allows interested parties to interrogate it to a deeper degree. TheyWorkForYou @mysociety.org_theyworkforyou_2015 is an example of a service, which takes published UK parliamentary data (via the Parliament API @houses_of_parliament_developing_2015) and transforms it into a simple format to be used by the general public. The site provides comparisons of Members of Parliament (MPs) with other MPs and breakdowns of how a single MP has voted on various issues @mysociety.org_theyworkforyou_2015. TheyWorkForYou stands as an example of how Passive Transparency by an organisation (in this case the UK Parliament) can be transformed into a form of Active Transparency that potentially makes an individual MP accountable for their actions in parliament. With this as a starting point, this research seeks to answer the question: what are the requirements for interfaces that support active transparency, in particular around NPO and LA financial data?
Accountable is a small-scale web tool that was designed for the purpose of exploring the ways in which future systems might support new transparent fiscal practices for NPOs, LAs, and their agencies. For the purposes of our research, we framed Accountable as a technology probe. Hutchinson et al. describe technology probes as having engineering, social, and design goals. As such, Accountable was intended to explore: the engineering challenge of scraping and reading unorthodox and irregular online datasets; the design challenge of visualising this data in a way that is interpretable and interrogatable by members of the public; and the social challenge of representing the practices of value-driven organisations in both monetary, and non-monetary terms @hutchinson_technology_2003. The name ‘Accountable’ was derived from the notion that trusted organisations should be accountable to their stakeholders, and that this should be seen as a proactive act of demonstrating their value instead of a response to auditing processes.
In the following sections we discuss the open data that underpins the Accountable systems, the way in which different data is visualised and represented by the tool, and the details of our study design and subsequent findings.
As discussed, NPOs and LAs are held to different transparency requirements by the UK government. They also interact frequently, with NPOs often acting as service providers for LAs. As NPOs often seek contracts or funding from LAs, it was decided to explore LA datasets in addition to NPO data in order to understand how the practices of NPOs are affected by LA transparency requirements.
In the UK, NPO data is presented online via the Charity Commission’s website. However these are produced in the charts and graphics generated via embedded Flash applications, making it difficult to scrape or parse useful data from the site. Individual reports produced and submitted by the NPOs themselves are also stored on the site in a downloadable PDF format; this presents another challenge for automatic retrieval and parsing.
In order to produce an interface on Accountable for participants in our study to interact with, a single NPO was selected and the data presented on the Charity Commission site was manually entered into the system. For the purposes of our study we selected a local branch of a well-known national NPO, chosen because the name would be familiar to participants. As with most financial data of NPOs that appears on the Charity Commission site, the data in this case primarily related to aggregated expenditure across categories such as “Charitable Activities” and “Investment Management”. Accountable used this data to present calculations focused on comparing percentages from the 2014 income, 2014 expenditure, and financial history (2010 - 2014) of the NPO.
The data representing the accounts that LAs are required to publish on their websites was acquired from two different English LAs (LA01, LA02). This data is related to all spending over 250 for various services, and provides details about the expense such as cost, vendor contracted, and the department responsible (among others). Both of the LAs provided this data in PDF and CSV formats.
Attempts to automate the retrieval and parsing of the machine-readable LA datasets were met with issues stemming from heterogeneous data structures. The data provided by each LA was structured very differently, each requiring a bespoke parser, and was inconsistently formatted, including between data taken from the same LA. The subset of each LA’s 2014 data was deemed appropriate for the purposes of the study, and each were internally consistent enough to be parsed effectively. Heterogeneity between the two datasets was a result of differing LA structures. It was necessary to map data to a common format that could describe data from each source. The vendors or suppliers in receipt of LA funds were easy to determine from the data, but was difficult to categorise each transaction (i.e. what funds were spent on).
Accountable was designed primarily to read data provided by NPOs and LAs, and present it in an easy-to-digest format to promote Active Transparency and encourage scrutiny and interrogation of the data.\
NPO Data Presentation
The presentation of information for NPOs was largely inspired by TheyWorkForYou website @mysociety.org_theyworkforyou_2015, which as noted uses parliamentary api data to present sentences in “plain english”, so that laypeople can compare MPs. This technique was applied to NPO financial data in Accountable, as an attempt to better describe their activities to interested parties.
Since, for the purposes of our study, Accountable was representing only a single NPO’s data, a “Quick Numbers” section (Figure [screen-accountable]) was developed in lieu of comparison with other NPOs. Using calculations based on the financial data presented by the Charity Commission, this section highlighted financial expenditure and income in various circumstances such as earning money through shops or spending funds on supplying services. The number of volunteers and employees are also presented. Additionally, whereas the Charity Commission often use terms such as “Governance” or “Trading to raise funds” to describe data in Accountable we avoid excessive use of this terminology. Instead data was described in simpler terms (e.g. “activities such as running shops or hosting events”). Finally, financial history was also presented as a chart (Figure [screen-chart]), with data from 2010 up until the most recently available records (2014), to complement the “Quick Numbers” section.
LA Data Presentation
Accountable allows users to scrutinise LA spending via comparing spends on services to another LA. They can select a category from each of the LAs to compare spending between the two organisations and services (Figure [screen-accountable]). These categories represented how expenditure in LAs is divided into its services. Care was taken with wording when describing expenditure, as the data does not necessarily pertain to what money has been spent on, only the service or department in the LA that spent it.
Records that had missing or redacted information were highlighted on the page to support scrutiny. The system defined missing information as any empty field, while redacted information is any field containing the word “redacted”.
As with the NPO data, expenditure data is also displayed on the profile in graph and table form to provide an overview. Displayed are the Total, Largest, Smallest and Average spends by the LA, a breakdown of the spending “via service”, and the Top Vendors by both frequency (number of payments) and payment total (total paid to a single vendor). The number of vendors to display can be configured by the server, and was set to 10 for demonstration in the study.
Our study focused on NPOs because of their position as both a producer and consumer of fiscal transparency. Furthermore, austerity measures in the UK have affected the services that LAs are able to deliver themselves, and therefore NPOs often seek service contracts or funding with LAs. As such, NPOs also represent stakeholders interested in LA financial data.
To gain insight into different aspects of financial transparency, the study aimed to explore perspectives on the data from representatives of NPOs as well as members of the public who had a self-stated interest in the activities of these organisations. Participants were recruited to take part in individual interviews or a group workshop depending on the nature of their involvement with NPOs. Those involved in volunteering, fund-raising, or donating to NPOs were invited to workshops to promote discussion and partake in a constructive critique of the Accountable and Charity Commission systems. Those representing an organisation in an official capacity, such as managers or trustees, were invited to interviews. This was in order to provide a space where they may feel more comfortable discussing their organisations’ potentially sensitive financial matters and practices in private. Participants are listed in Table [tab:participants].
Participants and Process
Representatives of NPOs were recruited for interview via an independent charity whose remit includes supporting the voluntary and community sector in Northern England.
Workshop attendees were recruited with a call distributed through our institutional mailing lists and social media. Explicitly requested were participants who regularly donated time, money, or effort to one or more NPOs. These participants were offered a 10 gift voucher or a donation of the same amount to an NPO of their choice for participation.
The interviews and workshops shared the overarching ambitions of exploring: how financial processes inside NPOs were represented; what NPOs and their stakeholders desired to see about NPO finances; and how they desired this to be communicated. The investigation process was adjusted to suit the context of an interview or workshop, as detailed below.
(r)<span>2-4</span> <span>**Participant**</span> <span>*Gender*</span> <span>*Interview / Workshop*</span> <span>*Role(s)*</span>
: Details of Participants.
A total of 7 semi-structured interviews were held with a variety of representatives from different NPOs. One interview had two interviewees from the same organisation, totalling 8 interview participants. Organisations each focused on different issues including the support of parent and toddler groups, support for those with mental disabilities, and community development. Most organisations focused on localised settings (i.e. community support in a specific area of the city), although one organisation supported other organisations across the local region. Each interview lasted approximately 30 to 45 minutes. Participants were either directly involved in daily activities as managers, or were trustees of the NPO.
During interviews, participants were first asked about the work that their charity did and the techniques that they used to achieve this. The interviews then discussed how the NPO spends and divides up funds based on the work that they do. Participants were also asked about how their organisations sought funds, interacted with an LA (or multiple LAs), and presented their work and achievements to parties such as the Charity Commission and others interested in the organisation.
During interviews, participants were shown the Accountable prototype, which was pre-loaded with data from the local branch of a popular UK charity. Participants were asked to imagine how the data in the system could represent their organisation, and how they felt the data communicated the relationship between their work and financial practices. Discussion was prompted around the critique of Accountable, directed towards the data, its presentation and communication by the system. Following this, participants were asked about their interactions with the LA, and shown copies of printed LA data and LA profile pages in Accountable. Again, participants were asked about their impressions of the data – what they felt was, and was not, shown. Participants were asked about what they, as a representative of their organisation, would like to see about LA spending and how it pertained to their activities.
A total of 2 workshops were run, each with 3 participants, lasting between 60 to 90 minutes, split into three main activities. Each participant had experience working with one or more NPO – their experiences included work with small-scale organisations such as support groups, drugs information and advocacy, Non-Profit festivals and events, as well as larger multinational NPOs. Workshop activities were designed to promote rich discussion for analysis alongside interview data.
To begin, participants were asked to introduce themselves by discussing recent work that they had done for an NPO, in what ways they felt that their contribution had helped the organisation (or otherwise) and how the organisation had communicated this to them. Following this, for the first activity participants were introduced to “Janet”, a persona developed for the workshops to support discussion around the practices of an NPO @mcquaid_when_2003 [@miaskiewicz_preliminary_2009]. Janet was described as often volunteering time, money, and effort to a popular NPO’s local branch. The group were given print-outs of the NPO’s data taken from the Charity Commission; this included both print-outs of the Charity Commission web pages as well as the annual reports submitted by the NPO to the commission. The group was then asked to discuss the data both in reference to their own experiences and Janet’s – highlighting and discussing what they felt was useful, or lacking in the data.
After some time with the data, participants were introduced to Accountable, and invited to explore and critique the data in a similar way. They were again asked to focus both on what they themselves and the Janet persona would have thought of the data and take the same considerations as with the printed Charity Commission data. Comparisons between the two presentations of the data were encouraged.
Finally participants were then given the opportunity to create their dream “profile page” for an NPO. Once finished, the group was asked to discuss the designs further, focusing on why they included certain details and to compare designs.
Data Collection and Analysis
Interviews and workshops were audio recorded and transcribed. The lead researcher coded the transcripts using an inductive thematic analysis @braun_using_2006, chosen for its epistemological agnosticism. During coding and clustering, differences in analytic interpretation were collectively discussed by the authors and resolved before codes were compiled into themes. As the interviews and workshops each concerned NPO finances, from different perspectives, the two datasets were treated as one thematically coherent data corpus.
The separation of interview and workshop participants emerged from both practical considerations to participant schedules, as well as an assumption that participants running NPOs may hold different views to other participants; which could have led to a tension in workshops that was counter-productive to discourse. We actually discovered that all participants, regardless of status or commitment to NPOS, held similar views about their work and that there was a large degree of coherence between the two groups. Our analysis of the data resulted in five themes, discussed below.
Making Volunteer Effort Visible
Many of our workshop participants volunteered their time as opposed to contributing financially to NPOs, with F01 explaining how they “help with organising stuff like […] phoning people and making sure they’re coming”, that she “[answered] calls”, “ran a workshop” and did “odd stuff”. This was echoed by F02, who also explained how she donated skills: “I made them a website and set up a Facebook page…” (F02). Another participant said he wasn’t “into the whole money thing” (F05), preferring to donate his efforts and skills rather than money. Perhaps unsurprisingly, volunteer effort was also seen to have an incredibly large impact on the running of NPOs. A lot of what might be defined as frontline services are delivered solely through volunteer effort; I04 explained how their organisation exists to support “parent and toddler groups across the city, and they are volunteer led”. This participant went on to explain how there are “111 parent and toddler groups” (I04), which are all “giving their time for nothing” (I04). Another participant explained how their role primarily exists to provide infrastructure support to “volunteer led” (I05) activities. Volunteer effort is also seen to be important from a funding standpoint as “you are getting asked more and more things like, ‘How many volunteers do you have?’” (IO5). In NPOs, the volume of volunteers can be quantified into “full time equivalent employees”, which are included in funding bids; “two half-time workers, for instance, that would obviously be one full-time” (I05). Of importance here is the hours that are being given to the organisation in order to assist the delivery of its services.
While volunteer effort and contributions were clearly seen to be important from the perspectives of both stakeholder groups, there was a feeling that these are often poorly represented in existing public reporting processes. The financial data on the Charity Commission website included details about the number of employees and volunteers that an NPO has. It was explained how the number of volunteers represented here was an important indicator of the organisation’s health. Looking through the example data taken from the Charity Commission website and used in Accountable, one participant went on to explain how it is useful to contrast the numbers of volunteers with the number of employees an organisation has: “You employ 190 people. That’s a lot of people. You’re talking, basically, about a medium- size enterprise level, and [in this case] you’ve managed to engage less volunteers than you have employees. That doesn’t look good.” (I06). As such, it was considered that at a simple level the open financial date provided enough information to begin interrogating the work of an organisation.
While offering a useful starting point, it was clear that the presentation of data in relation to the equivalent numbers of full-time staff (be these paid or volunteers) was too simplistic. I06 went on to explain further that the data presented here should not be taken at face-value and was lacking a necessary degree of detail: “it doesn’t tell you how much they donate in terms of time.” (I06). Furthermore, it said little about whether the same people were donating time, or whether a large pool of people donated small pieces of time, or the type of work and skills they were giving which may influence the economic and social value of performed work. This was echoed by I03, who felt it would be more meaningful to see information that articulated: “so many people volunteered, which equates to X number of hours, which put in X amount of time, so money-wise, monetary value would be… X” (I03).
Accounting for Projects
When asked about how they organized the funding of activities in the organisation, the NPO representatives explained how their funding is often tightly tied to specific projects: “money is given per project, so we only spend money on projects that we’ve specifically fundraised the money for” (I03). One participant explained that the funding they receive is “all in chunks” (I04). An organisation will also use examples of past projects when applying for funding as means for building a case for further funding on a “project by project basis” (I07). These types of funds were thus described as “restricted” by participants, in that they “have to spend [the funds] on that particular piece of work” (I08). Therefore, “often the health of a charity financially is about how much they’ve got in unrestricted funds” (I08), which can be used to start new activities or projects, or to provide match-funding for new bids.
When discussing these issues, participants started to explain how there was no reflection of the compartmentalisation of funding in the financial data presented by the Charity Commission. This is despite it being seen as a useful means for indicating the activities of an organisation, as well as communicating the ways in which its activities might be restricted based on the funding it has received: “This gives a whole picture, I would assume, of the whole organisation, and maybe [example charity], like us, have different project areas” (I04); “this is the other thing; it doesn’t tell you what proportion of that funding is actually restricted and how” (I06). It was noted by several of the NPO representatives that this lack of detail was all the more confusing given that typically organisations have to divide their accounts into individual projects for their own accounting processes and for tax and funder auditing. However, it was felt that “when it all comes together on this bit of paper as your end-of-year report, I can’t see those figures in that report” (I04). The headings and definitions provided by the Charity Commission also did not explain the project – context of what an organisation was doing -– “Then the spending – again the vast bulk of it on charitable activities. That’s not very meaningful is it?” (I08). As such, the annual reports produced by individual organisations made available on their own websites (and occasionally also uploaded to the Charity Commission website as additional information) were instead seen as the go-to information as they highlight “the major projects of the past year”.
Both interview and workshop participants stated a desire to see the activities and projects of an NPO presented alongside the financial data to support its interrogation. Interviewees stated that having a view of the data that “gives you a percentage of your income and how it’s spent on the project” would be more meaningful, and that “you’d probably want to break it down to project activities, grants, and some sort of, also, qualitative accounting” (I06). I07 explained that a project-by-project breakdown “[is] one of the things that will basically get the attention of maybe a potential funder. Listing the project and then your achievements, I think that would be the dream type of one-pager.” (I07). This was echoed by workshop participants who had volunteered for NPOs, who explained that they would hope to see “some key events that they have organised in the past few years and how much money they’ve spent on it” (F03), and “a section about the activities, because that’s what I don’t think was in there so much” (F04). These participants also highlighted how their interest in the future work of an organisation was also promoted through the types of projects that it ran and had funded: “A section saying what they plan to do in the future and maybe even backed up with a rough idea of how much money is going to be spent and where they’re going to get it from” (F01). There is a palpable sense of projects and concrete activities being “what most people would be interested in” (F04) when looking at an organisation, both in terms of demonstrating and interrogating what work that organisation had done in a given period, and what it hoped to achieve in the near future.
Administration and Charitable Spending
A clear point of tension in our data related to the different perspectives participants had on what are and are not acceptable activities and roles to fund in an NPO. Some of the workshop participants viewed NPOs spending money on administration or governance as a negative, expressing instead a desire to see funds go directly to charitable activities: “I would want to know. Now I don’t really want to donate to [a specific animal charity] because I feel like the money is not going towards the [animals], it’s going towards the people that run it.” (F02). When asked about how much they would expect an organisation to spend on its charitable activities, this participant went on to state: “I would expect a charity to spend almost all of what it gets [… and not] stockpiling, it suggests they don’t actually need it. Maybe they’d like to put some away for some big projects. I’d expect them to spend maybe 90% of the income..” (F02).
The NPO representatives however were very conscious of the public perception regarding administrative costs. I05 explained that “nobody wants to pay for a chief exec’s post, for instance, but it’s a vital post in the organisation”. Contrasting the workshops, interviewees viewed administrative costs as part of the work itself and would often incorporate this into funding bids and project proposals: “We have, on most of the projects, a management fee, which supports the work of the management committee” (I04). I05 went on to explain how “that’s where a lot of organisations will have a set figure for each piece of work that they will take off in terms of overheads, somewhere around 12% usually, to cover those costs” (I05). As such, from the perspective of an NPO, administration or governance was seen as as an essential component that allows for effective delivery of front-line services: “we can’t function unless we have an administrator or finance officer” (I05). However it was felt there was a growing difficulty in accessing funds for these types of roles and duties:
The lack of funding for paid staff time was often led to quandaries in terms of access to other types of funding. I05 went on to explain how: “There is quite a lot of money out there for capital spend, so if you wanted to buy some equipment to take young people to do an activity […] But there is no money to pay the staff to get them there” (I05). As such, organisations become more reliant on volunteer commitments – however, paid staff time is often required to write funding bids in the first place, or to manage critical aspects of them when funded.
An NPO typically exists because of the needs of its beneficiaries. Without them there would be no reason for the organisation to operate, and their experience and changed circumstances are often considered direct evidence that the organisation is performing its duties. In the open data hosted on the Charity Commission, the ways in which beneficiaries are engaged with and how they benefit is not detailed; who the beneficiaries of an organisation’s activities might be is not mentioned in an organisation’s overview page. Annual reports produced by NPOs for the Charity Commission do provide brief insights into beneficiaries in reference to the Charity’s aims. However, generally speaking there is little to no detail given as to how the financial data reported for the purposes of public accountability relates to the activities that these organisations do for the communities, individuals or issues it seeks to support.
Several of the NPO representatives highlighted that the lack of evidence around who benefits from the work of their organisations was at odds with their own internal reporting and auditing processes. One participant explained how they carefully collected information about all of their volunteers activities: “we do that indirectly through monitoring, […] we’ve got a quarterly monitoring system where we input our stuff into their database, and they pre-populate these things to say ‘You’ve seen this many people this many times’…” (I03). Through internal auditing and reporting mechanisms like this, our participants explained how they had a good understanding of the numbers of beneficiaries that the organisation had come into contact with and, to varying degrees of detail, the quality and extent of this contact. Quite often these were embedded within the (private) reporting processes to the funders of specific projects or activities. That this offers the potential to link project financial data with the “work done” was not lost on these participants: “… the information I would like to put across would be first of all the project, like what was funded, then what we did […] this is the impact that we had’.” (I07). I05 explained how their organisation had a good understanding of how many people gained or were ‘impacted’ upon by a specific piece of work, and that this could demonstrate a link between funds received and quantity, if not quality, of outputs: “Because then […] you can really look at how much work is costing per person…” (I05).
Workshop participants mirrored these sentiments, noting that they would like to see: “something like what kind of changes have been made to people.” (F03), “[It] only works if it’s a charity that’s supposed to help humans, but the number of people that are being impacted by their work..” (F02). Explicit throughout these again was making “what impacts they’ve had”‘ (F02) more visible and “the things they’re most proud of” (F02). Participants saw description of work as a way of judging whether organisations were performing in accordance to their remit, “I kind of like this sort of having objective and achievement next to each other” (F05).
Revealing Funding Opportunities
NPOs seek out funding opportunities from a variety of sources, including LAs. During interviews, NPOs were asked about their interactions with LAs and discussed the LA data on the Accountable system. It was noted by some participants that this data provided a potentially useful resource for understanding how LAs went about allocating their funds and how much of those funds were being allocated to organisations in the charity or non-profit sector. Our participant representing an organisation that provides infrastructure support to other NGOs explained how LAs have a remit to ensure that a certain amount of funds are distributed to NPOs based in the local region. Therefore, the public availability of data like this is useful to “churn through” and attempt to make sense of: “we can then get back to the council [LA] and highlight how they are not delivering on this requirement” (I05). However, they admitted that the data as archived and made available is difficult to parse, and that it was often impossible to tell whether the money was being spent on activities or services that could by provided by NPOs, leading to a large amount of “reading between the lines” of the data.
Other participants from NPOs also critiqued the data presented. This in part echoed the points of I05; several of our participants found the data and information contained in these extensive spreadsheets “impossible” to make sense of. When reviewing the visualisations of this data on Accountable the participants explained how this made the data easier to explore and compare across different LA service areas and departments but did not appear to be immediately of relevance to their running of an NPO. Rather than seeing LA data on expenditure, they were more interested in understanding what funding opportunities may be present in an LA’s budget: “I’m not sure how useful it is for me to know the council break-up, because I need to know what money is available to use, what pots […] it’s more detail I need” (I03). Another participant qualified this around searching for money for their projects: “that’s what I would like to know really, if they still have money for a project where we can bid or an application that we can submit or something like that” (I07). It was clear that the perceived impoverishment of the LA data as provided in its archived and visualised forms was tied to a wider set of concerns NPOs have around awareness of what money is still available, what budgets have yet to be spent, and how potential funding streams and individual funders themselves might be better aggregated and brought together. This was explained in depth by I07:
Our findings highlight how, regardless of presentation, the ‘open’ financial data provided by the Charity Commission is neither an adequate vehicle for demonstrating the work of an NPO, nor a detailed resource to be interrogated by those either invested, or interested in, its activities. Interviews with representatives of NPOs further highlighted that the funding mechanisms and structure of work performed by organisations are not necessarily reflected in the financial data submitted to the Charity Commission, and thus a large amount of context and meaning is lost in its presentation. This was reinforced by our volunteer participants, in that they deemed the financial information presented by the Charity Commission was not useful for what they were interested in seeing about an organisation’s spending – namely its activities, projects and work. Currently, the aggregated financial data highlights only ‘bottom line’ figures, such as totals spent or earned by an organisation on high-level categories such as Charitable Activities and Trading to Raise Funds; a figure which is known to have little relevance to NPOs @henderson_performance_2002. Returning to the discussion of literature that opened this paper, this is exemplar of Heald’s input transparency as sums of money are presented without the contextual information essential to explaining or justifying the work of the NPO @heald_varieties_2006 [@heald_fiscal_2003]. Our findings note how the work conducted by an NPO or charity is inextricably bound to funding that gives projects structure, and at the same time restricts what funds can be spent on. Organisations and their investors desire to see projects and activities reflected in the data to contextualise it and give it meaning.
Furthermore, the lack of contextual data on the Charity Commission indicates that it understands transparency as Schauer’s Passive Transparency @schauer_transparency_2011, in that what is presented is high-level expenditure data without comment to its interpretation or the responsibilities of the organisation. This is true even when the annual reports of NPOs are taken into consideration, since accounts and spending are not directly tied to activities, projects, or aims and objectives. Beneficiaries of NPOs are only talked about in high-level terms, and the substantial efforts put forward by volunteers are similarly lost as details are not carried from the organisation to the Charity Commission. There also exists a tension between the perceived and actual value of administrative costs of an organization, which are often also directly linked to an organisation’s ability to deliver services. Organisations are finding it increasingly difficult to raise funds for essential administrative roles or purposes, yet are expected to be showing that they have invested the majority of money into providing for its beneficiaries. That administrative work is a key part of service provision is not clear in the data presented to those investing in an organisation; leading to issues of trust as the data does not reflect commitment to its purported aims and objectives @macmillan_relationship_2005. NPOs also discover issues when seeking funding from LAs, as it is unclear what funding remains in the LA budgets that could benefit their work.
The results of our study raise a number of questions around the role open data technologies in NPOs and other trusted organisations. In the following section we discuss a number of the implications this raises for the design of future systems that represent and provide access to data, and for the procedures of capturing and organising such data.
Design for Visibility, not Transparency
Confusion about what constitutes organisational transparency indicates that producing systems to support it may be limited in scope, as the term is open to interpretation @schauer_transparency_2011 [@oliver_what_2004]. Rather, design work around the opening up of information in the non-profit sector might be better focused on making visible the projects and work of the organisation. Funding, as discussed, is most often tied directly to a single piece of work or project. Visibility of this work, however, is lost when the focus is on aggregated expenditure. If the annual reports provided to the Charity Commission were to outline investments based on specific projects, then a direct link can be seen between expenditure and the work of the charity. Work thus becomes visible, and data becomes concerned with the responsibilities of the organisation as well as its audience; fitting with notions of active transparency and answerability in the process @schauer_transparency_2011 [@oliver_what_2004; @fox_uncertain_2007].
Systems seeking to support the communication of an organisation’s work through data should, therefore, be designed for visibility instead of transparency to avoid misconceptions about what it is they’re trying to do. Regardless of whether a system is designed to interrogate or demonstrate NPO behaviour – this research underlines how little of an organisation can be seen when presented with input transparencies , how this fails to make them interrogatable and, thus, unaccountable @heald_varieties_2006 [@heald_fiscal_2003; @fox_uncertain_2007]. Outcomes would likely be incredibly difficult to measure through financial data – or in some instances impossible to measure in any known tangible way @heald_fiscal_2003 [@henderson_performance_2002; @hook_making_2015] – yet organisations had a deep understanding of their beneficiaries, which could be more indicative of their activities. The efforts of volunteers also have a profound effect on organisations; participants indicated that many activities were entirely volunteer- led, and that the number of volunteers can be a measure of an NPO’s health. Therefore, we suggest that making visible the effort of volunteers, and the number and type of beneficiaries will go some way to accounting for what a non-profit organisation is doing. We might imagine therefore that systems like Accountable might provide enhanced comparisons between the number of employees against volunteers, and the volume of beneficiaries who may have come into contact with volunteers over the course of a project or set period of time. While still far from perfect, this might support organisational accountability in reference to their stated aims and objectives, as well as provide data that reinforces the value of their services.
Our findings also highlighted tensions between NPO expenditure on administrative functions versus the direct spend on beneficiaries and charitable activities @oliver_what_2004 [@henderson_performance_2002]. While members of the public and those donating money to an NPO might expect funds to be directed straight to charitable activities, it is clear that that administration and governance costs are a critically important feature of effective service delivery and the completion of projects. As it stands, open data around the activities of NPOs does not communicate well these administration costs. An initial reaction to this, given our notion of visibility, would be to highlight how much of project or activity spend is tied to administrative work and roles; however, doing so may in itself reinforce existing feelings of mistrust between the public and NPOs, especially given recent stories in the popular press that propagate (potentially misleading) views on the misuse of public and donated funds by some organisations @hope_true_2015 [@wilding_it_2015]. Instead, we suggest that it isn’t the cost of administration and management that needs to be represented, but rather the cost of it not existing; in other words, in what ways does having various paid staff roles provide a critical infrastructure and enable the day-to-day volunteering and charitable activities that NPOs are commonly expected to perform? Furthermore, it is important to use this data to contextualize what is and what is not reasonable staff costs, and how this may scale depending on the size and scope of the organisation. Providing this, however, requires going beyond the primarily numerical data associated with the types of open financial data provided by LAs and the Charity Commission. We address this issue in the following section.
Produce Qualitative Accounting
The focus on presenting ‘bottom line’ spending data is both an input transparency and data that is known to be ineffective of determining NPO performance @heald_varieties_2006 [@heald_fiscal_2003; @henderson_performance_2002]. Even if we were to take projects into consideration, financial spend is inadequate to determine whether an organisation is meeting its goals as it fails to reflect whether a spend is appropriate or not @heald_fiscal_2003 [@henderson_performance_2002]. Our findings reflect these issues, as organisations noted the efforts they were taking to continue to effectively deliver services while struggling with a lack of administration staff. To more appropriately indicate the reasons and activities behind NPO spending data and to support designing for visibility, we suggest moving to include additional qualitative information in financial reporting.
Incorporating the capture of qualitative information in accounting practices would go some way to making visible the efforts of the organisation and provide context to expenditure. Our study’s findings noted that organisations keep track of information such as volunteer activity, and the number and location of services that they provide to beneficiaries; however, such information is typically captured relatively informally and often in an unsystematic manner. Furthermore, the intertwining of funding and projects also indicates that a contextual reference for spending can be created by entering this data alongside financial transactions. Metadata about spending, such as project or activity, approximate location, number of beneficiaries, and volunteer effort, would provide a better view of the work NPOs were doing with specific types of funds. Visibility would be enhanced through the presence of the qualitative data, which would also provide greater insight about the output of the organisation, increasing accountability @heald_fiscal_2003 [@fox_uncertain_2007]. Outputs (e.g., the number of people using NPO services) are not necessarily immediately or directly linked to eventual outcomes (e.g., how did people gain from participating in these sessions), but they do give a better starting point than input data for those looking to understand the potential impact of an organisation’s work @heald_fiscal_2003 [@henderson_performance_2002]. NPOs can use this qualitative financial data to reinforce claims about the extent and quality of their work @heald_fiscal_2003 [@macmillan_relationship_2005]. Alignment of activities and goals was something that participants expressed keen interest in, and as such using data to present this would benefit organisations attempting to demonstrate their work and commitment to their aims and objectives @macmillan_relationship_2005.
Support Interrogation through Common Formats
Currently, the systems that are used to hold NPOs and LAs accountable for their financial expenditure do not support interrogation or scrutiny since they cannot be effectively retrieved and inspected. An issue that became immediately apparent in the design and development of Accountable was the need for common data formats and standards. While this may appear to be a moot point, it’s a critical one as we observed and indeed experienced first hand how despite the popular rhetoric of public accountability and transparency, existing ways of archiving and presenting data fell short. For example, considering the Charity Commission data, use of Flash technology to generate web pages @adobe_adobe_2015, and irregularly formatted PDF format reports, meant the underlying data could not be easily retrieved. Furthermore, while there appears to be some benefit to providing tools for NPOs and other stakeholders to interrogate LA financial data, structural differences between LAs and inconsistent approaches to categorizing data made it unclear as to what money was being spent on @newcastle_city_council_payments_2015 [@northumberland_county_council_payments_2015].
One recommendation that comes from this is to ensure that systems designed to support public accountability move away from technologies and practices that make the underlying data inaccessible for scrutiny. In our specific context, it is likely that at the time that the Charity Commission site was developed technologies like Flash were the only viable option to effectively chart information. Modern web technologies can, however, achieve the same effects whilst allowing the data to be mechanically retrieved. Since these charts are likely populated from information stored elsewhere on the system, it is feasible to implement alternative views in standard formats or feeds – such as that provided by the UK Government’s Parliament API @houses_of_parliament_developing_2015. At the same time, while the LA data sets are already provided in a common file format, they would benefit from the imposition of data standards to ensure that field names and date formats are common across different geographical areas and jurisdictions. This highlights not just the need for new ways to present this data, but also the production of tools that allow those who manage or oversee such open data to think through how they collate and structure data for audiences in ways that allow it to be inspected and compared with other related organisations.
In this paper we have set out to explore issues to do with the role of openly available data in supporting the public accountability of trusted organisations. We did this through the specific example of the financial practices of non-profits and local governmental authorities, and the ways in which they publish aspects of their spending online in the aid of supporting transparency around the spending of public and donated funds. While data describing the finances of trusted organisations is seen as a means for making these organisations more accountable to the public, our study highlighted how such data makes information about the (often voluntary) efforts of the organisation and its beneficiaries invisible. From this, we have discussed how technologies intended to support the accountability of trusted organisations should strive for visibility of activity rather than prevailing notions of transparency through data. We have highlighted the implications for the presentation, capture and organization of data.
Outside of finance, our findings show that there is a need for adequately representing human processes in Linked or Open Data in ways that are legible to people. Financial data is produced and exposed by organisations in order to make them transparent and accountable, but as seen with our specific examples of UK Charity Commission data this requires further annotation to make visible the activities of the organisation. In the specific context of designing for the non-profit sector, this means considering visibility instead of transparency for data technologies to be framed around an organisation’s activities instead of how much income and expenditure they have generated and spent. In our case, the funding, spending, and activities of NPOs were all found to orbit around a project structure; with funds granted and restricted to specific projects and activities, including administration costs. Capturing metadata about project funding, volunteer activities, and beneficiaries in financial records leads to a more comprehensive vantage point from which to hold an organisation accountable. Achieving this will require future consideration to data formats and standards that support this, giving HCI research an opportunity to help shape future international discussions on transparency and accountability.
We’d like to thank our participants for giving their time to take part in our study. This research was funded through the EPSRC Centre for Doctoral Training in Digital Civics (EP/L016176/1). Data supporting this publication is openly available under an ‘Open Data Commons Open Database License’. Additional metadata are available at: 10.17634/154300-3. Please contact Newcastle Research Data Service at firstname.lastname@example.org for access instructions.