Category Archives: impact

Impact?

By René Bekkers & Pamala Wiepking

In decisions on academic careers, the societal impact that researchers have with their research is gaining importance. This is an addition to incentives for academic impact. Relevant indicators for academic impact are how often the researcher has been cited (the so-called H-index) and the impact factor (IF) of the journals in which the researcher has published.

 

What the Journal Impact Factor is not

It is widely believed that it is more difficult to get published in journals with a higher IF because they are more attractive and can afford to desk reject a larger proportion of the submissions. However, journals with higher IFs do not necessarily publish research of higher quality. Because scientists in some countries can quickly advance their careers and even receive monetary bonuses by publishing in high IF journals, they also attract a lot of low-quality research. Once in a while, a piece of garbage slips through because the peer review process is not perfect.

Also publishing an article in a high IF journal does not necessarily imply that the article will be more widely read or receive a higher number of citations. The overwhelming majority of articles in high IF journals receive low numbers of citations, but a few articles that are highly cited determine the IF.

 

Article impact: citations

The highest impact paper we have published is a literature review of empirical studies on philanthropy. It was originally written as a background paper for a request for proposals of the John Templeton Foundation (JTF), and later published in three separate, standalone articles: one in Nonprofit & Voluntary Sector Quarterly (NVSQ) and two in Voluntary Sector Review (VSR). As a self-archived working paper, the JTF paper already attracted some attention, but the NVSQ article quickly received large numbers of citations, and continues to do so. The journals in which we published the papers were not the highest IF journals in which we published throughout our careers. In fact, papers we published in higher IF journals have attracted way fewer citations.

literature_review_citations

From what we have heard from readers the paper is useful to many people because it provides a map of the landscape of research on philanthropy. Our review pointed them to studies that are relevant to their specific research questions. Studies that they would otherwise not have found. Often this impact is invisible because researchers do not cite our review, but instead cite the papers we reviewed. So the impact we have with the paper is visible not so much in its number of citations, but in the number of readers. There is no way to track whether people actually read it, but paper has been downloaded thousands of times.

 

Did we achieve the objectives of the paper?

The number of downloads is a quantifiable, generic indicator of impact. But a high number of downloads is not why we wrote this paper. If you see research as an intervention, its impact should be evaluated relative to intention: did it achieve the intended goals?

We wrote the paper to provide “a reference resource for classical intuitions” (p.925) for researchers who have an idea for a study, but do not know what previous research has found. We hoped that our review would reduce “the lack of awareness of research in distant times and disciplines” (p.945). We wanted to acquaint researchers in different fields with each other’s work. Two goals that we did not explicitly state in the paper because they seemed overly ambitious were to integrate these relatively isolated bodies of research and ultimately to establish a common knowledge base for research on philanthropy.

It is very difficult to determine to what extent we have been successful with this paper. Perhaps in a decade or two it may become clear how useful our contribution has been in establishing a common knowledge base for research on philanthropy. Also it would be difficult to quantify whether our review actually integrated different fields. One could count the number of citations to research in other disciplines before and after our review was published, and in studies that cite our review and those that do not. According to our own standards, however, those numbers would still not prove impact.

Yet we do believe we have made a difference. Many academics who were new to the topic of philanthropy have told us that our review was helpful in finding their way in the literature. We also know that our readers are more aware of research in other disciplines than the citations demonstrate. Word limits for journal articles lead researchers to omit relevant references, and they focus on work published in the same journal and discipline.

 

Did we change the practice of fundraising?

What about the societal impact? As we wrote in our paper (p. 926), we hoped that our review would not only be useful for an academic audience but also for practitioners. What impact did we have on the practice of fundraising? We could count the number of talks and seminars we were invited to give and actually gave and the number of attendees at these events. But this is not measuring impact, and not why we wrote the paper. We hoped that fundraisers would take advantage of the insights gained in the studies we reviewed to increase fundraising effectiveness (p. 926). We learned from conversations with fundraisers and other philanthropy professionals who read our paper or attended our talks that insights from our literature review indeed have influenced the way they look at donors and fundraising. The insights from academia may have helped them to better understand why their donors choose to donate to their organization, and how they can use this information to build better and stronger relationships with those donors. We feel happy to have contributed to this, but also believe the effectiveness of fundraising, especially in relation to donor satisfaction, can always be improved further. We look forward to keep providing academic insights to support fundraisers and other philanthropy professionals in this challenge!

Leave a comment

Filed under impact, philanthropy

Resilience and Philanthropy

This post in pdf

With the year 2020 on the horizon, the recently published work programme for Research & Innovation from European Commission for the years 2016-2017 is organized around a limited set of Societal Challenges. Europe defined these challenges after a long process of lobbying and consultation with many stakeholders. Going through the list I could not help thinking that something was missing. I do not mean that the list of challenges is a result of a political process and does not seem to reflect an underlying vision of Europe. I am thinking about the current refugee crisis. The stream of refugees arriving at the gates of Europe poses new challenges to Europe, in many areas: humanitarian assistance, citizenship, poverty, inclusion, access to education, and jobs. The stream of refugees also raises important questions for philanthropy. How will Europe deal with these challenges? How resilient is Europe? Will governments, nonprofit organizations and citizens be able to deal with this challenge? In the definition of the Rockefeller Foundation, resilience is the capacity of individuals, communities and systems to survive, adapt, and grow in the face of stress and shocks, and even transform when conditions require it. I define resilience as the mobilization of resources for the improvement of welfare in the face of adversity.

Among refugees, who are seeking a better future for themselves and their children, we see resilience. Threatened by adversity in their home countries, they take grave risks by placing their fate in the hands of human traffickers, foreign police officers. They rely on each other and their inner strength, hoping that what they left behind is worse than their future. We see a lack of resilience in Europe. The continent was not ready for the large stream of refugees. Some member states pass on the stream to each other by closing their borders. Other national governments try to accommodate refugees seeking asylum, but face barriers in finding housing, and resistance from groups of citizens who oppose accommodation of refugees in their communities. At the same time we see a willingness to help among other citizens, who offer assistance in the form of volunteer time, food and other goods. Perhaps the response of citizens is related to their own levels of resilience.

Resilience is not just the ability to withstand adversity or change by not changing at all. Resilience is not just sitting it out, or a strategy based on a rational computation of risks, the avoidance of risks, or flexibility and absorption of shocks. The resilient actor adapts to new situations and grows.  Neither is resilience an immutable trait of individuals, a matter of luck in the genetic lottery. Resilience has often been studied at the individual level in psychology. Resilience requires will power, perseverance, self-esteem, creativity, a proactive attitude, optimism, intrinsic motivation, inner strength, a long term orientation to the future, willingness to change for the better, risk-taking, using the force of your opponent, problem solving ability, and intelligence.

The questions for research on resilience require social scientists to study not only the response of individual citizens, but also of social systems: informal networks of citizens, social groups, nonprofit organizations, nations, and supra-national institutions. How are resilience-related traits related to philanthropy at the level of groups and systems? How can resilience among organizations be fostered? How do nonprofit organizations build and on resilience of target groups? Resilience is a very useful concept to apply to each of the societal challenges of Europe. The classic welfare state was a system that created resilience for society as a whole, reducing the need for resilience among individual citizens. The modern activating welfare state requires resilience among citizens as a condition for support. Welfare state support becomes more like charity: we favor victims of natural disasters that try to make the best of their lives and welfare recipients that are actively seeking a job.

As nonprofit organizations are trying to respond to the refugee crisis, they are also facing adversity themselves. In the United Kingdom, fundraising practices by charities have recently come under attack. In the Dutch nonprofit sector, cuts in government funding to arts and culture organizations have been a major source of adversity in the past years. Further cuts have been announced to organizations in international relief and development. In our research at the Center for Philanthropic Studies at VU Amsterdam we have asked: how willing are Dutch citizens to increase private contributions to charities when the government is lowering their financial support? Not much, is what our research shows. While some may have believed that citizens would compensate lower income from government grants through increased donations, this has not happened. When the cuts to the arts and culture organizations were announced, the minister for Education, Arts and Science said that cultural organizations should do more to raise funds from private sources and should rely less on government grants. The culture change in the cultural sector is taking place, slowly. Some organizations were not ready for this change and simply discontinued their activities. Most have decided to do with less, and see what opportunities they may have to increase fundraising income. Some have done well. On the whole, the increase in private contributions is marginal, and much less than the loss in government grants.

For nonprofit organizations, the refugee crisis poses a challenge, but also an opportunity to mobilize citizen support in an effective manner. By offering their support to the government, working together effectively, and channeling the willingness to volunteer they can demonstrate the societal impact that nonprofit organizations may have. This would be a much needed demonstration when trust in charitable organizations is low.

Leave a comment

Filed under disaster relief, empathy, Europe, foundations, helping, impact, Netherlands, philanthropy, psychology, trust

Wat zegt het CBF-Keur voor goede doelen?

Het Financieel Dagblad besteedt een lang artikel aan de betekenis van het CBF-Keur voor goede doelen naar aanleiding van de vraag: “Waar blijft mijn gedoneerde euro?” Het “keurmerk en boekhoudregels zijn geen garantie voor een zinvolle besteding”, volgens de krant. Verderop in het artikel staat mijn naam genoemd bij de stelling dat het CBF-Keur ‘fraude of veel te hoge kosten niet uitsluit’ en zelfs dat het ‘nietszeggend’ zou zijn. Inderdaad zegt het feit dat een goed doel over het CBF-Keur beschikt niet dat de organisatie perfect werkt. Het maakt fraude niet onmogelijk en dwingt organisaties ook niet altijd tot de meest efficiënte besteding van beschikbare middelen. In het verleden zijn misstanden bij verschillende CBF-Keurmerkhouders in het nieuws gekomen, die bij sommige organisaties hebben geleid tot intrekking van het keurmerk.

Maar helemaal ‘nietszeggend’ is het CBF-Keur ook weer niet. Zo denk ik er ook niet over. Het CBF-Keur zegt wel degelijk wat. Voordat een organisatie het CBF-Keur mag voeren moet het een uitgebreide procedure door om aan eisen te voldoen aan financiële verslaggeving, onafhankelijkheid van het bestuur, kosten van fondsenwerving, en de formulering van beleidsplannen. Dit zijn relevante criteria. Zij zorgen ervoor dat je als donateur erop kunt vertrouwen dat de organisatie op een professionele manier werkt. Het CBF-Keur zegt alleen niet zoveel over de efficiëntie van de bestedingen van een goed doel. Veel mensen denken dat wel, zo constateerden we in onderzoek uit 2009.

Het is lastige materie. Garantie krijg je op een product dat je koopt in de winkel, waardoor je het terug kunt brengen als het niet functioneert of binnen korte tijd stuk gaat. Zulke garanties zijn moeilijk te geven voor giften aan goede doelen. Een dergelijke garantie zou je alleen kunnen geven als de kwaliteit van het werk van goede doelenorganisaties gecontroleerd kan worden en er een minimumeis voor te formuleren valt. Dat lijkt mij onmogelijk. Het CBF-Keur is niet zoiets als een rijbewijs dat je moet hebben voordat je een auto mag besturen. De markt voor goede doelen is vrij toegankelijk; iedereen mag de weg op. Sommige goede doelen hebben een keurmerk, maar dat zegt vooral hoeveel ze betaald hebben voor de benzine, in wat voor auto ze rijden en wie er achter het stuur zit. Het zegt nog niet zoveel over de hoeveelheid ongelukken die ze ooit hebben mee gemaakt of veroorzaakt, en of dat de kortste of de snelste weg is.

Vorig jaar stelde de commissie-De Jong voor om een autoriteit filantropie in te stellen, die organisaties zou gaan controleren voordat ze de markt voor goede doelen op mogen. Er zou een goede doelen politie komen die ook op de naleving van de regels mag controleren en boetes mag uitdelen. Dat voorstel was te duur voor de overheid. Voor de goede doelen was het onaantrekkelijk omdat zij aan nieuwe regels zouden moeten gaan voldoen. Bovendien was het niet duidelijk of die nieuwe regels ook echt het aantal ongelukken zou verlagen. Het is op dit moment überhaupt niet duidelijk hoe goed de bestuurders van goede doelen de weg kennen en hoeveel ongelukken ze maken. Een beter systeem zou moeten beginnen met een meting van het aantal overtredingen in het goede doelen verkeer en een telling van het aantal bestuurders met en zonder rijbewijs. Vervolgens zou het goed zijn om een rijopleiding op te zetten die iedereen die de markt op wil kan volgen en in staat stelt de vaardigheden op te doen waarover elke bestuurder moet beschikken. Ik hoop dat het artikel in het Financieel Dagblad tot een discussie leidt die dit duidelijk maakt.

Intussen heeft het CBF gereageerd met de verzekering dat er gewerkt wordt aan uitwerking van richtlijnen voor ‘reactief toezicht op prestaties’. Ook de VFI, branchevereniging voor goede doelen, kwam met een reactie van die strekking. Dat is goed nieuws. Maar die nieuwe richtlijnen zijn er nog lang niet. In de tussentijd geeft het CBF keurmerken af en publiceren de Nederlandse media – die na Finland de meest vrije ter wereld zijn – af en toe een flitspaalfoto van wegmisbruikers. Dat lijkt voldoende te zijn om het goede doelen verkeer zichzelf te laten regelen en de ergste ongelukken te voorkomen. Want die zijn er maar weinig.

Leave a comment

Filed under charitable organizations, fraud, household giving, impact, incentives, law, philanthropy, policy evaluation, politics