Category Archives: data

A Conversation About Data Transparency

The integrity of the research process serves as the foundation for excellence in research. While transparency does not guarantee credibility, it guarantees you will get the credibility you deserve. Therefore we are developing criteria for transparency standards with regards to the reporting of methods and data.

We invite you to be part of this important conversation at the 48th ARNOVA Conference in San Diego, on Friday, November 22, in the 2:15 to 3:45 pm session in Executive Center Room 2B. This is session #166 in the conference program. In the session, we would like to hear from you which characteristics of data and methods transparency that help you review research and utilize past work as building blocks for future research. What are useful standards that improve the collective research process?

This session is very interactive. After a short introduction by the NVSQ editors, we split up in groups of researchers that work with the same type of data, such as data from ethnography, interviews, experiments, surveys, 990s. In each group we try to formulate criteria for transparency that allow readers to assess the quality of research. Finally, we share the results of the small group sessions with the larger group.

Please join us for this important conversation!

To help us organize the session, please tell us what kind of data you work with in your research:

https://vuass.eu.qualtrics.com/jfe/form/SV_3rAS8LY4fMavflr

We look forward to seeing you on Friday, November 22, at 2:15 – 3:45 pm, in Executive Center Room 2B.

Best wishes,

The Working Group on Data Transparency: René Bekkers, Lehn Benjamin, Jesse Lecy, Paloma Raggo and Kelly LeRoux

NVSQ Editors: Chao Guo, Susan Phillips, and Angela Bies

Leave a comment

Filed under data, experiments, methodology, open science, survey research

Gevonden: student-assistent Geven in Nederland 2020

De werkgroep Filantropische Studies van de Faculteit Sociale Wetenschappen aan de Vrije Universiteit Amsterdam is het expertisecentrum op het gebied van onderzoek naar filantropie in Nederland. De werkgroep houdt zich bezig met vragen zoals: Waarom geven mensen vrijwillig geld aan goede doelen? Waarom verrichten mensen vrijwilligerswerk? Hoeveel geld gaat er om in de filantropische sector? Voor het onderzoek Geven in Nederland heeft de werkgroep een student-assistent gevonden: Florian van Heijningen. Welkom!

Leave a comment

Filed under bequests, Center for Philanthropic Studies, charitable organizations, corporate social responsibility, data, foundations, household giving, Netherlands, philanthropy, statistical analysis, survey research

Research on giving in the Netherlands continues, funding secured

We are pleased to announce that the Center for Philanthropic Studies has been able to secure funding for continued research on giving in the Netherlands. The funding enables data collection for the Giving in the Netherlands Panel Survey among households, as well as data collection on corporations, foundations, charity lotteries, and bequests.

In the past 20 years, Giving in the Netherlands has been the prime source of data on trends in the size and composition of philanthropy in the Netherlands. Continuation of the research was uncertain for more than a year because the ministry of Justice and Security withdrew 50% of its funding, calling upon the philanthropic sector to co-fund the research. In an ongoing dialogue with the philanthropic sector, the VU-Center sought stronger alignment of the research with the need for research in practice. The Center has organized round table discussions and an advisory group of experts from the sector has been composed. The Center will use the insights from this dialogue in the research.

Meanwhile the fieldwork has started. Preliminary estimates of giving in the Netherlands will be discussed at a symposium for members of branch organizations in the philanthropic sector in the Fall of 2019. Full publication of the results is scheduled mid-April 2020, at the National Day of Philanthropy.

5 Comments

Filed under Center for Philanthropic Studies, charitable organizations, corporate social responsibility, data, foundations, fundraising, household giving, informal giving, Netherlands, survey research, trends

Grootste goededoelenorganisaties ontvangen minder uit giften

Het ‘sectoronderzoek’ van Goede Doelen Nederland onder de 24 grootste goede doelen organisaties is weer verschenen. https://www.goededoelennederland.nl/system/files/public/Sector/190726%20Overzicht%20cijfers%20grote%20goede%20doelen%20met%20meer%20dan%2020%20miljoen.pdf

In het persbericht is onder de optimistische kop “Maatschappelijke betrokkenheid bij goede doelen onveranderd groot” te lezen dat “dat de resultaten in 2018 nagenoeg hetzelfde zijn als in 2017”. Ik zie iets anders. Drie tekenen aan de wand:

  1. De vrijgevigheid in Nederland neemt af. In 2018 ontvingen de grootste goede doelen in Nederland €571,6 miljoen, uit giften van particulieren en bedrijven. In 2017 was dit nog €583,6 miljoen. Een afname van 2%. De afname van giften van particulieren was bijna 4%. Tegelijk was de inflatie in Nederland 1,7%. De ontvangen euro’s zijn dus ook nog minder waard geworden.
  2. Een toename in inkomsten uit overheidssubsidie (goed voor 35% van de inkomsten van deze goede doelen), nalatenschappen (goed voor 12% van de inkomsten) en de goede doelen loterijen (nu 10% van de inkomsten) hebben de daling van de vrijgevigheid van particulieren en bedrijven opgevangen, maar niet goed gemaakt.
  3. De grootste goede doelen in Nederland besteden exact nul euro (= €0) aan opleidingen.

Update, 26 september 2019: ook de Volkskrant heeft de jaarverslagen van de grootste goededoelenorganisaties geanalyseerd en komt tot vergelijkbare conclusies. Zie hier het hoofdartikel dat op de voorpagina stond en hier het achtergrondartikel. Navraag bij het CBF leert dat de nul euro besteed aan opleidingen geen betrekking heeft op opleidingen van medewerkers, maar opleidingen van doelgroepen; de grootste goededoelenorganisaties zijn dus niet actief op het terrein van onderwijs.

Leave a comment

Filed under bequests, charitable organizations, data, economics, fundraising, household giving, Netherlands, trends

Conditional Review Acceptance Policy (R2)

“Thank you for your invitation to review. Did the authors provide the data and the code they have used to produce the paper? If not, can you ask them to?”

This is my rerevised Conditional Review Acceptance Policy (CRAP – R2). Five years ago, I introduced a conditional review acceptance policy (CRAP). Later I revised the policy. In the current rerevision I have dropped the requirement that the paper should be published in open access. In the past five years, we have progressed enormously on the path to open publishing. Moreover, in the coming years Plan S is likely to deliver the promise that all academic research can be read online at no additional cost to the reader. So the remaining challenge is open access to data and code. Without access to the materials that form the basis for a paper, it is very hard to evaluate whether the paper is a good reflection of these materials.

So if you invite me to review a paper, please send me the data and code, preferably at a public repository.

Leave a comment

Filed under academic misconduct, data, open science

Global Giving: Open Grant Proposal

Here’s an unusual thing for you to read: I am posting a brief description of a grant proposal that I will submit for the ‘vici’-competition of the Netherlands Organization for Scientific Research 2019 later this year. You can download the “pre-proposal” here. It is called “Global Giving”. With the study I aim to describe and explain philanthropy in a large number of countries across the world. I invite you to review the “pre-proposal” and suggest improvements; please use the comments box below, or write to me directly.

You may have heard the story that university researchers these days spend a lot of their time writing grant proposals for funding competitions. Also you may have heard the story that chances of success in such competitions are getting smaller and smaller. These stories are all true. But the story you seldom hear is how such competitions actually work: they are a source of stress, frustration, burnouts and depression, and a complete waste of the precious time of the smartest people in the world. Recently, Gross and Bergstrom found that “the effort researchers waste in writing proposals may be comparable to the total scientific value of the research that the funding supports”.

Remember the last time you saw the announcement of prize winners in a research grant competition? I have not heard a single voice in the choir of the many near-winners speak up: “Hey, I did not get a grant!” It is almost as if everybody wins all the time. It is not common in academia to be open about failures to win. How many vitaes you have seen recently contain a list of failures? This is a grave distortion of reality. Less than one in ten applications is succesful. This means that for each winning proposal there are at least nine proposals that did not get funding. I want you to know how much time is wasted by this procedure. So here I will be sharing my experiences with the upcoming ‘vici’-competition.

single-shot-santa

First let me tell you about the funny name of the competition. The name ‘vici’ derives from roman emperor Caesar’s famous phrase in Latin: ‘veni, vidi, vici’, which he allegedly used to describe a swift victory. The translation is: “I came, I saw, I conquered”. The Netherlands Organization for Scientific Research (‘Nederlandse organisatie voor Wetenschappelijk Onderzoek’, NWO) thought it fitting to use these names as titles of their personal grant schemes. The so-called ‘talent schemes’ are very much about the personal qualities of the applicant. The scheme heralds heroes. The fascination with talent goes against the very nature of science, where the value of an idea, method or result is not measured by the personality of the author, but by its validity and reliability. That is why peer review is often double blind and evaluators do not know who wrote the research report or proposal.

plt132

Yet in the talent scheme, the personality of the applicant is very important. The fascination with talent creates Matthew effects, first described in 1968 by Robert K. Merton. The name ‘Matthew effect’ derives from the biblical phrase “For to him who has will more be given” (Mark 4:25). Simply stated: success breeds success. Recently, this effect has been documented in the talent scheme by Thijs Bol, Matthijs de Vaan and Arnout van de Rijt. When two applicants are equally good but one – by mere chance – receives a grant and the other does not, the ‘winner’ is ascribed with talent and the ‘loser’ is not. The ‘winner’ then gets a tremendously higher chance of receiving future grants.

As a member of committees for the ‘veni’ competition I have seen how this works in practice. Applicants received scores for the quality of their proposal from expert reviewers before we interviewed them. When we had minimal differences between the expert reviewer scores of candidates – differing only in the second decimal – personal characteristics of the researchers such as their self-confidence and manner of speaking during the interview often made the difference between ‘winners’ and ‘losers’. Ultimately, such minute differences add up to dramatically higher chances to be a full professor 10 years later, as the analysis in Figure 4 of the Bol, De Vaan & Van de Rijt paper shows.

matthew

My career is in this graph. In 2005, I won a ‘veni’-grant, the early career grant that the Figure above is about. The grant gave me a lot of freedom for research and I enjoyed it tremendously. I am pretty certain that the freedom that the grant gave me paved the way for the full professorship that I was recently awarded, thirteen years later. But back then, the size of the grant did not feel right. I felt sorry for those who did not make it. I knew I was privileged, and the research money I obtained was more than I needed. It would be much better to reduce the size of grants, so that a larger number of researchers can be funded. Yet the scheme is there, and it is a rare opportunity for researchers in the Netherlands to get funding for their own ideas.

This is my third and final application for a vici-grant. The rules for submission of proposals in this competition limit the number of attempts to three. Why am I going public with this final attempt?

The Open Science Revolution

You will have heard about open science. Most likely you will associate it with the struggle to publish research articles without paywalls, the exploitation of government funded scientists by commercial publishers, and perhaps even with Plan S. You may also associate open science with the struggle to get researchers to publish the data and the code they used to get to their results. Perhaps you have heard about open peer review of research publications. But most likely you will not have heard about open grant review. This is because it rarely happens. I am not the first to publish my proposal; the Open Grants repository currently contains 160 grant proposals. These proposals were shared after the competitions had run. The RIO Journal published 52 grant proposals. This is only a fraction of all grant proposals being created, submitted and reviewed. The many advantages of open science are not limited to funded research, they also apply to research ideas and proposals. By publishing my grant proposal before the competition, the expert reviews, the recommendations of the committee, my responses and experiences with the review process, I am opening up the procedure of grant review as much as possible.

Stages in the NWO Talent Scheme Grant Review Procedure

Each round of this competition takes almost a year, and proceeds in eight stages:

  1. Pre-application – March 26, 2019 <– this is where we are now
  2. Non-binding advice from committee: submit full proposal, or not – Summer 2019
  3. Full proposal – end of August 2019
  4. Expert reviews – October 2019
  5. Rebuttal to criticism in expert reviews – end of October 2019
  6. Selection for interview – November 2019
  7. Interview – January or February 2020
  8. Grant, or not – March 2020

If you’re curious to learn how this application procedure works in practice,
check back in a few weeks. Your comments and suggestions on the ideas above and the pre-proposal are most welcome!

Leave a comment

Filed under altruism, charitable organizations, data, economics, empathy, experiments, fundraising, happiness, helping, household giving, incentives, methodology, open science, organ donation, philanthropy, politics, principle of care, psychology, regression analysis, regulation, sociology, statistical analysis, survey research, taxes, trends, trust, volunteering, wealth

Uncertain Future for Giving in the Netherlands Panel Survey

By Barbara Gouwenberg and René Bekkers

At the Center for Philanthropic Studies we have been working hard to secure funding for three rounds of funding for the Giving in the Netherlands Study, including the Giving in the Netherlands Panel Survey for the years 2020-2026. During the previous round of the research, the ministry of Justice and Security has said that it would no longer fund the study on its own, because the research is important not only for the government but also for the philanthropic sector. The national government no longer sees itself as the sole funder of the research.

The ministry does think the research is important and is prepared to commit funding for the research in the form of a 1:1 matching subsidy to contributions received by VU Amsterdam from other funders. To strengthen the societal relevance and commitment for the Giving in the Netherlands study the Center has engaged in a dialogue with relevant stakeholders, including the council of foundations, the association of fundraising organizations, and several endowed foundations and fundraising charities in the Netherlands. The goal of these talks was to get science and practice closer together. From these talks we have gained three important general insights:

  • The Giving in the Netherlands study contributes to the visibility of philanthropy in the Netherlands. This is important for the legitimacy of an autonomous and growing sector.
  • It is important to engage in a conversation with relevant stakeholders before the fieldwork for a next round starts, in order to align the research more strongly with practice.
  • After the analyses have been completed, communication with relevant stakeholders about the results should be improved. Stakeholders desire more conversations about the application of insights from the research in practice.

The center includes these issues in the plans for the upcoming three editions. VU Amsterdam has been engaged in conversations with branch organizations and individual foundations in the philanthropic sector for a long time, in order to build a sustainable financial model for the future of the research. However, at the moment we do not have the funds together to continue the research. That is why we did not collect data for the 2018 wave of the Giving in the Netherlands Panel Survey. As a result, we will not publish estimates for the size and composition of philanthropy in the Netherlands in spring 2019. We do hope that after this gap year we can restart the research next year, with a publication of new estimates in 2020.

Your ideas and support are very welcome at r.bekkers@vu.nl.

2 Comments

Filed under Center for Philanthropic Studies, charitable organizations, contract research, data, experiments, foundations, fundraising, household giving, methodology, Netherlands, philanthropy, policy evaluation, statistical analysis, survey research