Category Archives: open science

Revolutionizing Philanthropy Research Webinar

January 30, 11am-12pm (EST) / 5-6pm (CET) / 9-10pm (IST)

Why do people give to the benefit of others – or keep their resources to themselves? What is the core evidence on giving that holds across cultures? How does giving vary between cultures? How has the field of research on giving changed in the past decades?

10 years after the publication of “A Literature Review of Empirical Studies of Philanthropy: Eight Mechanisms that Drive Charitable Giving” in Nonprofit and Voluntary Sector Quarterly, it is time for an even more comprehensive effort to review the evidence base on giving. We envision an ambitious approach, using the most innovative tools and data science algorithms available to visualize the structure of research networks, identify theoretical foundations and provide a critical assessment of previous research.

We are inviting you to join this exciting endeavor in an open, global, cross-disciplinary collaboration. All expertise is very much welcome – from any discipline, country, or methodology. The webinar consists of four parts:

  1. Welcome: by moderator Pamala Wiepking, Lilly Family School of Philanthropy and VU Amsterdam;
  2. The strategy for collecting research evidence on giving from publications: by Ji Ma, University of Texas;
  3. Tools we plan to use for the analyses: by René Bekkers, Vrije Universiteit Amsterdam;
  4. The project structure, and opportunities to participate: by Pamala Wiepking.

The webinar is interactive. You can provide comments and feedback during each presentation. After each presentation, the moderator selects key questions for discussion.

We ask you to please register for the webinar here: https://iu.zoom.us/webinar/register/WN_faEQe2UtQAq3JldcokFU3g.

Registration is free. After you register, you will receive an automated message that includes a URL for the webinar, as well as international calling numbers. In addition, a recording of the webinar will be available soon after on the Open Science Framework Project page: https://osf.io/46e8x/

Please feel free to share with everyone who may be interested, and do let us know if you have any questions or suggestions at this stage.

We look forward to hopefully seeing you on January 30!

You can register at https://iu.zoom.us/webinar/register/WN_faEQe2UtQAq3JldcokFU3g

René Bekkers, Ji Ma, Pamala Wiepking, Arjen de Wit, and Sasha Zarins

1 Comment

Filed under altruism, bequests, charitable organizations, crowdfunding, economics, experiments, fundraising, helping, household giving, informal giving, open science, philanthropy, psychology, remittances, sociology, survey research, taxes, volunteering

The Magic of Science

Dinosaurs are like magic. They capture the attention because of their size and sharp teeth. The fact they are no longer among us may also contribute to their popularity. In science, we still have dinosaurs. They do date back to the prehistoric age, when scientists could build careers on undisclosed data and procedures. But we have entered the new age of open science, with comets and earthquakes causing dark clouds in the sky and blocking our view of the sun.

dino_bummer

In the prehistoric age, a lot of science was like magic. The wizard waved his wand, and…. poof: there was the result that only the wizard could reproduce. If nobody can repeat your trick, it’s not science. When you dig up old research, you are stuck with a lot of ‘magic’. Make sure you can detect it.

magic

Unlike real magic, the tricks of illusionists are highly reproducible. It may take some time to learn tricks and you will need the appropriate equipment, but if you know the secret recipe, you can dress up like a magician, and perform the very same act you could not figure out when you were in the audience.

Needless to say, it is our collective responsibility to disclose all the tricks and equipment we use in our research. Here’s a list of things we can do to make this happen.

Leave a comment

Filed under open science

A Conversation About Data Transparency

The integrity of the research process serves as the foundation for excellence in research on nonprofit and voluntary action. While transparency does not guarantee credibility, it guarantees you will get the credibility you deserve. Therefore we are developing criteria for transparency standards with regards to the reporting of methods and data.

We started this important conversation at the 48th ARNOVA Conference in San Diego, on Friday, November 22, 2019. In the session, we held a workshop to survey which characteristics of data and methods transparency that help review research and utilize past work as building blocks for future research.

This session was well attended and very interactive. After a short introduction by the editors of NVSQ, the leading journal in the field, we split up in three groups of researchers that work with the same type of data. One group for data from interviews, one for survey data, and one for administrative data such as 990s. In each group we first took 10 minutes for ourselves, formulating criteria for transparency that allow readers to assess the quality of research. All participants received colored sticky notes, and wrote down one idea per note: laudable indicators on green notes, and bad signals on red notes.

IMG-2421

Next, we put the notes on the wall and grouped them. Each cluster received a name on a yellow note. Finally, we shared the results of the small group sessions with the larger group.

IMG-2424

Though the different types of data to some extent have their own quality indicators, there were striking parallels in the match between theory and research design, ethics, sampling, measures, analysis, coding, interpretation, and write-up of results. After the workshop, we collected the notes. I’ve summarized the results in a report about the workshop. In a nutshell, all groups distinguished five clusters of criteria:

  • A. Meta-criteria: transparency about the research process and the data collection in particular;
  • B. Before data collection: research design and sampling;
  • C. Characteristics of the data as presented: response, reliability, validity;
  • D. Decisions about data collected: analysis and causal inference;
  • E. Write-up: interpretation of and confidence in results presented.

bty

Here is the full report about the workshop. Do you have suggestions about the report? Let me know!

Leave a comment

Filed under data, experiments, methodology, open science, survey research

Conditional Review Acceptance Policy (R2)

“Thank you for your invitation to review. Did the authors provide the data and the code they have used to produce the paper? If not, can you ask them to?”

This is my rerevised Conditional Review Acceptance Policy (CRAP – R2). Five years ago, I introduced a conditional review acceptance policy (CRAP). Later I revised the policy. In the current rerevision I have dropped the requirement that the paper should be published in open access. In the past five years, we have progressed enormously on the path to open publishing. Moreover, in the coming years Plan S is likely to deliver the promise that all academic research can be read online at no additional cost to the reader. So the remaining challenge is open access to data and code. Without access to the materials that form the basis for a paper, it is very hard to evaluate whether the paper is a good reflection of these materials.

So if you invite me to review a paper, please send me the data and code, preferably at a public repository.

Leave a comment

Filed under academic misconduct, data, open science

Global Giving: Open Grant Proposal

Here’s an unusual thing for you to read: I am posting a brief description of a grant proposal that I will submit for the ‘vici’-competition of the Netherlands Organization for Scientific Research 2019 later this year. You can download the “pre-proposal” here. It is called “Global Giving”. With the study I aim to describe and explain philanthropy in a large number of countries across the world. I invite you to review the “pre-proposal” and suggest improvements; please use the comments box below, or write to me directly.

You may have heard the story that university researchers these days spend a lot of their time writing grant proposals for funding competitions. Also you may have heard the story that chances of success in such competitions are getting smaller and smaller. These stories are all true. But the story you seldom hear is how such competitions actually work: they are a source of stress, frustration, burnouts and depression, and a complete waste of the precious time of the smartest people in the world. Recently, Gross and Bergstrom found that “the effort researchers waste in writing proposals may be comparable to the total scientific value of the research that the funding supports”.

Remember the last time you saw the announcement of prize winners in a research grant competition? I have not heard a single voice in the choir of the many near-winners speak up: “Hey, I did not get a grant!” It is almost as if everybody wins all the time. It is not common in academia to be open about failures to win. How many vitaes you have seen recently contain a list of failures? This is a grave distortion of reality. Less than one in ten applications is succesful. This means that for each winning proposal there are at least nine proposals that did not get funding. I want you to know how much time is wasted by this procedure. So here I will be sharing my experiences with the upcoming ‘vici’-competition.

single-shot-santa

First let me tell you about the funny name of the competition. The name ‘vici’ derives from roman emperor Caesar’s famous phrase in Latin: ‘veni, vidi, vici’, which he allegedly used to describe a swift victory. The translation is: “I came, I saw, I conquered”. The Netherlands Organization for Scientific Research (‘Nederlandse organisatie voor Wetenschappelijk Onderzoek’, NWO) thought it fitting to use these names as titles of their personal grant schemes. The so-called ‘talent schemes’ are very much about the personal qualities of the applicant. The scheme heralds heroes. The fascination with talent goes against the very nature of science, where the value of an idea, method or result is not measured by the personality of the author, but by its validity and reliability. That is why peer review is often double blind and evaluators do not know who wrote the research report or proposal.

plt132

Yet in the talent scheme, the personality of the applicant is very important. The fascination with talent creates Matthew effects, first described in 1968 by Robert K. Merton. The name ‘Matthew effect’ derives from the biblical phrase “For to him who has will more be given” (Mark 4:25). Simply stated: success breeds success. Recently, this effect has been documented in the talent scheme by Thijs Bol, Matthijs de Vaan and Arnout van de Rijt. When two applicants are equally good but one – by mere chance – receives a grant and the other does not, the ‘winner’ is ascribed with talent and the ‘loser’ is not. The ‘winner’ then gets a tremendously higher chance of receiving future grants.

As a member of committees for the ‘veni’ competition I have seen how this works in practice. Applicants received scores for the quality of their proposal from expert reviewers before we interviewed them. When we had minimal differences between the expert reviewer scores of candidates – differing only in the second decimal – personal characteristics of the researchers such as their self-confidence and manner of speaking during the interview often made the difference between ‘winners’ and ‘losers’. Ultimately, such minute differences add up to dramatically higher chances to be a full professor 10 years later, as the analysis in Figure 4 of the Bol, De Vaan & Van de Rijt paper shows.

matthew

My career is in this graph. In 2005, I won a ‘veni’-grant, the early career grant that the Figure above is about. The grant gave me a lot of freedom for research and I enjoyed it tremendously. I am pretty certain that the freedom that the grant gave me paved the way for the full professorship that I was recently awarded, thirteen years later. But back then, the size of the grant did not feel right. I felt sorry for those who did not make it. I knew I was privileged, and the research money I obtained was more than I needed. It would be much better to reduce the size of grants, so that a larger number of researchers can be funded. Yet the scheme is there, and it is a rare opportunity for researchers in the Netherlands to get funding for their own ideas.

This is my third and final application for a vici-grant. The rules for submission of proposals in this competition limit the number of attempts to three. Why am I going public with this final attempt?

The Open Science Revolution

You will have heard about open science. Most likely you will associate it with the struggle to publish research articles without paywalls, the exploitation of government funded scientists by commercial publishers, and perhaps even with Plan S. You may also associate open science with the struggle to get researchers to publish the data and the code they used to get to their results. Perhaps you have heard about open peer review of research publications. But most likely you will not have heard about open grant review. This is because it rarely happens. I am not the first to publish my proposal; the Open Grants repository currently contains 160 grant proposals. These proposals were shared after the competitions had run. The RIO Journal published 52 grant proposals. This is only a fraction of all grant proposals being created, submitted and reviewed. The many advantages of open science are not limited to funded research, they also apply to research ideas and proposals. By publishing my grant proposal before the competition, the expert reviews, the recommendations of the committee, my responses and experiences with the review process, I am opening up the procedure of grant review as much as possible.

Stages in the NWO Talent Scheme Grant Review Procedure

Each round of this competition takes almost a year, and proceeds in eight stages:

  1. Pre-application – March 26, 2019 <– this is where we are now
  2. Non-binding advice from committee: submit full proposal, or not – Summer 2019
  3. Full proposal – end of August 2019
  4. Expert reviews – October 2019
  5. Rebuttal to criticism in expert reviews – end of October 2019
  6. Selection for interview – November 2019
  7. Interview – January or February 2020
  8. Grant, or not – March 2020

If you’re curious to learn how this application procedure works in practice,
check back in a few weeks. Your comments and suggestions on the ideas above and the pre-proposal are most welcome!

Leave a comment

Filed under altruism, charitable organizations, data, economics, empathy, experiments, fundraising, happiness, helping, household giving, incentives, methodology, open science, organ donation, philanthropy, politics, principle of care, psychology, regression analysis, regulation, sociology, statistical analysis, survey research, taxes, trends, trust, volunteering, wealth

Onderzoek Geven in Nederland in gevaar

Door Barbara Gouwenberg – uit de nieuwsbrief van de werkgroep Filantropische Studies aan de VU (december 2018)

Het Centrum voor Filantropische Studies werkt momenteel met man en macht om de financiering voor het onderzoek Geven in Nederland voor de komende 6 jaar (3 edities) veilig te stellen. Het Ministerie van Justitie en Veiligheid (J&V) heeft bij de opzet van Geven in Nederland 2017 medio 2015 te kennen gegeven dat het onderzoek niet langer alleen door de overheid zal worden gefinancierd, met als belangrijkste argumentatie dat het onderzoek van belang is voor overheid én sector filantropie. De overheid ziet zichzelf niet langer als enige verantwoordelijke voor de financiering van het onderzoek.

Het Ministerie van J&V wil zich wel voor een langere tijd structureel verbinden aan Geven in Nederland en geeft 1:1 matching voor financiële bijdragen die de VU vanuit de sector ontvangt.

Om de maatschappelijke relevantie van – en commitment voor – het onderzoek Geven in Nederland te versterken heeft de VU de afgelopen maanden de dialoog opgezocht met diverse relevante doelgroepen. Doel: wetenschap en praktijk dichter bij elkaar brengen.

Deze rondgang heeft ons – naast specifieke inzichten – drie belangrijke algemene inzichten opgeleverd; te weten:

  • ‘Geven in Nederland’ draagt bij aan de zichtbaarheid van maatschappelijk initiatief in Nederland. Belangrijk ter legitimatie van een zelfstandige en snel groeiende sector.
  • Communicatie met relevante doelgroepen vóór de start van het onderzoek dient verbeterd te worden met als doel om inhoudelijk beter aansluiting te vinden bij praktijk en beleid.
  • Communicatie over onderzoeksresultaten naar relevante doelgroepen dient verbeterd te worden. Het gaat dan om de praktische toepasbaarheid van het onderzoek, de vertaling van de onderzoeksresultaten naar de praktijk.

De onderzoekers nemen deze verbeterpunten mee in hun plan van aanpak voor de komende drie edities. De VU is al enige tijd in gesprek met de brancheorganisaties en individuele fondsen om tot een duurzaam financieringsmodel voor de toekomst te komen. Op dit moment is de continuering van het onderzoek echter nog niet gegarandeerd. Dat betekent dat er helaas geen Geven in Nederland 2019 komt en dus ook geen presentatie van de nieuwe onderzoeksresultaten zoals u van ons gewend bent op de Dag van de Filantropie. We spreken echter onze hoop uit dat we zeer binnenkort met een Geven in Nederland 2020 kunnen starten!

Leave a comment

Filed under Center for Philanthropic Studies, charitable organizations, contract research, data, foundations, fundraising, household giving, methodology, Netherlands, open science, philanthropy, statistical analysis, survey research, trends, VU University

DOI the DATA

As an open science enthusiast I try to lead by example – you will gather from my blog that I am an online activist when it comes to the incentives in academia and the evaluation of research careers. Last Wednesday I took a few minutes to create a poster for an online campaign to encourage researchers to cite the data they have collected or other data they are using. A lot of work done by researchers to collect data does not get the recognition it should get. This is because the evaluation of research careers in tenure and promotion decisions and grant competitions is currently based on citations of papers, but not on the use of data or software or code. If researchers would cite open data, code and software those who invested time for the benefit of us all would get a bit more of the recognition they deserve. So: DOI your data and get credit for your data. Need help? See https://dataverse.org/best-practices/data-citation and http://help.osf.io/m/sharing/l/524208-create-dois

Here is the poster – post it on your walls, virtual or physical, and put it on your timeline.

DOI_DATA

Leave a comment

Filed under data, open science