Introducing Mega-analysis

How to find truth in an ocean of correlations – with breakers, still waters, tidal waves, and undercurrents? In the old age of responsible research and publication, we would collect estimates reported in previous research, and compute a correlation across correlations. Those days are long gone.

In the age of rat race research and publication it became increasingly difficult to do a meta-analysis. It is a frustrating experience for anyone who has conducted one: endless searches on the Web of Science and Google Scholar to collect all published research, input the estimates in a database, find that a lot of fields are blank, email authors for zero-order correlations and other statistics they had failed to report in their publications and get very little response.

Meta-analysis is not only a frustrating experience, it is also a bad idea when results that authors do not like do not get published. A host of techniques have been developed to find and correct publication bias, but the problem that we do not know the results that do not get reported is not solved easily.

As we enter the age of open science,  we do not have to rely any longer on the far from perfect cooperation from colleagues who have moved to a different university, left academia, died, or think you’re trying to prove them wrong and destroy your career. We can simply download all the raw data and analyze them.

Enter mega-analysis: include all the data points relevant for a certain hypothesis, cluster them by original publication, date, country, or any potentially relevant property of the research design, and add the substantial predictors you find documented in the literature. The results reveal not only the underlying correlations between substantial variables, but also the differences between studies, periods, countries and design properties that affect these correlations.

The method itself is not new. In epidemiology, and Steinberg et al. (1997) labeled it ‘meta-analysis of individual patient data’. In human genetics, genome wide association studies (GWAS) by large international consortia are common examples of mega-analysis.

Mega-analysis includes the file-drawer of papers that never saw the light of day after they were put in. It also includes the universe of papers that have never been written because the results were unpublishable.

If meta-analysis gives you an estimate for the universe of published research, mega-analysis can be used to detect just how unique that universe is in the milky way. My prediction would be that correlations in published research are mostly further from zero than the same correlation in a mega-analysis.

Mega-analysis bears great promise for the social sciences. Samples for population surveys are large, which enables optimal learning from variations in sampling procedures, data collection mode, and questionnaire design. It is time for a Global Social Science Consortium that pools all of its data. As an illustration, I have started a project on the Open Science Framework that mega-analyzes generalized social trust. It is a public project: anyone can contribute. We have reached mark of 1 million observations.

The idea behind mega-analysis originated from two different projects. In the first project, Erik van Ingen and I analyzed the effects of volunteering on trust, to check if results from an analysis of the Giving in the Netherlands Panel Survey (Van Ingen & Bekkers, 2015) would replicate with data from other panel studies. We found essentially the same results in five panel studies, although subtle differences emerged in the quantative estimates. In the second project, with Arjen de Wit and colleagues from the Center for Philanthropic Studies at VU Amsterdam, we analyzed the effects of volunteering on well-being conducted as part of the EC-FP7 funded ITSSOIN study. We collected 845.733 survey responses from 154.970 different respondents in six panel studies, spanning 30 years (De Wit, Bekkers, Karamat Ali & Verkaik, 2015). We found that volunteering is associated with a 1% increase in well-being.

In these projects, the data from different studies were analyzed separately. I realized that we could learn much more if the data are pooled in one single analysis: a mega-analysis.


De Wit, A., Bekkers, R., Karamat Ali, D., & Verkaik, D. (2015). Welfare impacts of participation. Deliverable 3.3 of the project: “Impact of the Third Sector as Social Innovation” (ITSSOIN), European Commission – 7th Framework Programme, Brussels: European Commission, DG Research.

Van Ingen, E. & Bekkers, R. (2015). Trust Through Civic Engagement? Evidence From Five National Panel StudiesPolitical Psychology, 36 (3): 277-294.

Steinberg, K.K., Smith, S.J., Stroup, D.F., Olkin, I., Lee, N.C., Williamson, G.D. & Thacker, S.B. (1997). Comparison of Effect Estimates from a Meta-Analysis of Summary Data from Published Studies and from a Meta-Analysis Using Individual Patient Data for Ovarian Cancer Studies. American Journal of Epidemiology, 145: 917-925.

Leave a comment

Filed under data, methodology, open science, regression analysis, survey research, trends, trust, volunteering

Gevonden: Student assistent voor het onderzoek Geven in Nederland

De werkgroep Filantropische Studies van de Faculteit Sociale Wetenschappen aan de Vrije Universiteit Amsterdam is het expertisecentrum op het gebied van onderzoek naar filantropie in Nederland. De werkgroep houdt zich bezig met vragen zoals: Waarom geven mensen vrijwillig geld aan goede doelen? Waarom verrichten mensen vrijwilligerswerk? Hoeveel geld gaat er om in de filantropische sector? Voor het onderzoek Geven in Nederland heeft de werkgroep Suzanne Felix aangenomen als onderzoeksassistent.



Geven in Nederland is een van de belangrijkste onderzoeksprojecten van de werkgroep. Sinds 1995 wordt het geefgedrag van huishoudens, individuen, fondsen, bedrijven en goededoelenloterijen elke twee jaar in kaart gebracht en samengevoegd tot een macro-economisch overzicht. De werkgroep Filantropische Studies brengt de resultaten van het onderzoek tweejaarlijks uit in het boek ‘Geven in Nederland’. Felix werkt mee aan het onderzoek naar nalatenschappen en giften door vermogensfondsen en huishoudens.
update: 3 september 2016

Leave a comment

Filed under Uncategorized

Brief guide to understand fMRI studies

RQ: Which regions of the brain are active when task X is performed?

Results: Activity in some regions Y is higher than in others.

Leave a comment

Filed under Uncategorized

Heeft de culturele sector de cultuuromslag naar ondernemerschap gemaakt?

Presentatie rapport Culturele instellingen in Nederland’

Werkgroep Filantropische Studies Vrije Universiteit Amsterdam


Vrijdag 10 juni 2016, Theater Griffioen, Uilenstede 106, 1183 AM, Amstelveen


In 2012 werd de Geefwet ingevoerd met een multiplier die de aftrekbaarheid van giften aan culturele instellingen verhoogde. Bovendien kregen culturele instellingen meer mogelijkheden eigen inkomsten te genereren uit commerciële activiteiten. Tegelijk kregen veel instellingen te maken met bezuinigingen en de vraag om meer ondernemerschap. Hoe hebben Nederlandse particulieren en bedrijven met een hart voor cultuur gereageerd op de verhoogde aftrekbaarheid van giften aan cultuur? Zijn zij ook inderdaad meer gaan geven? En hoe hebben de culturele instellingen gereageerd op de bezuinigingen enerzijds en de multiplier anderzijds? Wat voor instellingen hebben de omslag naar ondernemerschap wel kunnen maken en wat voor instellingen niet?

Deze vragen stonden centraal in een onderzoek dat de werkgroep Filantropische Studies heeft uitgevoerd op verzoek van het ministerie van OCW naar de effecten van de Geefwet op het genereren van inkomsten door culturele instellingen. Het onderzoek verschaft inzicht in de stand van zaken van de culturele sector op dit gebied en de mate waarin de Geefwet bijdraagt aan de versterking van de culturele sector door stimulering van giften aan cultuur.

U bent van harte welkom op een symposium waarop de onderzoekers de resultaten presenteren aan de culturele sector. U kunt zich hier aanmelden.



15.30    Aanmelden

16.00    Presentatie onderzoek door prof. dr. René Bekkers

16.30    Annabelle Birnie, Drents Museum

16.45    Marielle Hendriks, Boekmanstichting

17.00    Drankje




Theater Griffioen, Uilenstede 106, 1183 AM, Amstelveen

Routebeschrijving – klik hier



Meer informatie

Meer informatie over het onderzoek vindt u op

Leave a comment

Filed under Uncategorized

Philanthropy: from Charity to Prosocial Investment

Contribution to the March 2016 edition of the European Research Network on Philanthropy (ERNOP) newsletter. PDF version here.

Philanthropy can take many forms. It ranges from the student who showed up at my doorstep with a collection tin to raise small contributions for legal assistance to the poor to the recent announcement by Facebook co-founder Mark Zuckerberg and his wife Priscilla Chan of the establishment of a $42 billion charitable foundation. The media focused on the question why Zuckerberg and Chan would put 99% of their wealth in a foundation. The legal form of the foundation allowed Zuckerberg to keep control over the shares without having to pay taxes. Leaving aside the difficult question what motivation the legal form confesses for the moment, my point is that a change is taking place in the face that philanthropy takes.

Entrepreneurial forms of philanthropy, manifesting a strategic investment orientation, become more visible. We see them in social impact bonds, in social enterprises, in venture philanthropy and in the investments of foundations in the development of new drugs and treatments. A reliable count of the prevalence of such prosocial investments is not available, but 2015 was certainly a memorable year: the first Ebola vaccine was produced in a lab funded by the Wellcome Trust and polio was eradicated from Africa through coordinated efforts supported by a coalition of the WHO, Unicef, the Rotary International Foundation, and the Bill and Melinda Gates Foundation.

Of course there are limitations to philanthropy. Some problems are just too big to handle, even for the wealthiest foundations on earth, using the most innovative forms of investments. The refugee crisis continues to challenge the resilience of Europe. NGOs are delivering relief aid in the most difficult circumstances. But these efforts are band aids, as long as political leaders are struggling to gather the will power to solve it together.

The Zuckerberg/Chan announcement revived previous critiques of philanthrocapitalism. Isn’t it dangerous to have so much money in so few hands? Can we rely on wealthy foundations to invest in socially responsible ways? Foundations are the freest institutions on earth and can take risks that governments cannot afford. But the track records of the corporations that gave rise to the current foundation fortunes are not immaculate, monopolizing markets and evading taxes. Wealthy foundations can have a significant impact on society and influence public policy, limiting the influence of governments. It is political will that enables the existence and facilitates the fortune of wealthy foundations. Ultimately, the realization that the interests of the people should not be harmed enables the activities of foundations. Hence the talk about the importance of giving back to society.

The sociologist Alvin Gouldner is famous for his 1960 article ‘The Norm of Reciprocity’, which describes how reciprocity works. He also wrote a second classic, much less known: ‘The Importance of Something for Nothing.’ In this follow-up (1973), he stresses the norm of beneficence: “This norm requires men to give others such help as they need. Rather than making help contingent upon past benefits received or future benefits expected, the norm of beneficence calls upon men to aid others without thought of what they have done or what they can do for them, and solely in terms of a need imputed to the potential recipient.” In a series of studies I co-authored with Mark Ottoni-Wilhelm, an economist from the Lilly Family School of Philanthropy at Indiana University, we call this norm ‘the principle of care’.

With this quote I return to the question about motivation. The letter to their daughter in which Zuckerberg and Chan announced their foundation reveals noble concerns for the future of mankind. It is not their child’s need that motivated them, but the needs of the world in which she is born. This is the genesis of true philanthropy. Pretty much like the awareness of need that the law student demonstrated at my doorstep.


Bekkers, R. & Ottoni-Wilhelm, M. (2016). Principle of Care and Giving to Help People in Need. European Journal of Personality.  

Gouldner, A.W. (1960). The Norm of Reciprocity: A Preliminary Statement. American Sociological Review, 25 (2): 161-178.

Gouldner, A.W. (1973). The Importance of Something for Nothing. In: Gouldner, A.W. (Ed.). For Sociology, Harmondsworth: Penguin.

Wilhelm, M.O., & Bekkers, R. (2010). Helping Behavior, Dispositional Empathic Concern, and the Principle of Care. Social Psychology Quarterly, 73 (1): 11-32.

Leave a comment

Filed under altruism, charitable organizations, foundations, law, philanthropy, principle of care, taxes

Conditional Review Acceptance Policy (R1)

“Thank you for your invitation to review. Please complete this form so that I can determine whether your review request is eligible.”

This is my new automatic reply to requests for review journal articles that I receive from editors and their assistants. In june 2014, I introduced a conditional review acceptance policy (CRAP). The policy was to review only those articles that the journal agrees to publish in a Free Open Access mode – making the article publicly available, without charging any fees for it from universities, authors, or readers. The revised policy now also includes the question whether the data and code will be publicly available, as proposed by the Peer Reviewers’ Openness (PRO) initiative. The revision rewards open science.



Leave a comment

Filed under Uncategorized

The Future of Foundation Support for Research and Innovation

Recently the EUFORI Study was published, in which a network of experts coordinated from our Center for Philanthropic Studies at the Vrije Universiteit Amsterdam mapped the support from foundations for research and innovation in Europe. Read the synthesis report here. The research was based on an extensive survey of 1 591 foundations supporting R&I in Europe and a qualitative analysis of 29 different country reports. We concluded that foundations contribute a significant amount of money to R&I: annually at least €5 billion, out of an asset base worth at least €127 billion. These are lower bound estimations, because it was impossible to estimate contributions by foundations that did not participate in the study.

The final chapter of the synthesis report presents six recommendations. The main objective of the recommendations made in this final chapter is to increase the potential of R&I foundations in Europe. Considering the underlying potential, actions towards greater support by foundations for research and innovation should and must involve engaging all actors: national governments, EU institutions, the foundations themselves, the corporate sector, universities and other research institutes, and the public at large.

Recommendation 1: Increase the visibility of R&I foundations

This recommendation is addressed to foundations, national governments, the EC and EU administration, businesses and the public at large. It relates to the current fragmented landscape of R&I foundations in Europe. The landscape of foundations in Europe is characterised by a few well-established foundations and many smaller foundations with modest resources mainly operating in the background. Growing visibility will enhance the impact of existing funding. If foundations become more aware of each other’s activities, the effects and impact of their contributions can be increased. Moreover, the other stakeholders involved such as the business community and research policy-makers will become knowledgeable about the foundations’ activities. From the perspective of the beneficiaries, research institutes, universities and researchers will more easily find their way to foundations. Visibility will lower the transaction costs for all the parties involved. For foundations, governments and businesses it will increase their knowledge about ongoing research/new research funded and vice versa. For grantmaking foundations it will facilitate the review process of research proposals and submissions; it is to be expected that more visibility will reduce the amount of incorrect applications. For the beneficiaries of the foundations’ support (research institutes, universities and researchers) – the grantseekers – it will increase their funding opportunities, they will more easily find their way to foundations, and it will facilitate submission processes. For potential (major) donors it will offer visible causes to benefit. Increasing the visibility of R&I foundations could have a positive effect on potential (major) donors as it could encourage them to support a research foundation. Increasing the visibility of and information about R&I foundations was already addressed by an expert group in 2005. They argued: ‘.. foundations and their donors would be more aware of the foundation landscape (increasing collaborative working and, possibly, giving), foundations’ contribution to various sectors could be properly assessed and the information could inform policy-making in this area. It is in fact a prerequisite to other actions’. The present EUFORI study is a step forward. A lot of information is now available. Next to this synthesis report, 29 country-reports, new data, an active network of researchers and the EUFORI website can contribute to the profiling of the R&I foundation sector in Europe.

With the exception of some large and well-established foundations in Europe, there is a lack of a common research identity among the foundations supporting R&I in most countries. Research and innovation are often not seen as a purpose/field in itself but are instead used as an instrument for other purposes and areas in which foundations specialise (such as health, technology, society). This is reflected by a lack of dialogue between the foundations supporting R&I (occasionally between foundations that deal with similar topics, e.g. foundations supporting cancer research). Bringing foundations together at a European level and following the recommendations of the expert group from 2005, the European Foundation Center (EFC) created the European Forum of Research Foundations. This forum provides a platform for a group of large and well-known R&I foundations in Europe. In order to increase the visibility of foundations supporting R&I at a national level, the encouragement of the creation of national forums of research foundations is recommended as the next step. The opportunities and mutual benefits for foundations supporting R&I at a national level should be explored. The next step: Explore the opportunities and mutual benefits of the creation of national forums of research foundations.

Recommendation 2: Explore synergies through collaboration

Unity in diversity is one of the main challenges for all the players involved in the R&I domain. These players can be distinguished in the domain of research (governments, business, foundations and research institutes/researchers), each with their own distinctive role. Together these groups can make a difference in increasing the potential for R&I. They can create synergy through collaboration, which should be interpreted in the broadest sense, varying from information sharing, networking, co-funding and partnerships. Mutual advantages can be derived from pooling expertise, sharing infrastructure, expanding activities, pooling money due to a lack of necessary funds, avoiding the duplication of efforts and creating economies of scale.

Get to know each other, meet and see where to reinforce each other’s efforts

Based on the conclusions of the EUFORI Study there is an indication for the need for improved dialogue, information exchange, networking and cooperation between the foundations supporting R&I, as well as between foundations, governments, business and research institutes (researchers). The needs, opportunities, mutual benefits and barriers for collaboration should be further explored, including mutual responsibilities when cooperating. The creation of national forums or networks of foundations supporting research and innovation, regular meetings between the foundations and other stakeholders involved (national government, EU government, research institutes and business) could bring these groups together.

An EU-wide study is recommended on the needs, opportunities, mutual benefits and barriers for collaboration between foundations, national governments, the European Commission, the business sector and research institutes. A network of national experts (mostly members from ERNOP) built for the EUFORI study can be of added value for this study and can facilitate the collaborative relations between the EC/ RTD, the R&I foundation sector and other stakeholders in Europe. It would be well-advised to set up an independent expert group before the start of this study with selected experts and stakeholder representatives in the field of foundations, the business sector, research institutes and public authorities at a national and European level. The expert group should provide input for the design of the study and could adopt an advisory role. Subsequently, it is recommended that the study will be finished by a follow-up conference for all the players involved aimed to discuss the implementation of the outcomes of the Collaboration Infrastructure Study. In this call for collaboration we have to consider two possible, interrelated pitfalls; namely the danger of ‘substitution’ and the danger of threatening the independence of foundations. Foundations, and civil initiatives in general, make their own choices and preferences and are based on social democracy. Governments, on the other hand, have their own responsibility based on political democracy. Businesses have their own market-driven values. Sometimes they reinforce each other, sometimes they may act as opponents. It concerns different worlds, differing in terms of constitution, values, legitimacy and organisation style. The independence of private R&I foundations should be respected. Foundations derive their legitimacy from many contacts with the ‘capillaries’ in society, thus offering them the opportunity to function ‘as the eyes and ears’ for innovation. This grass-roots connection represents the philanthropic tradition in Europe: ‘voluntary action to serve the public good’. The next step: Launch a Collaboration Infrastructure Study.

Continue reading the recommendations from the EUFORI study here.




Leave a comment

Filed under Uncategorized