Text

Science funding is borked: Part II

Previously I had argued in “Science funding is borked" that we should be giving out many more and smaller grants, similar to the 500 startups approach. 

Now in a new paper published in PLOS ONE and reported in Times Higher Education, it seems that this argument is starting to gain some data-driven support. 

Here’s the PLOS ONE paper Big Science vs. Little Science: How Scientific Impact Scales with Funding

Text

Science funding is borked

Michael Jordan, arguably the greatest basketball player yet. His son? Not so much. Wayne Gretsky, also arguably one of the greatest hockey players ever. His father? Not so much. The list of superstar athletes goes on. Similarly, children of the world’s tallest people are never as tall as their parents. What’s going on here?

It’s a well known statistical ‘rule’ known as regression toward the mean. In any sampling with extreme outliers, a second measurement will not have such extreme outliers. It happens throughout nature and is also visible athletics. Though less proven, it is also visible in business and in culture.

We can take the investment world as an example. For every Google or Facebook, there are at least 10x the number of failed investments or ‘small wins’ at best. The venture capital industry is at an all-time low over the past decade in terms of return on investment, ROI. The big VC firms will sink hundreds of millions of capital investment into approximately 10 companies per year in hopes that ONE will be a ‘homerun.’ Over the past decade, even that one in ten strategy has often failed. This part is important: These are not just a random 10 companies either. They are companies with past success (e.g. revenue or customers) and often run by experienced founders with past successes. Do you see what is going on here? Regression towards the mean.

That statistical phenomenon is why I am troubled by the strategy of science funding agencies today. An article in last week’s UK Guardian highlights how funding agencies such as the Wellcome Trust, RCUK, European Research Council, Royal Society, etc all have the strategy of funding the ‘superstars’ of academia. The same strategy is occurring in the U.S. as well. The Guardian article highlights that this strategy is terrible for up-and-coming academics who find it increasingly difficult to be awarded funding. It is also drying up funding money across a breadth of universities in favor of concentrating that money into just a few. This is certainly a bad strategy in terms of what it means to individual researchers, but perhaps even more importantly, it is a bad strategy for science as a whole. Why? That whole regression towards the mean thing as discussed above.

'Superstar' academics are similar to the Michael Jordans and the successful startup entrepreneurs. Regression towards the mean indicates that follow-on success is rare. One could argue that since it is the post-docs doing the research in most superstar academics’ labs that this statistical principle no longer applies. I would argue that is still does, as often only ‘superstar’ grad students or post-docs are selected to join the ‘superstar’ lab. That’s speaking about the people, but we must also remember that the projects themselves become concentrated and are subject to regression towards the mean. With fewer projects funded (in the venture capital example above, fewer companies invested in) it means fewer breakthroughs will happen. This is troubling. 

Regression towards the mean is really just the statistical principle behind such clichés such as “don’t put all your eggs into one basket”, “lightning never strikes twice,” and “one-hit wonder.” Why are funding agencies choosing to ignore a well-known principle? It makes no sense.

Here is what I propose. First, we investigate more fully the consequences of current funding strategies that concentrate research money into just a few individuals or universities. Personally I’d forgo the first point, but this is governement or large private foundations who don’t change things without further study, so I’m being realistic here. The second thing that we do, and where I would personally start, is to start copying the “Dave McClure model.” 

Dave was one of the first engineers at PayPal before anyone knew that PayPal existed. A few years ago he noticed something about the investment strategies of VCs. He recognized the regression towards the mean principle taking place. Dave has done something most other investors have not done, which is to invest small chunks into a TON of companies. He’s already approaching 500 companies since starting this strategy in 2010. In fact, his early-stage investment group is called 500 startups. And his success rate? Much higher than the old investment model of pouring huge amounts into a tiny number of companies. Yet, Dave does more than just give money to new ideas. He also nurtures those ideas within a framework and network of mentors who have ‘been there, done that.’ This makes a lot of sense.

Can we apply the 500 startups strategy to science funding then? I think we can. Start by awarding smaller chunks of funding to many more academics*. BUT, we must also create a mentorship framework, which today is virtually non-existant at the faculty level in academia. The software industry has ‘best practices’ for programming, etc. Surely by now, we have best practices for the scientific method, writing, thinking, navigating the tenure track, etc within academia. Yet, I hardly see all of this codified and combined with personal case studies from successful academics. Funding agencies should not leave it to individual departments to set that up, because it will rarely happen. If funding agencies, and more importantly the public, want a larger ROI then we need to rethink how we distribute and utilize grant money. 

Let’s unbork the funding crisis.

*[This seemingly would upset many academics who already have large research budgets. There are solutions to this as well, which I won’t go into now.]

jason