Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
The Economics of Science and Innovation Rewards and incentives in science Lecture 10 Natalia Zinovyeva Aalto University
[email protected] 1 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Forms of reward
What does motivate scientists? I I
Taste for science But, perhaps, direct other rewards matter too: F
Non-monetary rewards: reputation and priority
F
Monetary rewards: prizes, wages/bonuses, royalties, etc.
Science is largely a non-market reward system, based on meritocracy and credit granted by peers (Stephan 1996) How can we ensure meritocracy? I
Problems with evaluation by peers Matthew effect
I
Evaluation biases and dynamic incentives
I
2 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Outline
1
Introduction
2
Monetary and non-monetary rewards
3
Evaluation of scientific quality Detecting evaluation biases Matthew effect in science
3 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Outline
1
Introduction
2
Monetary and non-monetary rewards
3
Evaluation of scientific quality Detecting evaluation biases Matthew effect in science
4 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Non-monetary rewards Priority and Reputation
Priority is the key reward mechanism in science (Merton 1957) I
Eponymy: Newton three laws of motion, Darwinian selection, Plank’s constant, Haley’s comet
I
Publications and citations
Strong incentive to be first I
Isaac Newton vs. Gottfried Leibniz
I
Charles Darwin vs. Alfred Russel Wallace
Reward based on priority can only be achieved by making the result publicly available ⇒ no secrecy about discoveries, results are immediately in public domain Think about the difference from IPR system. Should we think about a system of open innovation with recognition of priority?
5 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Prizes
Reputation and research record can then, of course, be monetized Scientific prizes is one of the ways: I In early 1990’s, more than 3,000 prizes in the US alone (Zuckerman 1992) I Nobel prize (1895): Physics, Chemistry, Physiology or Medicine, Literature, and Peace ($1.4 million) I Nobel Memorial Prize in Economic Sciences (1968) ($1.6 million) I Charles Stark Draper Prize - Engineering ($500,000) I Crafoord Prize (1980) - Astronomy and Mathematics; Geosciences, Biosciences ($600,000, research purposes) I Fields Medal (1936) - Mathematics ($15,000)! I John Bates Clark Medal (1947) - Economics ($ 0)! I Abel Prize (2001) - Mathematics ($1 million) I Shaw Prize (2004) - Astronomy, Life Sciences and Medicine, Mathematics ($1 million) 6 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Wages and pomotions
Wage offers, bonuses and promotions: I
I
Promotion and salaries of academic researchers normally depend on achieved publication record In the US in Life Sciences, the mean/90th percentile salary of... F
...an assistant professor in a public university is $76,000/$105,000
F
...a full professor in a public university is $128,500/$200,000.
I
Around 30% higher in private universities.
I
Salaries of assistant professors are higher in fields with strong non-academic job options (computer science, engineering).
In Europe, in many countries salaries are fixed by law. Yet, promotion and hiring are still heavily linked to publication record. Some universities explicitly reward with bonuses the high impact publications.
7 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?!
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?! Several difficult empirical questions: Does patenting I
... encourage technology transfer and licensing? (perhaps)
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?! Several difficult empirical questions: Does patenting I
... encourage technology transfer and licensing? (perhaps)
I
... help to attract private funding? (perhaps)
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?! Several difficult empirical questions: Does patenting I
... encourage technology transfer and licensing? (perhaps)
I
... help to attract private funding? (perhaps)
I
... encourage academic inventors? (perhaps)
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?! Several difficult empirical questions: Does patenting I
... encourage technology transfer and licensing? (perhaps)
I
... help to attract private funding? (perhaps)
I
... encourage academic inventors? (perhaps)
I
... crowd out basic research? (no hard evidence)
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Rents from patents. Should universities be allowed/encouraged to patent?! Several difficult empirical questions: Does patenting I
... encourage technology transfer and licensing? (perhaps)
I
... help to attract private funding? (perhaps)
I
... encourage academic inventors? (perhaps)
I
... crowd out basic research? (no hard evidence)
In the US, since 1980 Bayh-Dole act grants universities the right to retain property rights to inventions deriving from federally funded research. Similar policies followed up in many countries
8 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
University patenting Figure 4.3: Universities’ and PROs’ patents are increasing under the PCT PRO and university PCT applications worldwide, absolute numbers (left) and as a percentage of total PCT applications (right), 1980-2010
PRO
University share
PRO share
7 6
8'000
5
6'000
4 3
4'000
2 2'000
1
0
0
Share in total PCT applications (%)
University
19 80 19 81 19 82 19 83 19 84 19 85 19 86 19 87 19 88 19 89 19 90 19 91 19 92 19 93 19 94 19 95 19 96 19 97 19 98 19 99 20 00 20 01 20 02 20 03 20 04 20 05 20 06 20 07 20 08 20 09 20 10
Number of PCT applications
10'000
Note: As noted in footnote 1, the distinction between universities and PROs often depends on the definition in a given country. The same note applies to the figures which follow. Source: WIPO Statistics Database, June 2011.
University patenting (and licensing) has increased over past decades, now around 7% Figure 4.4 reports the growing share of university and Among high-income countries, the US has the largest of all PRO patent applications worldwide applications from middleand high-income coun- number of university and PRO filings under the PCT with tries as a share of total PCT applications for three periods
52,303 and 12,698 filings respectively (see Figures 4.5
starting in 1980.
and 4.6).43 The second largest source of PRO applications is France with 9,068, followed by Japan with 6,850.
Figure 4.4: Universities and PROs
9 / 28
(10 percent), Germany and SouthMonetary Africa (8 and percent each). rewards Introduction non-monetary 2007, and less activity in Israel and Evaluation the UK. of scientific quality
University-firm joint patenting Figure 4.9: The share of joint university-firm patent applications under the PCT is increasing rapidly Joint university-firm PCT applications in absolute numbers (left) and as a percentage share of total university PCT applications (right): 1980-2010 High-income countries
Middle-income countries
Share in high-income countries
1.800
20
1.600
18
1.400
16 14
1.200
12
1.000
10
800
8
600
6
19
19
19
19
83 19 84 19 85 19 86 19 87 19 88 19 89 19 90 19 91 19 92 19 93 19 94 19 95 19 96 19 97 19 98 19 99 20 00 20 01 20 02 20 03 20 04 20 05 20 06 20 07 20 08 20 09 20 10
0 81
2
0 82
4
200
80
400
/PUFi6OJWFSTJUZýSNDPPXOFSTIJQwSFGFSTUPUIFTJUVBUJPOXIFSFUIFSFBSFBUMFBTUUXPBQQMJDBOUT POFCFJOHBVOJWFSTJUZBOEBOPUIFSCFJOHBDPNQBOZ Inventors are not considered. The share of university-firm applications in total PCT applications by middle-income countries are not shown due to their high volatility. Since 2001 this share has been in the range between 16.9 percent and 34.5 percent. Source: WIPO Statistics Database, June 2011.
University-firm patenting has increased as well. Today around 18% of patents in high-income countries are the result of collaborative research between firms and 150 universities. 10 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Monetary incentives Royalties
Royalties for academic inventors: I
14% of faculty in the US reported being an inventor of a patent in previous 5 years (Stephan 2012).
I
U.S. universities licensing revenue – $82 million in 1990, $1,9 billion – in 2007. 76% comes from life sciences. Faculty typically get around 42%. Blockbuster University Patents:
I
I
F
Cohen-Boyer patent for gene splicing (1972, Stanford and UC-SF, $255 million)
F
Anti-inflammatory drug (Remicade) (2007, NYU, $650 million)
F
Drug against fibromyalgia (Lyrica) (2006, Northwestern U, $700 million)
But only a handful of faculty earn from royalties: Stephan (2012) calculates that only 400 researchers in the US share most of royalties.
11 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Other rewards and incentives
Academic start-ups I
More than 3,000 start-ups in the US in 1980-2000, around 8% go public, IPOs of university start-ups – median value of $3 million to $9 million.
Consultancy work for private and public organizations (Note potential conflicts of interests!) Research grants Fees from teaching activities
12 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Outline
1
Introduction
2
Monetary and non-monetary rewards
3
Evaluation of scientific quality Detecting evaluation biases Matthew effect in science
13 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
How to evaluate scientific quality? Appropriate attribution of merit is key for ensuring correct incentives in sciences How to achieve meritocracy? Typically, scientists and their research are evaluated by peers via peer review process But this process has its own problems. Various issues arise: I Quality is subjective. Evaluations might be affected by herding. I Preferences for the same subfield I Poor incentives of peers. Favoritism and cronyism I Stereotypes Various solutions are discussed/used: I Single/double blind evaluations (information flow vs. bias?) I Conflict of interest rules (better information vs bias?) I Quotas (do they work?) I Bibliometric qualify indicators (can imperfect indicators create bad dynamic incentives?) I Feedback mechanisms: reputation, competition for funds
14 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Effect of connections: bias or information? Zinovyeva and Bagues (2015 AEJ:Applied)
The most informed evaluators – colleagues, coauthors, etc. – might be biased 1 Bias F
F
2
Common research interests: “It is merely human nature that we overrate the importance of our own types of research and underrate the importance of the types that appeal to others.” (Schumpeter 1954) Personal relationships between candidates and evaluators lead to subjective evaluations
Better information F
F
Regarding research merits and prospects (research pipeline, contribution to co-authored papers, etc.) Regarding other dimensions (teaching quality, ability to raise research grants, etc.)
Two implications: I Difficulty to detect biases: connected candidates may have higher chances of success both when connected evaluators are more informed and when evaluators are biased. I Potential inefficiency of conflict of interest rules
15 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Spanish qualification evaluations Zinovyeva and Bagues (2015 AEJ:Applied)
We empirically assess the potential problem using information from national scientific qualifications (required for promotion) in Spain during 2002-2006 Several interesting features: I Large-scale: all academic disciplines and several levels of promotion (associate professor (AP) and full professor (FP)) I Variety of connections, both strong and weak (conflict of interest rules barely implemented) I Committee members are randomly assigned to evaluation committees I External validity: relatively similar to the system in place in France, Italy and several other continental european countries Procedure: 1 The Ministry publishes the call for evaluations 2 In the following 20 days candidates can apply 3 Once the list of eligible candidates is formed, evaluators (7 per committee) are selected by random draw out of the list of eligible evaluators 4 Evaluation takes place 16 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Data Zinovyeva and Bagues (2015 AEJ:Applied)
Overall 967 national committees in 188 disciplines were held during 2002-2006 I
31,750 applications
I
3,573 successful candidates
Pool of evaluators and outcome of the random draw I
29,942 eligible evaluators
I
6,769 committee members
Information: I
Gender, age, and affiliation
I
Publications
I
Participation in dissertations, either as author, advisor or committee member
I
Success in peer review evaluations for following promotions 17 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Indicators of connections Zinovyeva and Bagues (2015 AEJ:Applied)
We define several indicators of academic connections: 1
Strong ties (Co-author/PhD advisor/Colleagues) (35% in FP exams, 29% in AP exams)
2
Weak ties (34% in FP exams, 7% in AP exams) I the evaluator was a member of candidate’s doctoral committee I the evaluator invited the candidate to sit in the thesis committee of one of her students (or vice versa) I the evaluator and the candidate sat on the same thesis committee
3
Indirect ties (17% in FP exams, 14% in AP exams) I same thesis advisor I common thesis committee member I common co-author
Candidates with strong, weak and indirect ties are on average better published and more successful. 18 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Do connections matter? Zinovyeva and Bagues (2015 AEJ:Applied)
We estimate the causal effect of connections on success: yie = α + βcie + Die γ + ie where yie indicates whether individual i qualified in exam e, cie is the actual number of connections in the committee, and Die is the indicator vector for the exact number of connections that candidate i expects to have in evaluation committee e. In other words, we exploit the random variation of actual committee composition, cie , around expected committee composition, Die . The key assumption is that the selection of committee members was random. More formally, E[(cie ie |Die ] = 0
19 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Table: Random assignment
Dep. variable: Strong ties Weak ties Indirect ties
Expected ties Strong ties Weak ties Indirect ties
Expected ties
1
2
3
4
5
Publications
Total AIS
Ph.D. students
Ph.D. committees
Citations
0.010 (0.007) 0.044*** (0.008) 0.168*** (0.016)
0.009 (0.007) 0.002 (0.007) 0.131*** (0.015)
-0.017*** (0.006) 0.180*** (0.010) -0.007 (0.010)
-0.013* (0.007) 0.299*** (0.011) -0.019* (0.011)
0.016** (0.007) -0.010 (0.007) 0.105*** (0.015)
No
No
No
No
No
-0.003 (0.012) -0.008 (0.013) -0.022 (0.016)
-0.001 (0.010) -0.007 (0.012) -0.006 (0.015)
0.003 (0.010) 0.009 (0.015) 0.002 (0.012)
-0.003 (0.011) 0.013 (0.016) 0.004 (0.014)
0.002 (0.011) -0.005 (0.011) -0.020 (0.015)
Yes
Yes
Yes
Yes
Yes 20 / 28
Introduction
Monetary and non-monetary rewards
Evaluation of scientific quality
Table: The effect of connections on candidates’ success
Strong tie Weak tie Indirect tie Controls: - Publications - Total AIS - Ph.D. students advised - Ph.D. committees Individual FE
1
2
3
4
5
6
7
All
FP exams
AP exams
Graduated in Spain
Uncommon surnames
All
All
0.060*** (0.004) 0.022*** (0.005) 0.003 (0.005)
0.057*** (0.006) 0.017*** (0.006) 0.002 (0.008)
0.068*** (0.006) 0.033*** (0.011) 0.008 (0.007)
0.062*** (0.004) 0.021*** (0.006) 0.001 (0.005)
0.062*** (0.006) 0.021*** (0.007) 0.005 (0.007)
0.060*** (0.004) 0.021*** (0.005) 0.003 (0.005)
0.041*** (0.005) 0.022*** (0.006) 0.009 (0.006)
0.013*** (0.003) 0.029*** (0.003) 0.012*** (0.002) 0.015*** (0.002)
0.029*** (0.009) 0.014* (0.007) 0.024*** (0.007) 0.010 (0.007) Yes
Adj. R-squared 0.039 0.065 0.041 0.050 0.047 0.077 0.180 No. of observations 31243 13444 17799 24264 15896 31243 22292 Notes: OLS estimates. Standard errors clustered by exam in parentheses. Quality controls included in columns 6 and 7 are normalized for candidates at the exam level. Columns 6 and 7 also include indicators for candidates’ age, past experience, and the number of simultaneous applications. All columns include the exact number of expected connections of each type (775 indicators). * – p-value