Science of Science PDF Research Review - Science

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/323502497 Science of science Article in Science · March 2018 DOI: 10.1126/science.aao0185 CITATIONS...

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/323502497 Science of science Article in Science · March 2018 DOI: 10.1126/science.aao0185 CITATIONS READS 839 41,857 14 authors, including: Santo Fortunato Katy Borner Indiana University Bloomington Indiana University Bloomington 174 PUBLICATIONS 40,665 CITATIONS 413 PUBLICATIONS 12,811 CITATIONS SEE PROFILE SEE PROFILE Dirk Helbing Stasa Milojevic ETH Zurich Indiana University Bloomington 749 PUBLICATIONS 68,682 CITATIONS 75 PUBLICATIONS 3,230 CITATIONS SEE PROFILE SEE PROFILE All content following this page was uploaded by Dirk Helbing on 25 September 2018. The user has requested enhancement of the downloaded file.  R ES E A RC H ◥ the impact of team research, finding that small teams REVIEW SUMMARY tend to disrupt science and technology with new ideas drawing on older and less prevalent ones. In contrast, large teams tend to develop recent, popular SCIENCE COMMUNITY ideas, obtaining high, but often short-lived, impact. Science of science OUTLOOK: SciSci offers a deep quantitative understanding of the relational structure between scientists, institutions, and ideas because it facil- Santo Fortunato,* Carl T. Bergstrom, Katy Börner, James A. Evans, Dirk Helbing, itates the identification of fundamental mecha- Staša Milojević, Alexander M. Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, nisms responsible for scientific discovery. These Alessandro Vespignani, Ludo Waltman, Dashun Wang, Albert-László Barabási* interdisciplinary data-driven efforts complement contributions from related fields such as sciento- BACKGROUND: The increasing availability of ADVANCES: Science can be described as a com- metrics and the economics and sociology of digital data on scholarly inputs and outputs—from plex, self-organizing, and evolving network of ◥ science. Although SciSci research funding, productivity, and collaboration scholars, projects, papers, and ideas. This rep- ON OUR WEBSITE seeks long-standing univer- to paper citations and scientist mobility—offers resentation has unveiled patterns characterizing Read the full article sal laws and mechanisms unprecedented opportunities to explore the struc- the emergence of new scientific fields through at http://dx.doi. that apply across various ture and evolution of science. The science of the study of collaboration networks and the path org/10.1126/ fields of science, a funda- science (SciSci) offers a quantitative understanding of impactful discoveries through the study of science.aao0185 mental challenge going.................................................. of the interactions among scientific agents across citation networks. Microscopic models have traced forward is accounting for Downloaded from http://science.sciencemag.org/ on May 23, 2018 diverse geographic and temporal scales: It provides the dynamics of citation accumulation, allowing undeniable differences in culture, habits, and insights into the conditions underlying creativity us to predict the future impact of individual preferences between different fields and coun- and the genesis of scientific discovery, with the papers. SciSci has revealed choices and trade-offs tries. This variation makes some cross-domain ultimate goal of developing tools and policies that scientists face as they advance both their own insights difficult to appreciate and associated that have the potential to accelerate science. In careers and the scientific horizon. For example, mea- science policies difficult to implement. The differ- the past decade, SciSci has benefited from an in- surements indicate that scholars are risk-averse, ences among the questions, data, and skills specif- flux of natural, computational, and social scien- preferring to study topics related to their current ic to each discipline suggest that further insights tists who together have developed big data–based expertise, which constrains the potential of future can be gained from domain-specific SciSci studies, capabilities for empirical analysis and generative discoveries. Those willing to break this pattern which model and identify opportunities adapted modeling that capture the unfolding of science, its institutions, and its workforce. The value prop- engage in riskier careers but become more likely to make major breakthroughs. Overall, the highest- to the needs of individual research fields. ▪ osition of SciSci is that with a deeper understand- impact science is grounded in conventional combi- The list of author affiliations is available in the full article online. ing of the factors that drive successful science, we nations of prior work but features unusual *Corresponding author. Email: [email protected] (S.F.); [email protected] (A.-L.B.) can more effectively address environmental, soci- combinations. Last, as the locus of research is Cite this article as S. Fortunato et al., Science 359, eaao0185 etal, and technological problems. shifting into teams, SciSci is increasingly focused on (2018). DOI: 10.1126/science.aao0185 The complexity of science. Science can be seen as an expanding and ILLUSTRATION: NICOLE SAMAY evolving network of ideas, scholars, and papers. SciSci searches for universal and domain-specific laws underlying the structure and dynamics of science. Fortunato et al., Science 359, 1007 (2018) 2 March 2018 1 of 1 R ES E A RC H ◥ and from innovation studies, it explores and REVIEW identifies pathways through which science con- tributes to invention and economic change. SciSci relies on a broad collection of quantitative SCIENCE COMMUNITY methods, from descriptive statistics and data visualization to advanced econometric methods, network science approaches, machine-learning Science of science algorithms, mathematical analysis, and compu- ter simulation, including agent-based modeling. The value proposition of SciSci hinges on the Santo Fortunato,1,2* Carl T. Bergstrom,3 Katy Börner,2,4 James A. Evans,5 hypothesis that with a deeper understanding of Dirk Helbing,6 Staša Milojević,1 Alexander M. Petersen,7 Filippo Radicchi,1 the factors behind successful science, we can en- Roberta Sinatra,8,9,10 Brian Uzzi,11,12 Alessandro Vespignani,10,13,14 Ludo Waltman,15 hance the prospects of science as a whole to more Dashun Wang,11,12 Albert-László Barabási8,10,16* effectively address societal problems. Identifying fundamental drivers of science and developing predictive models to capture its Networks of scientists, institutions, evolution are instrumental for the design of policies that can improve the scientific enterprise— and ideas for example, through enhanced career paths for scientists, better performance evaluation for Contemporary science is a dynamical system of organizations hosting research, discovery of novel effective funding vehicles, and even undertakings driven by complex interactions identification of promising regions along the scientific frontier. The science of science uses among social structures, knowledge representa- large-scale data on the production of science to search for universal and domain-specific tions, and the natural world. Scientific knowledge patterns. Here, we review recent developments in this transdisciplinary field. is constituted by concepts and relations embodied Downloaded from http://science.sciencemag.org/ on May 23, 2018 in research papers, books, patents, software, and T other scholarly artifacts, organized into scientific he deluge of digital data on scholarly out- millions of data points pertaining to scientists disciplines and broader fields. These social, con- put offers unprecedented opportunities to and their output and capturing research from all ceptual, and material elements are connected explore patterns characterizing the struc- over the world and all branches of science. Sec- through formal and informal flows of informa- ture and evolution of science. The science ond, SciSci has benefited from an influx of and tion, ideas, research practices, tools, and samples. of science (SciSci) places the practice of collaborations among natural, computational, Science can thus be described as a complex, self- science itself under the microscope, leading to and social scientists who have developed big organizing, and constantly evolving multiscale a quantitative understanding of the genesis of data–based capabilities and enabled critical network. scientific discovery, creativity, and practice and tests of generative models that aim to capture Early studies discovered an exponential growth developing tools and policies aimed at accelerat- the unfolding of science, its institutions, and in the volume of scientific literature (2), a trend ing scientific progress. its workforce. that continues with an average doubling period The emergence of SciSci has been driven by One distinctive characteristic of this emerging of 15 years (Fig. 1). Yet, it would be naïve to two key factors. The first is data availability. In field is how it breaks down disciplinary bounda- equate the growth of the scientific literature with addition to the proprietary Web of Science (WoS), ries. SciSci integrates findings and theories from the growth of scientific ideas. Changes in the the historic first citation index (1), multiple data multiple disciplines and uses a wide range of publishing world, both technological and eco- sources are available today (Scopus, PubMed, data and methods. From scientometrics, it takes nomic, have led to increasing efficiency in the Google Scholar, Microsoft Academic, the U.S. the idea of measuring science from large-scale production of publications. Moreover, new pub- Patent and Trademark Office, and others). Some data sources; from the sociology of science, it lications in science tend to cluster in discrete of these sources are freely accessible, covering adopts theoretical concepts and social processes; areas of knowledge (3). Large-scale text analysis, Fig. 1. Growth of science. (A) Annual production of scientific articles indexed in the WoS database. (B) Growth of ideas covered by articles indexed in the WoS. This was determined by counting unique title phrases (concepts) in a fixed number of articles (4). 1 Center for Complex Networks and Systems Research, School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN 47408, USA. 2Indiana University Network Science Institute, Indiana University, Bloomington, IN 47408, USA. 3Department of Biology, University of Washington, Seattle, WA 98195-1800, USA. 4Cyberinfrastructure for Network Science Center, School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN 47408, USA. 5Department of Sociology, University of Chicago, Chicago, IL 60637, USA. 6Computational Social Science, ETH Zurich, Zurich, Switzerland. 7Ernest and Julio Gallo Management Program, School of Engineering, University of California, Merced, CA 95343, USA. 8Center for Network Science, Central European University, Budapest 1052, Hungary. 9Department of Mathematics, Central European University, Budapest 1051, Hungary. 10Institute for Network Science, Northeastern University, Boston, MA 02115, USA. 11Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA. 12Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL 60208, USA. 13Laboratory for the Modeling of Biological and Sociotechnical Systems, Northeastern University, Boston, MA 02115, USA. 14ISI Foundation, Turin 10133, Italy. 15Centre for Science and Technology Studies, Leiden University, Leiden, Netherlands. 16Center for Cancer Systems Biology, Dana-Farber Cancer Institute, Boston, MA 02115, USA. *Corresponding author. Email: [email protected] (S.F.); [email protected] (A.-L.B.) Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 1 of 7 R ES E A RC H | R E V IE W using phrases extracted from titles and abstracts the risk of failure to publish at all. Scientific Measurements show that the allocation of bio- to measure the cognitive extent of the scientific awards and accolades appear to function as medical resources in the United States is more literature, have found that the conceptual territory primary incentives to resist conservative tend- strongly correlated to previous allocations and of science expands linearly with time. In other encies and encourage betting on exploration research than to the actual burden of diseases words, whereas the number of publications grows and surprise (3). Despite the many factors shaping (18), highlighting a systemic misalignment be- exponentially, the space of ideas expands only what scientists work on next, macroscopic pat- tween biomedical needs and resources. This mis- linearly (Fig. 1) (4). terns that govern changes in research interests alignment casts doubts on the degree to which Frequently occurring words and phrases in along scientific careers are highly reproducible, funding agencies, often run by scientists embedded article titles and abstracts propagate via citation documenting a high degree of regularity under- in established paradigms, are likely to influence networks, punctuated by bursts corresponding lying scientific research and individual careers (14). the evolution of science without introducing to the emergence of new paradigms (5). By Scientists’ choice of research problems affects additional oversight, incentives, and feedback. applying network science methods to citation primarily their individual careers and the careers networks, researchers are able to identify com- of those reliant on them. Scientists’ collective Novelty munities as defined by subsets of publications choices, however, determine the direction of Analyses of publications and patents consistently that frequently cite one another (6). These com- scientific discovery more broadly (Fig. 2). Con- reveal that rare combinations in scientific dis- munities often correspond to groups of authors servative strategies (15) serve individual careers coveries and inventions tend to garner higher holding a common position regarding specific well but are less effective for science as a whole. citation rates (3). Interdisciplinary research is issues (7) or working on the same specialized Such strategies are amplified by the file drawer an emblematic recombinant process (19); hence, subtopics (8). Recent work focusing on biomedical problem (16): Negative results, at odds with the successful combination of previously discon- science has illustrated how the growth of the established hypotheses, are rarely published, nected ideas and resources that is fundamental literature reinforces these communities (9). As leading to a systemic bias in published research to interdisciplinary research often violates expecta- new papers are published, associations (hyper- and the canonization of weak and sometimes tions and leads to novel ideas with high impact Downloaded from http://science.sciencemag.org/ on May 23, 2018 edges) between scientists, chemicals, diseases, false facts (17). More risky hypotheses may have (20). Nevertheless, evidence from grant appli- and methods (“things,” which are the nodes of been tested by generations of scientists, but only cations shows that, when faced with new ideas, the network) are added. Most new links fall be- those successful enough to result in publications expert evaluators systematically give lower scores tween things only one or two steps away from are known to us. One way to alleviate this con- to truly novel (21–23) or interdisciplinary (24) re- each other, implying that when scientists choose servative trap is to urge funding agencies to pro- search proposals. new topics, they prefer things directly related actively sponsor risky projects that test truly The highest-impact science is primarily grounded to their current expertise or that of their col- unexplored hypotheses and take on special in- in conventional combinations of prior work, yet laborators. This densification suggests that the terest groups advocating for particular diseases. it simultaneously features unusual combinations existing structure of science may constrain what will be studied in the future. Densification at the boundaries of science is also a signal of transdisciplinary exploration, fusion, and innovation. A life-cycle analysis of eight fields (10) shows that successful fields undergo a process of knowledge and social uni- fication that leads to a giant connected component in the collaboration network, corresponding to a sizeable group of regular coauthors. A model in which scientists choose their collaborators through random walks on the coauthorship net- work successfully reproduces author productivity, the number of authors per discipline, and the interdisciplinarity of papers and authors (11). Problem selection How do scientists decide which research prob- lems to work on? Sociologists of science have long hypothesized that these choices are shaped Fig. 2. Choosing experiments to accelerate collective discovery. (A) The average efficiency rate by an ongoing tension between productive tradi- for global strategies to discover new, publishable chemical relationships, estimated from all tion and risky innovation (12, 13). Scientists who MEDLINE-indexed articles published in 2010. This model does not take into account differences in adhere to a research tradition in their domain the difficulty or expense of particular experiments. The efficiency of a global scientific strategy is often appear productive by publishing a steady expressed by the average number of experiments performed (vertical axis) relative to the number of stream of contributions that advance a focused new, published biochemical relationships (horizontal axis), which correspond to new connections research agenda. But a focused agenda may limit in the published network of biochemicals co-occurring in MEDLINE-indexed articles. Compared a researcher’s ability to sense and seize oppor- strategies include randomly choosing pairs of biochemicals, the global (“actual”) strategy inferred tunities for staking out new ideas that are re- from all scientists publishing MEDLINE articles, and optimal strategies for discovering 50 and quired to grow the field’s knowledge. For example, 100% of the network. Lower values on the vertical axis indicate more efficient strategies, showing a case study focusing on biomedical scientists that the actual strategy of science is suboptimal for discovering what has been published. The choosing novel chemicals and chemical relation- actual strategy is best for uncovering 13% of the chemical network, and the 50% optimal strategy is ships shows that as fields mature, researchers most efficient for discovering 50% of it, but neither are as good as the 100% optimal strategy for tend to focus increasingly on established knowl- revealing the whole network. (B) The actual, estimated search process illustrated on a hypothetical edge (3). Although an innovative publication tends network of chemical relationships, averaged from 500 simulated runs of that strategy. The strategy to result in higher impact than a conservative one, swarms around a few “important,” highly connected chemicals, whereas optimal strategies are much high-risk innovation strategies are rare, because more even and less likely to “follow the crowd” in their search across the space of scientific the additional reward does not compensate for possibilities. [Adapted from (15)] Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 2 of 7 R ES E A RC H | R E V IE W (25–27). Papers of this type are twice as likely to according to their citation scores, which may previous output, markedly boosts the num- receive high citations (26). In other words, a be rooted in a selection bias that offers better ber of citations collected by that paper in the balanced mixture of new and established ele- career opportunities to better scientists (43, 44). first years after publication (47). After this ments is the safest path toward successful re- Moreover, scientists tend to move between in- initial phase, however, impact depends on the ception of scientific advances. stitutions of similar prestige (45). Nevertheless, reception of the work by the scientific com- when examining changes in impact associated munity. This finding, along with the work re- Career dynamics with each move as quantified by citations, no ported in (46), suggests that, for productive Individual academic careers unfold in the con- systematic increase or decrease was found, not scientific careers, reputation is less of a critical text of a vast market for knowledge production even when scientists moved to an institution of driver for success than talent, hard work, and and consumption (28). Consequently, scientific considerably higher or lower rank (46). In other relevance. careers have been examined not only in terms words, it is not the institution that creates the A policy-relevant question is whether creativity of individual incentives and marginal productivity impact; it is the individual researchers that make and innovation depend on age or career stage. (i.e., relative gain versus effort) (29), but also an institution. Decades of research on outstanding researchers institutional incentives (30, 31) and competition Another potentially important career factor and innovators concluded that major break- (32). This requires combining large repositories is reputation—and the dilemma that it poses throughs take place relatively early in a career, of high-resolution individual, geographic, and for manuscript review, proposal evaluation, and with a median age of 35 (48). In contrast, recent temporal metadata (33) to construct represen- promotion decisions. The reputation of paper work shows that this well-documented propen- tations of career trajectories that can be ana- authors, measured by the total citations of their sity of early-career discoveries is fully explained lyzed from different perspectives. For example, by productivity, which is high in the early stages one study finds that funding schemes that are of a scientist’s career and drops later (49). In tolerant of early failure, which reward long-term other words, there are no age patterns in in- success, are more likely to generate high-impact novation: A scholar’s most cited paper can be any Downloaded from http://science.sciencemag.org/ on May 23, 2018 publications than grants subject to short review of his or her papers, independently of the age or cycles (31). Interacting systems with competing career stage when it is published (Fig. 3). A time scales are a classic problem in complex sys- stochastic model of impact evolution also indi- tems science. The multifaceted nature of science cates that breakthroughs result from a combina- is motivation for generative models that high- tion of the ability of a scientist and the luck of light unintended consequences of policies. For picking a problem with high potential (49). example, models of career growth show that non- tenure (short-term) contracts are responsible Team science for productivity fluctuations, which often result During past decades, reliance on teamwork has in a sudden career death (29). increased, representing a fundamental shift in Gender inequality in science remains preva- the way that science is done. A study of the lent and problematic (34). Women have fewer authorship of 19.9 million research articles and publications (35–37) and collaborators (38) and 2.1 million patents reveals a nearly universal less funding (39), and they are penalized in hiring shift toward teams in all branches of science decisions when compared with equally qualified (50) (Fig. 4). For example, in 1955, science and men (40). The causes of these gaps are still un- engineering teams authored about the same clear. Intrinsic differences in productivity rates number of papers as single authors. Yet by 2013, and career length can explain the differences the fraction of team-authored papers increased in collaboration patterns (38) and hiring rates to 90% (51). (35) between male and female scientists. On the Nowadays, a team-authored paper in science other hand, experimental evidence shows that and engineering is 6.3 times more likely to re- biases against women occur at very early career ceive 1000 citations or more than a solo-authored stages. When gender was randomly assigned paper, a difference that cannot be explained by among the curricula vitae of a pool of applicants, self-citations (50, 52). One possible reason is a the hiring committee systematically penalized team's ability to come up with more novel com- female candidates (40). Most studies so far have binations of ideas (26) or to produce resources focused on relatively small samples. Improvements that are later used by others (e.g., genomics). in compiling large-scale data sets on scientific Measurements show that teams are 38% more careers, which leverage information from differ- likely than solo authors to insert novel combina- ent sources (e.g., publication records, grant ap- Fig. 3. Impact in scientific careers. (A) Publica- tions into familiar knowledge domains, support- plications, and awards), will help us gain deeper tion record of three Nobel laureates in physics. ing the premise that teams can bring together insight into the causes of inequality and motivate The horizontal axis indicates the number of years different specialties, effectively combining knowl- models that can inform policy solutions. after a laureate’s first publication, each circle edge to prompt scientific breakthroughs. Having Scientists’ mobility is another important factor corresponds to a research paper, and the height more collaborations means greater visibility offering diverse career opportunities. Most mo- of the circle represents the paper’s impact, through a larger number of coauthors, who will bility studies have focused on quantifying the quantified by c10, the number of citations likely introduce the work to their networks, an brain drain and gain of a country or a region after 10 years. The highest-impact paper of a enhanced impact that may partially compensate (41, 42), especially after policy changes. Research laureate is denoted with an orange circle. for the fact that credit within a team must be on individual mobility and its career effect re- (B) Histogram of the occurrence of the highest- shared with many colleagues (29). mains scant, however, primarily owing to the impact paper in a scientist’s sequence of Work from large teams garners, on average, difficulty of obtaining longitudinal information publications, calculated for 10,000 scientists. The more citations across a wide variety of domains. about the movements of many scientists and flatness of the histogram indicates that the Research suggests that small teams tend to dis- accounts of the reasons underlying mobility de- highest-impact work can be, with the same rupt science and technology with new ideas and cisions. Scientists who left their country of origin probability, anywhere in the sequence of papers opportunities, whereas large teams develop ex- outperformed scientists who did not relocate, published by a scientist (49). isting ones (53). Thus, it may be important to Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 3 of 7 R ES E A RC H | R E V IE W fund and foster teams of all sizes to temper the bureaucratization of science (28). Teams are growing in size, increasing by an average of 17% per decade (50, 54), a trend under- lying a fundamental change in team composi- tions. Scientific teams include both small, stable “core” teams and large, dynamically changing extended teams (55). The increasing team size in most fields is driven by faster expansion of ex- tended teams, which begin as small core teams but subsequently attract new members through a process of cumulative advantage anchored by productivity. Size is a crucial determinant of team Fig. 4. Size and impact of teams. Mean team size has been steadily growing over the past century. survival strategies: Small teams survive longer The red dashed curves represent the mean number of coauthors over all papers; the black curves if they maintain a stable core, but larger teams consider just those papers receiving more citations than the average for the field. Black curves are persist longer if they manifest a mechanism for systematically above the dashed red ones, meaning that high-impact work is more likely to be membership turnover (56). produced by large teams than by small ones. Each panel corresponds to one of the three main As science has accelerated and grown increas- disciplinary groups of papers indexed in the WoS: (A) science and engineering, (B) social sciences, ingly complex, the instruments required to ex- and (C) arts and humanities. pand the frontier of knowledge have increased in scale and precision. The tools of the trade become unaffordable to most individual inves- seminal papers can accumulate 10,000 or more features of citation dynamics, such as the obso- Downloaded from http://science.sciencemag.org/ on May 23, 2018 tigators, but also to most institutions. Collabora- citations. This uneven citation distribution is a lescence of knowledge, decreasing the citation tion has been a critical solution, pooling resources robust, emergent property of the dynamics of probability with the age of the paper (76–79), to scientific advantage. The Large Hadron Collider science, and it holds when papers are grouped and a fitness parameter, unique to each paper, at CERN, the world’s largest and most power- by institution (68). If the number of citations of capturing the appeal of the work to the scientific ful particle collider, would have been unthink- a paper is divided by the average number of community (77, 78). Only a tiny fraction of papers able without collaboration, requiring more than citations collected by papers in the same dis- deviate from the pattern described by such a 10,000 scientists and engineers from more than cipline and year, the distribution of the result- model—some of which are called “sleeping beau- 100 countries. There is, however, a trade-off with ing score is essentially indistinguishable for all ties,” because they receive very little notice for increasing size that affects the value and risk disciplines (69, 70) (Fig. 5A). This means that decades after publication and then suddenly re- associated with “big science” (2). Although it may we can compare the impact of papers published ceive a burst of attention and citations (80, 81). be possible to solve larger problems, the burden in different disciplines by looking at their relative The generative mechanisms described above of reproducibility may require duplicating initial citation values. For example, a paper in mathe- can be used to predict the citation dynamics of efforts, which may not be practically or econom- matics collecting 100 citations represents a higher individual papers. One predictive model (77) as- ically feasible. disciplinary impact than a paper in microbiol- sumes that the citation probability of a paper Collaborators can have a large effect on scien- ogy with 300 citations. depends on the number of previous citations, tific careers. According to recent studies (57, 58), The tail of the citation distribution, capturing an obsolescence factor, and a fitness parameter scientists who lose their star collaborators ex- the number of high-impact papers, sheds light (Fig. 5, B and C). For a given paper, one can es- perience a substantial drop in their productivity, on the mechanisms that drive the accumulation timate the three model parameters by fitting the especially if the lost collaborator was a regular of citations. Recent analyses show that it follows model to the initial portion of the citation history coauthor. Publications involving extremely strong a power law (71–73). Power-law tails can be gen- of the paper. The long-term impact of the work collaborators gain 17% more citations on average, erated through a cumulative advantage process can be extrapolated (77). Other studies have iden- pointing to the value of career partnership (59). (74), known as preferential attachment in net- tified predictors of the citation impact of indi- Given the increasing number of authors on work science (75), suggesting that the probability vidual papers (82), such as journal impact factor the average research paper, who should and does of citing a paper grows with the number of cita- (72). It has been suggested that the future h-index gain the most credit? The canonical theory of tions that it has already collected. Such a mod- (83) of a scientist can be accurately predicted (84), credit (mis)allocation in science is the Matthew el can be augmented with other characteristic although the predictive power is reduced when effect (60), in which scientists of higher statuses involved in joint work receive outsized credit for their contributions. Properly allocating individual credit for a collaborative work is difficult because Box 1. Lessons from SciSci. we cannot easily distinguish individual contribu- tions (61). It is possible, however, to inspect the co- 1. Innovation and tradition: Left bare, truly innovative and highly interdisciplinary ideas may citation patterns of the coauthors’ publications to not reach maximum scientific impact. To enhance their impact, novel ideas should be placed in determine the fraction of credit that the commu- the context of established knowledge (26). nity assigns to each coauthor in a publication (62). 2. Persistence: A scientist is never too old to make a major discovery, as long as he or she stays productive (49). Citation dynamics 3. Collaboration: Research is shifting to teams, so engaging in collaboration is beneficial. Scholarly citation remains the dominant mea- Works by small teams tend to be more disruptive, whereas those by big teams tend to have surable unit of credit in science. Given the re- more impact (4, 50, 53). liance of most impact metrics on citations (63–66), 4. Credit: Most credit will go to the coauthors with the most consistent track record in the the dynamics of citation accumulation have been domain of the publication (62). scrutinized by generations of scholars. From foun- 5. Funding: Although review panels acknowledge innovation, they ultimately tend to dational work by Price (67), we know that the discount it. Funding agencies should ask reviewers to assess innovation, not only expected distribution of citations for scientific papers is success (24). highly skewed: Many papers are never cited, but Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 4 of 7 R ES E A RC H | R E V IE W related research domains such as the economics (30) and sociology of science (60, 86). Causal estimation is a prime example, in which econ- ometric matching techniques demand and lever- age comprehensive data sources in the effort to simulate counterfactual scenarios (31, 42). Assess- ing causality is one of the most needed future developments in SciSci: Many descriptive studies reveal strong associations between structure and outcomes, but the extent to which a specific struc- ture “causes” an outcome remains unexplored. Engaging in tighter partnerships with exper- imentalists, SciSci will be able to better identify associations discovered from models and large- scale data that have causal force to enrich their policy relevance. But experimenting on science may be the biggest challenge SciSci has yet to face. Running randomized, controlled trials that can alter outcomes for individuals or institutions of science, which are mostly supported by tax dollars, is bound to elicit criticisms and pushback (87). Hence, we expect quasi-experimental ap- Downloaded from http://science.sciencemag.org/ on May 23, 2018 proaches to prevail in SciSci investigations in the near future. Most SciSci research focuses on publications as primary data sources, implying that insights and findings are limited to ideas successful enough to merit publication in the first place. Yet most scientific attempts fail, sometimes spectacularly. Given that scientists fail more often than they succeed, knowing when, why, and how an idea fails is essential in our attempts to understand and improve science. Such studies could provide meaningful guidance regarding the reproducibility crisis and help us account for the file drawer problem. They could also substantially further our understanding of human imagination by revealing the total pipeline of creative activity. Science often behaves like an economic sys- tem with a one-dimensional “currency” of cita- Fig. 5. Universality in citation dynamics. (A) The citation distributions of papers published in tion counts. This creates a hierarchical system, the same discipline and year lie on the same curve for most disciplines, if the raw number of citations in which the “rich-get-richer” dynamics suppress c of each paper is divided by the average number of citations c0 over all papers in that discipline the spread of new ideas, particularly those from and year. The dashed line is a lognormal fit. [Adapted from (69)] (B) Citation history of four papers junior scientists and those who do not fit within published in Physical Review in 1964, selected for their distinct dynamics, displaying a “jump-decay” the paradigms supported by specific fields. Science pattern (blue), experiencing a delayed peak (magenta), attracting a constant number of citations can be improved by broadening the number over time (green), or acquiring an increasing number of citations each year (red). (C) Citations and range of performance indicators. The develop- of an individual paper are determined by three parameters: fitness li, immediacy mi, and longevity ment of alternative metrics covering web (88) si. By rescaling the citation history of each paper in (B) by the appropriate (l, m, s) parameters, and social media (89) activity and societal im- the four papers collapse onto a single universal function, which is the same for all disciplines. pact (90) is critical in this regard. Other mea- [Adapted from (77)] surable dimensions include the information (e.g., data) that scientists share with competitors (91), the help that they offer to their peers (92), and accounting for the scientist’s career stage and plement. The differences among the questions, their reliability as reviewers of their peers’ works the cumulative, nondecreasing nature of the data, and skills required by each discipline suggest (93). But with a profusion of metrics, more work h-index (85). Eliminating inconsistencies in the that we may gain further insights from domain- is needed to understand what each of them does use of quantitative evaluation metrics in science specific SciSci studies that model and predict and does not capture to ensure meaningful in- is crucial and highlights the importance of un- opportunities adapted to the needs of each field. terpretation and avoid misuse. SciSci can make derstanding the generating mechanisms behind For young scientists, the results of SciSci offer an essential contribution by providing models commonly used statistics. actionable insights about past patterns, helping that offer a deeper understanding of the mech- guide future inquiry within their disciplines (Box 1). anisms that govern performance indicators in Outlook The contribution of SciSci is a detailed under- science. For instance, models of the empirical Despite the discovery of universals across science, standing of the relational structure between patterns observed when alternative indicators substantial disciplinary differences in culture, scientists, institutions, and ideas, a crucial starting (e.g., distributions of paper downloads) are used habits, and preferences make some cross-domain point that facilitates the identification of funda- will enable us to explore their relationship insights difficult to appreciate within particular mental generating processes. Together, these data- with citation-based metrics (94) and to recognize fields and associated policies challenging to im- driven efforts complement contributions from manipulations. Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 5 of 7 R ES E A RC H | R E V IE W The integration of citation-based metrics with 7. U. Shwed, P. S. Bearman, The temporal structure of scientific 31. P. Azoulay, J. S. Graff Zivin, G. Manso, Incentives and alternative indicators will promote pluralism consensus formation. Am. Sociol. Rev. 75, 817–840 (2010). creativity: Evidence from the academic life sciences. doi: 10.1177/0003122410388488; pmid: 21886269 Rand J. Econ. 42, 527–554 (2011). doi: 10.1111/ and enable new dimensions of productive special- 8. J. Bruggeman, V. A. Traag, J. Uitermark, Detecting j.1756-2171.2011.00140.x ization, in which scientists can be successful in communities through network data. Am. Sociol. Rev. 77, 32. R. Freeman, E. Weinstein, E. Marincola, J. Rosenbaum, different ways. Science is an ecosystem that re- 1050–1063 (2012). doi: 10.1177/0003122412463574 F. Solomon, Competition and careers in biosciences. Science quires not only publications, but also communi- 9. F. Shi, J. G. Foster, J. A. Evans, Weaving the fabric of science: 294, 2293–2294 (2001). doi: 10.1126/science.1067477; Dynamic network models of science’s unfolding structure. pmid: 11743184 cators, teachers, and detail-oriented experts. We Soc. Networks 43, 73–85 (2015). doi: 10.1016/ 33. J. A. Evans, J. G. Foster, Metaknowledge. Science need individuals who can ask novel, field-altering j.socnet.2015.02.006 331, 721–725 (2011). doi: 10.1126/science.1201765; questions, as well as those who can answer them. 10. L. M. A. Bettencourt, D. I. Kaiser, J. Kaur, Scientific discovery pmid: 21311014 It would benefit science if curiosity, creativity, and topological transitions in collaboration networks. 34. V. Larivière, C. Ni, Y. Gingras, B. Cronin, C. R. Sugimoto, J. Informetr. 3, 210–221 (2009). doi: 10.1016/ Bibliometrics: Global gender disparities in science. and intellectual exchange—particularly regard- j.joi.2009.03.001 Nature 504, 211–213 (2013). doi: 10.1038/504211a; ing the societal implications and applications of 11. X. Sun, J. Kaur, S. Milojević, A. Flammini, F. Menczer, pmid: 24350369 science and technology—are better appreciated Social dynamics of science. Sci. Rep. 3, 1069 (2013). 35. S. F. Way, D. B. Larremore, A. Clauset, in Proceedings of and incentivized in the future. A more pluralistic doi: 10.1038/srep01069; pmid: 23323212 the 25th International Conference on World Wide Web 12. T. S. Kuhn, The Essential Tension: Selected Studies in (WWW ‘16) (ACM, 2016), pp. 1169–1179. approach could reduce duplication and make Scientific Tradition and Change (Univ. of Chicago Press, 1977). 36. J. Duch et al., The possible role of resource requirements and science flourish for society (95). 13. P. Bourdieu, The specificity of the scientific field and academic career-choice risk on gender differences in An issue that SciSci seeks to address is the the social conditions of the progress of reasons. publication rate and impact. PLOS ONE 7, e51332 (2012). allocation of science funding. The current peer Soc. Sci. Inf. (Paris) 14, 19–47 (1975). doi: 10.1177/ doi: 10.1371/journal.pone.0051332; pmid: 23251502 053901847501400602 37. J. D. West, J. Jacquet, M. M. King, S. J. Correll, C. T. Bergstrom, review system is subject to biases and inconsisten- 14. T. Jia, D. Wang, B. K. Szymanski, Quantifying patterns of The role of gender in scholarly authorship. PLOS ONE 8, cies (96). Several alternatives have been proposed, research-interest evolution. Nat. Hum. Behav. 1, 0078 (2017). e66212 (2013). doi: 10.1371/journal.pone.0066212; such as the random distribution of funding (97), doi: 10.1038/s41562-017-0078 pmid: 23894278 person-directed funding that does not involve 15. A. Rzhetsky, J. G. Foster, I. T. Foster, J. A. Evans, Choosing 38. X. H. T. Zeng et al., Differences in collaboration patterns Downloaded from http://science.sciencemag.org/ on May 23, 2018 experiments to accelerate collective discovery. Proc. Natl. across discipline, career stage, and gender. PLOS Biol. proposal preparation and review (31), opening Acad. Sci. U.S.A. 112, 14569–14574 (2015). 14, e1002573 (2016). doi: 10.1371/journal.pbio.1002573; the proposal review process to the entire online doi: 10.1073/pnas.1509757112; pmid: 26554009 pmid: 27814355 population (98), removing human reviewers 16. R. Rosenthal, The file drawer problem and tolerance for null 39. T. J. Ley, B. H. Hamilton, The gender gap in NIH grant altogether by allocating funds through a per- results. Psychol. Bull. 86, 638–641 (1979). doi: 10.1037/ applications. Science 322, 1472–1474 (2008). doi: 10.1126/ 0033-2909.86.3.638 science.1165878; pmid: 19056961 formance measure (99), and scientist crowd- 17. S. B. Nissen, T. Magidson, K. Gross, C. T. Bergstrom, 40. C. A. Moss-Racusin, J. F. Dovidio, V. L. Brescoll, M. J. Graham, funding (100). Publication bias and the canonization of false facts. eLife 5, J. Handelsman, Science faculty’s subtle gender biases favor A critical area of future research for SciSci e21451 (2016). doi: 10.7554/eLife.21451; pmid: 27995896 male students. Proc. Natl. Acad. Sci. U.S.A. 109, 16474–16479 concerns the integration of machine learning 18. L. Yao, Y. Li, S. Ghosh, J. A. Evans, A. Rzhetsky, Health ROI as (2012). doi: 10.1073/pnas.1211286109; pmid: 22988126 a measure of misalignment of biomedical needs and 41. R. Van Noorden, Global mobility: Science on the move. and artificial intelligence in a way that involves resources. Nat. Biotechnol. 33, 807–811 (2015). doi: 10.1038/ Nature 490, 326–329 (2012). doi: 10.1038/490326a; machines and minds working together. These nbt.3276; pmid: 26252133 pmid: 23075963 new tools portend far-reaching implications 19. C. S. Wagner et al., Approaches to understanding and 42. O. A. Doria Arrieta, F. Pammolli, A. M. Petersen, Quantifying for science because machines might broaden a measuring interdisciplinary scientific research (IDR): A review the negative impact of brain drain on the integration of of the literature. J. Informetr. 5, 14–26 (2011). doi: 10.1016/ European science. Sci. Adv. 3, e1602232 (2017). doi: 10.1126/ scientist’s perspective more than human col- j.joi.2010.06.004 sciadv.1602232; pmid: 28439544 laborators. For instance, the self-driving vehi- 20. V. Larivière, S. Haustein, K. Börner, Long-distance 43. C. Franzoni, G. Scellato, P. Stephan, The mover’s advantage: cle is the result of a successful combination of interdisciplinarity leads to higher scientific impact. PLOS ONE The superior performance of migrant scientists. Econ. Lett. known driving habits and information that 10, e0122565 (2015). doi: 10.1371/journal.pone.0122565; 122, 89–93 (2014). doi: 10.1016/j.econlet.2013.10.040 pmid: 25822658 44. C. R. Sugimoto et al., Scientists have most impact when was outside of human awareness, provided by they’re free to move. Nature 550, 29–31 (2017). 21. K. J. Boudreau, E. C. Guinan, K. R. Lakhani, C. Riedl, Looking sophisticated machine-learning techniques. Mind- across and looking beyond the knowledge frontier: doi: 10.1038/550029a; pmid: 28980663 machine partnerships have improved evidence- Intellectual distance, novelty, and resource allocation in 45. A. Clauset, S. Arbesman, D. B. Larremore, Systematic based decision-making in a wide range of health, science. Manage. Sci. 62, 2765–2783 (2016). doi: 10.1287/ inequality and hierarchy in faculty hiring networks. Sci. Adv. mnsc.2015.2285; pmid: 27746512 1, e1400005 (2015). doi: 10.1126/sciadv.1400005; economic, social, legal, and business problems 22. E. Leahey, J. Moody, Sociological innovation through subfield pmid: 26601125 (101–103). How can science be improved with integration. Soc. Currents 1, 228–256 (2014). doi: 10.1177/ 46. P. Deville et al., Career on the move: Geography, mind-machine partnerships, and what arrange- 2329496514540131 stratification, and scientific impact. Sci. Rep. 4, 4770 ments are most productive? These questions 23. A. Yegros-Yegros, I. Rafols, P. D’Este, Does interdisciplinary (2014). pmid: 24759743 promise to help us understand the science of research lead to higher citation impact? The different 47. A. M. Petersen et al., Reputation and impact in academic effect of proximal and distal interdisciplinarity. PLOS ONE careers. Proc. Natl. Acad. Sci. U.S.A. 111, 15316–15321 the future. 10, e0135095 (2015). doi: 10.1371/journal.pone.0135095; (2014). doi: 10.1073/pnas.1323111111; pmid: 25288774 pmid: 26266805 48. D. K. Simonton, Creative productivity: A predictive and 24. L. Bromham, R. Dinnage, X. Hua, Interdisciplinary research explanatory model of career trajectories and landmarks. RE FE RENCES AND N OT ES has consistently lower funding success. Nature 534, Psychol. Rev. 104, 66–89 (1997). doi: 10.1037/ 1. E. Garfield, Citation indexes for science; a new dimension 684–687 (2016). doi: 10.1038/nature18315; pmid: 27357795 0033-295X.104.1.66 in documentation through association of ideas. Science 122, 25. D. Kim, D. B. Cerigo, H. Jeong, H. Youn, Technological novelty 49. R. Sinatra, D. Wang, P. Deville, C. Song, A.-L. Barabási, 108–111 (1955). doi: 10.1126/science.122.3159.108; profile and inventions future impact. EPJ Data Sci. 5, 8 Quantifying the evolution of individual scientific impact. pmid: 14385826 (2016). doi: 10.1140/epjds/s13688-016-0069-1 Science 354, aaf5239 (2016). doi: 10.1126/science.aaf5239; 2. D. J. S. Price, Little Science, Big Science (Columbia Univ. 26. B. Uzzi, S. Mukherjee, M. Stringer, B. Jones, Atypical pmid: 27811240 Press, 1963). combinations and scientific impact. Science 342, 468–472 50. S. Wuchty, B. F. Jones, B. Uzzi, The increasing dominance 3. J. G. Foster, A. Rzhetsky, J. A. Evans, Tradition and (2013). doi: 10.1126/science.1240474; pmid: 24159044 of teams in production of knowledge. Science 316, innovation in scientists’ research strategies. 27. J. Wang, R. Veugelers, P. Stephan, “Bias against novelty in 1036–1039 (2007). doi: 10.1126/science.1136099; Am. Sociol. Rev. 80, 875–908 (2015). doi: 10.1177/ science: A cautionary tale for users of bibliometric pmid: 17431139 0003122415601618 indicators” (NBER Working Paper No. 22180, National Bureau 51. N. J. Cooke, M. L. Hilton, Eds., Enhancing the Effectiveness of 4. S. Milojević, Quantifying the cognitive extent of science. of Economic Research, 2016). Team Science (National Academies Press, 2015). J. Informetr. 9, 962–973 (2015). doi: 10.1016/ 28. J. P. Walsh, Y.-N. Lee, The bureaucratization of science. Res. 52. V. Larivière, Y. Gingras, C. R. Sugimoto, A. Tsou, Team size j.joi.2015.10.005 Policy 44, 1584–1600 (2015). doi: 10.1016/ matters: Collaboration and scientific impact since 1900. 5. T. Kuhn, M. Perc, D. Helbing, Inheritance patterns in citation j.respol.2015.04.010 J. Assoc. Inf. Sci. Technol. 66, 1323–1332 (2015). networks reveal scientific memes. Phys. Rev. X 4, 041036 29. A. M. Petersen, M. Riccaboni, H. E. Stanley, F. Pammolli, doi: 10.1002/asi.23266 (2014). doi: 10.1103/PhysRevX.4.041036 Persistence and uncertainty in the academic career. 53. L. Wu, D. Wang, J. A. Evans, Large teams have developed 6. R. Klavans, K. W. Boyack, Which type of citation analysis Proc. Natl. Acad. Sci. U.S.A. 109, 5213–5218 (2012). science and technology; small teams have disrupted it. generates the most accurate taxonomy of scientific and doi: 10.1073/pnas.1121429109; pmid: 22431620 arXiv:1709.02445 [physics.soc-ph] (7 September 2017). technical knowledge? J. Assoc. Inf. Sci. Technol. 68, 984–998 30. P. E. Stephan, How Economics Shapes Science (Harvard Univ. 54. B. F. Jones, The burden of knowledge and the “death (2016). doi: 10.1002/asi.23734 Press, 2012). of the renaissance man”: Is innovation getting harder? Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 6 of 7 R ES E A RC H | R E V IE W Rev. Econ. Stud. 76, 283–317 (2009). doi: 10.1111/ 74. D. de Solla Price, A general theory of bibliometric and other 93. S. Ravindran, “Getting credit for peer review,” Science, 8 j.1467-937X.2008.00531.x cumulative advantage processes. J. Am. Soc. Inf. Sci. 27, February 2016; www.sciencemag.org/careers/2016/02/ 55. S. Milojević, Principles of scientific research team formation 292–306 (1976). doi: 10.1002/asi.4630270505 getting-credit-peer-review. and evolution. Proc. Natl. Acad. Sci. U.S.A. 111, 3984–3989 75. A.-L. Barabási, R. Albert, Emergence of scaling in random 94. R. Costas, Z. Zahedi, P. Wouters, Do “altmetrics” correlate (2014). doi: 10.1073/pnas.1309723111; pmid: 24591626 networks. Science 286, 509–512 (1999). doi: 10.1126/ with citations? Extensive comparison of altmetric 56. G. Palla, A.-L. Barabási, T. Vicsek, Quantifying social group science.286.5439.509; pmid: 10521342 indicators with citations from a multidisciplinary perspective. evolution. Nature 446, 664–667 (2007). doi: 10.1038/ 76. P. D. B. Parolo et al., Attention decay in science. J. Informetr. J. Assoc. Inf. Sci. Technol. 66, 2003–2019 (2015). nature05670; pmid: 17410175 9, 734–745 (2015). doi: 10.1016/j.joi.2015.07.006 doi: 10.1002/asi.23309 57. G. J. Borjas, K. B. Doran, Which peers matter? The relative 77. D. Wang, C. Song, A.-L. Barabási, Quantifying long-term 95. A. Clauset, D. B. Larremore, R. Sinatra, Data-driven impacts of collaborators, colleagues, and competitors. scientific impact. Science 342, 127–132 (2013). doi: 10.1126/ predictions in the science of science. Science 355, 477–480 Rev. Econ. Stat. 97, 1104–1117 (2015). doi: 10.1162/ science.1237825; pmid: 24092745 (2017). doi: 10.1126/science.aal4217 REST_a_00472 96. S. Wessely, Peer review of grant applications: What 78. Y.-H. Eom, S. Fortunato, Characterizing and modeling 58. P. Azoulay, J. G. Zivin, J. Wang, Superstar extinction. Q. J. Econ. do we know? Lancet 352, 301–305 (1998). doi: 10.1016/ citation dynamics. PLOS ONE 6, e24926 (2011). doi: 10.1371/ 125, 549–589 (2010). doi: 10.1162/qjec.2010.125.2.549 S0140-6736(97)11129-1; pmid: 9690424 journal.pone.0024926; pmid: 21966387 59. A. M. Petersen, Quantifying the impact of weak, strong, and 97. N. Geard, J. Noble, paper presented at the 3rd World 79. M. Golosovsky, S. Solomon, Stochastic dynamical model of a super ties in scientific careers. Proc. Natl. Acad. Sci. U.S.A. Congress on Social Simulation, Kassel, Germany, 6 to 9 growing citation network based on a self-exciting point 112, E4671–E4680 (2015). doi: 10.1073/pnas.1501444112; September 2010. process. Phys. Rev. Lett. 109, 098701 (2012). doi: 10.1103/ pmid: 26261301 98. Calm in a crisis. Nature 468, 1002 (2010). doi: 10.1038/ 60. R. K. Merton, The Matthew effect in science. Science 159, PhysRevLett.109.098701; pmid: 23002894 80. A. F. J. van Raan, Sleeping Beauties in science. 4681002a; pmid: 21170024 56–63 (1968). doi: 10.1126/science.159.3810.56 Scientometrics 59, 467–472 (2004). doi: 10.1023/B: 99. R. Roy, Funding science: The real defects of peer review and 61. L. Allen, J. Scott, A. Brand, M. Hlava, M. Altman, Publishing: SCIE.0000018543.82441.f1 an alternative to it. Sci. Technol. Human Values 10, 73–81 Credit where credit is due. Nature 508, 312–313 (2014). 81. Q. Ke, E. Ferrara, F. Radicchi, A. Flammini, Defining and (1985). doi: 10.1177/016224398501000309 doi: 10.1038/508312a; pmid: 24745070 62. H.-W. Shen, A.-L. Barabási, Collective credit allocation in identifying Sleeping Beauties in science. Proc. Natl. Acad. 100. J. Bollen, D. Crandall, D. Junk, Y. Ding, K. Börner, An efficient science. Proc. Natl. Acad. Sci. U.S.A. 111, 12325–12330 Sci. U.S.A. 112, 7426–7431 (2015). doi: 10.1073/ system to fund science: From proposal review to peer-to- (2014). doi: 10.1073/pnas.1401992111; pmid: 25114238 pnas.1424329112; pmid: 26015563 peer distributions. Scientometrics 110, 521–528 (2017). 63. L. Waltman, A review of the literature on citation impact 82. I. Tahamtan, A. Safipour Afshar, K. Ahamdzadeh, Factors doi: 10.1007/s11192-016-2110-3 affecting number of citations: A comprehensive review of the Downloaded from http://science.sciencemag.org/ on May 23, 2018 indicators. J. Informetr. 10, 365–391 (2016). doi: 10.1016/ 101. M. S. Kohn et al., IBM’s health analytics and clinical decision j.joi.2016.02.007 literature. Scientometrics 107, 1195–1225 (2016). support. Yearb. Med. Inform. 9, 154–162 (2014). 64. J. E. Hirsch, An index to quantify an individual’s scientific doi: 10.1007/s11192-016-1889-2 doi: 10.15265/IY-2014-0002; pmid: 25123736 research output. Proc. Natl. Acad. Sci. U.S.A. 102, 83. J. E. Hirsch, Does the h index have predictive power? 102. J. Kleinberg, H. Lakkaraju, J. Leskovec, J. Ludwig, 16569–16572 (2005). doi: 10.1073/pnas.0507655102; Proc. Natl. Acad. Sci. U.S.A. 104, 19193–19198 (2007). S. Mullainathan, “Human decisions and machine predictions” pmid: 16275915 doi: 10.1073/pnas.0707962104; pmid: 18040045 (National Bureau of Economic Research, 2017). 65. H. F. Moed, Citation Analysis in Research Evaluation (Springer, 2010). 84. D. E. Acuna, S. Allesina, K. P. Kording, Future impact: 103. B. Liu, R. Govindan, B. Uzzi, Do emotions expressed 66. E. Garfield, Citation analysis as a tool in journal evaluation. Predicting scientific success. Nature 489, 201–202 (2012). online correlate with actual changes in decision-making?: Science 178, 471–479 (1972). doi: 10.1126/ doi: 10.1038/489201a; pmid: 22972278 The case of stock day traders. PLOS ONE 11, science.178.4060.471; pmid: 5079701 85. O. Penner, R. K. Pan, A. M. Petersen, K. Kaski, S. Fortunato, e0144945 (2016). doi: 10.1371/journal.pone.0144945; 67. D. J. de Solla Price, Networks of scientific papers. Science On the predictability of future impact in science. pmid: 26765539 149, 510–515 (1965). doi: 10.1126/science.149.3683.510; Sci. Rep. 3, 3052 (2013). doi: 10.1038/srep03052; pmid: 14325149 pmid: 24165898 AC KNOWLED GME NTS 68. Q. Zhang, N. Perra, B. Gonçalves, F. Ciulla, A. Vespignani, 86. J. R. Cole, H. Zuckerman, in The Idea of Social Structure: Papers in Honor of Robert K. Merton, L. A. Coser, Ed. This work was supported by Air Force Office of Scientific Characterizing scientific production and consumption in (Harcourt Brace Jovanovich, 1975), pp. 139–174. Research grants FA9550-15-1-0077 (A.-L.B., R.S., and A.V.), physics. Sci. Rep. 3, 1640 (2013). doi: 10.1038/srep01640; 87. P. Azoulay, Research efficiency: Turn the scientific method on FA9550-15-1-0364 (A.-L.B. and R.S.), FA9550-15-1-0162 pmid: 23571320 ourselves. Nature 484, 31–32 (2012). doi: 10.1038/484031a; (J.A.E. and D.W.), and FA9550-17-1-0089 (D.W.); National 69. F. Radicchi, S. Fortunato, C. Castellano, Universality of pmid: 22481340 Science Foundation grants NCSE 1538763, EAGER 1566393, citation distributions: Toward an objective measure of 88. M. Thelwall, K. Kousha, Web indicators for research and NCN CP supplement 1553044 (K.B.) and SBE1158803 (J.A.E.); scientific impact. Proc. Natl. Acad. Sci. U.S.A. 105, evaluation. Part 1: Citations and links to academic articles National Institutes of Health awards P01 AG039347 and 17268–17272 (2008). doi: 10.1073/pnas.0806977105; from the Web. Prof. Inf. 24, 587–606 (2015). doi: 10.3145/ U01CA198934 (K.B.) and IIS-0910664 (B.U.); Army Research pmid: 18978030 70. L. Waltman, N. J. van Eck, A. F. J. van Raan, Universality epi.2015.sep.08 Office grant W911NF-15-1-0577 and Northwestern University of citation distributions revisited. J. Assoc. Inf. Sci. Technol. 89. M. Thelwall, K. Kousha, Web indicators for research Institute on Complex Systems (B.U.); DARPA (Defense 63, 72–77 (2012). doi: 10.1002/asi.21671 evaluation. Part 2: Social media metrics. Prof. Inf. 24, Advanced Research Projects Agency) Big Mechanism program 71. M. Golosovsky, S. Solomon, Runaway events dominate the 607–620 (2015). doi: 10.3145/epi.2015.sep.09 grant 14145043 and the John Templeton Foundation’s heavy tail of citation distributions. Eur. Phys. J. Spec. Top. 90. L. Bornmann, What is societal impact of research and how grant to the Metaknowledge Network (J.A.E.); Intellectual 205, 303–311 (2012). doi: 10.1140/epjst/e2012-01576-4 can it be assessed? A literature survey. Adv. Inf. Sci. 64, Themes Initiative “Just Data” project (R.S.); and European 72. C. Stegehuis, N. Litvak, L. Waltman, Predicting the long-term 217–233 (2013). Commission H2020 FETPROACT-GSS CIMPLEX grant citation impact of recent publications. J. Informetr. 9, 91. C. Haeussler, L. Jiang, J. Thursby, M. Thursby, Specific and 641191 (R.S. and A.-L.B.). Any opinions, findings, 642–657 (2015). doi: 10.1016/j.joi.2015.06.005 general information sharing among competing academic and conclusions or recommendations expressed in this 73. M. Thelwall, The discretised lognormal and hooked power law researchers. Res. Policy 43, 465–475 (2014). doi: 10.1016/ material are those of the authors and do not necessarily distributions for complete citation data: Best options for j.respol.2013.08.017 reflect the views of our funders. modelling and regression. J. Informetr. 10, 336–346 (2016). 92. A. Oettl, Sociology: Honour the helpful. Nature 489, 496–497 doi: 10.1016/j.joi.2015.12.007 (2012). doi: 10.1038/489496a; pmid: 23018949 10.1126/science.aao0185 Fortunato et al., Science 359, eaao0185 (2018) 2 March 2018 7 of 7  Science of science Santo Fortunato, Carl T. Bergstrom, Katy Börner, James A. Evans, Dirk Helbing, Stasa Milojevic, Alexander M. Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, Alessandro Vespignani, Ludo Waltman, Dashun Wang and Albert-László Barabási Science 359 (6379), eaao0185. DOI: 10.1126/science.aao0185 The whys and wherefores of SciSci The science of science (SciSci) is based on a transdisciplinary approach that uses large data sets to study the mechanisms underlying the doing of science−−from the choice of a research problem to career trajectories and progress within a field. In a Review, Fortunato et al. explain that the underlying rationale is that with a deeper understanding of the Downloaded from http://science.sciencemag.org/ on May 23, 2018 precursors of impactful science, it will be possible to develop systems and policies that improve each scientist's ability to succeed and enhance the prospects of science as a whole. Science, this issue p. eaao0185 ARTICLE TOOLS http://science.sciencemag.org/content/359/6379/eaao0185 REFERENCES This article cites 91 articles, 25 of which you can access for free http://science.sciencemag.org/content/359/6379/eaao0185#BIBL PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions Use of this article is subject to the Terms of Service Science (print ISSN 0036-8075; online ISSN 1095-9203) is published by the American Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. 2017 © The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works. The title Science is a registered trademark of AAAS. View publication stats