O'Really?

A crude score for benchmarking scientists

Have you ever wanted to compare different scientists by their publication record? It’s not always an easy task, but here is a crude and handy way to benchmark people by their journal publications in Nature, Science or PNAS using PubMed. Let’s call it the NSPNAS score, it’s not the h-index and it’s far from perfect, but it can be useful.

Imagine these scenarios:

1. You’re a young scientist comtemplating who to do an undergraduate project, Masters degree or PhD with.
2. You’ve finished your PhD and are wondering which lab could be your Stairway to PostDoc Heaven [1].
3. You’re lucky enough to have landed a faculty position and you want to check the credibility of your new colleagues.
4. You want to do some industrial espionage on your competitors in different labs around the world.
5. You’re a Scientist dammit, and naturally you’re a curious person who just likes to measure things.

In any of these situations, you’ll probably want to look up the people concerned using Google Scholar which will give you a good idea of their research history. But you’re not interested in publications in the Journal of Few Subscribers or the Proceedings of the Boring Incomprehensible Nonsense Society (BINS), even if Google Scholar lists hundreds of their citations. Instead, you care about counting the Big Bang impact publications they have in the über-journals: Nature, Science and PNAS. You can find these publications in PubMed with this simple query:

Surname +Initials[au]+(nature[journal] or science[journal] or Proc Natl Acad Sci U S A[journal])

…and you can obviously modify this query to include popular journals from your own field as appropriate.

Where NSPNAS works

Note, NSPNAS scores were correct at the time of writring in 2007, but will change over time.

When you substitute an authors name and initials into the beginning of that query, you get your NSPNAS score. So Systems Biologist Douglas Kell for example, surname and initials “Kell+D[au]”, has an NSPNAS score of 6.

If the person in question has a unique or unusual surname and initials, its fairly easy to find their score: Nodalpointer Chris Mungall has an NSPNAS score of two while nodalpointer Jason Stajich has an NSPNAS score of three. These results suggest a positive correlation between Californian sunshine and NSPNAS. Meanwhile, back in rainy old Britain, Ensemblian Ewan Birney scores a formidable sixteen, which is just scary for a bloke in his thirties.

Where NSPNAS doesn’t work

Unfortunately, authors with common names like John Smith (who has more than 340 hits) can’t be easily benchmarked with this type of query, without trawling through hundreds of false positives. More importantly, some influential scientists score very low or zero, despite the fact that their work has been important in the world of biomedical science an beyond. This is especially true for Computer Scientists, Mathematicians and Informaticians, for example:

• Some bloke called Tim (see picture, top right) scores a measly two and neither of these papers are particularly inspiring or highly cited. Contrary to popular belief, Tim didn’t invent the internet, but did play a leading rôle in the creation of the web. Can you imagine a world without it?
• Googler Sergey Brin scores scores zero (once you exclude the false positive). But bioinformatics, and life generally, without search engines like Google is almost unimaginable. Sergey’s most heavily cited paper, co-authored with Larry Page, describes a prototype search engine called “Google”. This paper was first published at the seventh World Wide Web conference (WWW7) way back in 1998.
• Googler Vint Cerf scores a pathetic one despite winning a Turing award (the Nobel Prize for Computer Science) for his co-invention of TCP/IP
• Stanford’s Mark Musen scores zero, but his Protégé Ontology Editor and its derivatives have been influential in biomedical informatics, and will probably play an important rôle in the creation of next generation of biomedical web applications.
• Leading mathematicians, such as Fields Medallists (the Nobel Prize for Mathematics) and winners of the Clay Millenium Prizes [2], typically score zero despite making fundamental, and indirect, contributions to biomedical science.
• Desperate PERL hacker Larry Wall scores zero, but bioinformatics without PERL would be quite different.
• This list is endless, so we’ll move on…

Many important members of the Dead Scientists Society also have low NSPNAS scores…

• Edgar Codd scores zero, but can you imagine biomedical science without his relational database?
• Edsger Dijkstra scores zero, but without him we’d probably still be taking the longest path to wherever we’re going.
• Charles Darwin scores zero, PNAS didn’t even exist in his lifetime and both Nature and Science were in their infancy when he died.
• Albert Einstein scores only one (and has even made it into PubMedCentral)
• Alan Turing also scores zero, because none of his biomedical publications were in NSPNAS. Try to imagine science without Computers and Artificial Intelligence, because without Turing, bioinformatics and computational biology might not even exist at all…

Conclusions

All these statistics remind us that many important ideas, techniques and results are not published in Nature, Science or PNAS and others are excluded from the PubMed index completely. It also confirms what we already know about peer-reviewed Journal publications not being the be-all and end-all of Engineering, Science or Medicine [3]. But NSPNAS still has its uses, provided the people you’re benchmarking have a rare name and didn’t snuff it before the PubMed index starts.

What is your NSPNAS score? If like me, you score a spectacular “nul points”, console yourself with the fact that you’re in good company with that score and given time, maybe you can change it.

This work is licensed under a

Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.