I might be able to quickly get a web application up and running, but like Torbjörn I have some serious qualms about this metric.
My main concern is that how many citations an article receives is largely determined by how big the researchers' field is. As a result, a paper might get cited more than any other article in its subject area, but cited less than most articles in the journal it was published in. According to your metric this article would be viewed as mediocre, but in reality it might be the most influential paper in its field.
Dear Jordan, you raised the concern which I've often encountered before. Yes, my metric is not designed to compensate for field differences in citation patterns. There are many other indices that try to do that. I firmly believe that field normalization is both not required and not possible, and is mutually exclusive with impact factor normalization. Let me explain it in detail.
I believe that correct field normalization is not possible in principle, because it is not possible to unambiguously define what a scientific field is and where its borders are. All such attempts are doomed to fail. Perhaps, we should just let the fields define themselves. And if the field is big, probably it is because it formed around an important problem which many scientists are attracted to, as opposed to some obscure field nobody is interested in. Hence the differences in citation patterns. And I believe these differences are fair and need not be corrected.
I also believe that field normalization is generally not required, because nobody is comparing apples and oranges. Imagine that a position is open for a mathematician. If citation indices are used in this hiring decision, mathematicians will be compared to mathematicians. No one in his sane mind would hire a biologist for this position, even if his h-index is 10 times higher!
Thirdly, you can normalize either to the journal or to the field. You cannot normalize to both. And if you normalize to the field only, you will lose all the point of this index - to correct for the impact factor bias.
Finally, and directly answering your concern: my index is designed to change the behavior of scientists when submitting articles to journals. Please, select journals that are publishing papers from your field. You will have no problems. If, however, you aim for a multidisciplinary journal, then be prepared to face competition. Because the readers of multidisciplinary journals, as general population of scientists, are not so much interested in obscure fields.
If I have convinced you, I hope we will see your application soon. Cheers!
I agree that researchers should publish articles in the appropriate journals, however I cannot blame researchers who have submitted their best work to prestigious journals instead of a field specific journal. And because the most prestigious journals publish articles from a range of fields I feel that smaller fields will be unfairly punished.
I also have some technical concerns about this metric. Let's say I take the time to calculate the median citation rate for a certain journal in 2015. Articles which were published in January 2015 will have an unfair advantage over articles which were published in December 2015. I would prefer to have a method of calculating the index that can control for what time of the year the article was published. For example, if an article was published in January 2015 I would prefer to find the median citation rate for articles published from January 2014 to January 2015. Or better yet, if an article was published on January 31st 2015 I would prefer to find the citation rate for articles from January 31st 2014 to January 31st 2015.
And I think this will be difficult to implement. I don't know if there is a good API out there to easily extract the citation counts for articles from a journal for a very specific range of dates. As Torbjörn mentioned he used Web of Science to get his citation data. I don't know if he did a bulk download like in this publication: http://biorxiv.org/content/early/2016/07/05/062109
Regardless, it appears you need to log in and navigate multiple pages.
In order for the web application to be as accurate as possible I would prefer to not have the median citation rates precalculated, but rather be able to calculate the rate on the fly to ensure it is as up to date as possible.
As I said before, I do not see the differences in citation rates between fields as unfair and in need for compensation. Regarding the precise time interval - yes, I agree, if such data are available, why not to use it. But I would rather suggest, for an article published on January 31st 2015, to calculate the median for articles published from to July 31st 2014 to July 31st 2015. Of course, if you will need to calculate a day-specific median for each article, and do not want to use precalculated medians for journals, the number of requests to database will grow substantially. I know that Google Scholar is limited to 1000 requests, which would probably allow to calculate this index for a single article only. So, to be realistic, the precalculated medians are required, unless there will be a cooperation with Google Scholar or WoS and they will implement this index natively, which I do not hope so much for. Perhaps we can start with a simple rough algorithm, and if the resource becomes popular, better solutions may emerge.
I'm sorry, but I'm not really a fan of article or journal metrics, and if I had to use a metric I would prefer a field normalized metric such as the one proposed here: http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002541, which already has a nice web tool: https://icite.od.nih.gov/
I think you are misunderstanding the term "field". You mention comparing mathematicians and biologists. I'm not concerned about this comparison. The type of field difference I am concerned with is comparing researchers who study miRNAs to researchers who work on less studied small RNAs such as piRNAs or tRFs. There are thousands of papers published each year on miRNAs which provide lots of opportunities for citations while there are relatively few papers published on tRNA fragments. Even within the field of miRNAs there are some miRNAs which are much more well studied than others, and publishing on a well known miRNA will no doubt give you more citations than working on a miRNA that no one else is studying and therefore publishing on. I view this as a problem that should be taken into account, but you are free to disagree.
Oh, I know about that metric. The algorithm is overcomplicated, and has a major drawback - if your article is cited by an article form influential field, your score will drop! It has been nicely criticized here: https://www.cwts.nl/blog?article=n-q2u294&title=nihs-new-citation-metric-a-step-forward-in-quantifying-scientific-impact
Yes, I'm misunderstanding the term field, as everybody else does. Because it is impossible to define it unambiguously. And there will always be disagreement about what it is and the normalization fairness. I also have a different impression about the effect of field size on the number of citations. I think the bigger the field, the less citations each article gets. Because it has to compete with much more articles that in a small field. In a small field, all groups are likely to know each other and cite each other, so citations per article will be high.
I see the logic you are trying to apply to field sizes and citation count, and admittedly it is easier for an article to fall through the cracks and not be read or cited in a large field, but for the most part your logic is faulty. When testing one's logic it is often helpful to look at extremes. Imagine there is only one other lab in your field. Assuming they are generous and cite you in all of their publications, your citations will be capped at how often they publish. Does that sound like a good situation?
I admire your various proposals for adjusting citation counts, but it will never be possible to come up with a number that makes everyone happy. If we are going to reduce an article to a single number, then researchers will naturally want whatever number makes them look the best, and it isn't possible for one metric to make every article look good. Perhaps we should use the solution found in the South Park episode "T.M.I", and just make a metric that declares every paper above average.
I understand the desire to reduce papers and scientists to a single number, and assuming that this number correlates with success in obtaining grants it makes sense for search committees to utilize these numbers, but I find using number of citations as a method to determine quality/impact of research deeply flawed.
Just like amateur cat photos on Reddit will get more views than works of art, questionable and simple research with a flashy title will get more views and citations than complicated and important work. It is possible to publish a paper that is ahead of your field and therefore underappreciated until the field catches up to you. For example, Albert Einstein never received a Nobel Prize for his theory of relativity because no one understood it.
Not to mention that some of the most highly cited papers have either been retracted or are generally considered laughable.
And there is also the fact that most people cite research without even reading the publication, and as a result many citations are incorrect in their context. And although a researcher might be upset to see his/her paper incorrectly cited, they likely don't complain because they are happy for their citation count to go up.
Scientists are for the most part just average people, and can't be relied upon to correctly identify important work. Instead of relying on metrics we should just read the paper to determine its worth.
Ok, you convinced me that citation numbers in a small field are capped by the rather limited publication output of that field, whereas in a large field there is no such limit. Still, I think it is fair, because small fields are small due to their limited relevance and thus low number of researchers who want to work in that field. Moreover, the best and most highly cited papers break the boundaries of a single field. You may want to read my comment piece here: http://comments.sciencemag.org/content/10.1126/science.aaa3796 You will also see why I defend citation metrics and find it useful. But, in general, I agree that nothing is better for evaluating a given article than a thorough read by an expert in the field. Isn't it what a peer review is for? Still, grant committees do not have time and expertise to thoroughly evaluate all papers of all researchers submitting their proposals, so some proxy for article importance is necessary. And, in my opinion, proxies based on citation counts are still better than proxies based on the number of publications or twitter mentions. Paraphrasing Churchill, citation metrics are the worst method to evaluate research, except for all the others.