Apr 12, 2014

Review: Scientometrics


As there are some proposals to write consumer reports of academic journals, I started to write reviews of journals I publish in. Together with Xiaofeng Wang and Pekka Abrahamsson, I recently published an article in Scientometrics.

Scientometrics describes itself as an “International Journal for all Quantitative Aspects of the Science of Science, Communication in Science and Science Policy“. It is quite multidisciplinary in its content. However, it mainly deals with measuring science. It is published by Akadémiai Kiadó and Springer Science+Business Media. Scientometrics is a traditional, subscription-based journal. It participates to Springer’s Open Choice program, where authors pay (quite high) article processing charges to make published articles open access. That makes Scientometrics a hybrid open access journal. The journal started publishing in 1978, making it the oldest journal I ever published with. Its current impact factor is 2.133 which, for my field, is pretty high (although I do not care that much about it, but it seems some people do). The journal declares an impressive (20+) number of abstracting and indexing services. Among those are Scopus, DBLP, ACM Digital Library, and the Web of Science. They do matter a lot in my field.

My experience as an author

Scientometrics has recently published the following article:

D. Graziotin, X. Wang, P. Abrahamsson, “A framework for systematic analysis of open access journals and its application in software engineering and information systems“, Scientometrics, Online First, 2014. DOI:10.1007/s11192-014-1278-7arXiv:1308.2597

The article deals with the creation of a framework for systematically analysing gold open access journals. It also contains a sample application in the fields of software engineering and information systems. Basically, we needed to understand the state of gold OA journals in our research fields. Therefore, we constructed a framework for analysing the journals and we applied it. It turns out that there are several issues for going gold open access in the fields of software engineering and information systems. Much works still has to be done in order to free up knowledge in our research fields. The content of the article will probably be the subject of another blog post. Let us focus on my submission experience.

I have to admit I stumbled across Scientometrics quite randomly. I am familiar with software engineering and information systems venues, but not scientometrics. Truth is, this paper was first submitted to an open access journal with an open peer review process. The paper was rejected there with mixed reviews. It is not the objective of this post to argue what happened there, but there might be another post to tell the whole story. In short: we had a strong feeling that the article was rejected more because of “social/political” issues instead of other problems, because it is critical to open access in the fields of software engineering and information systems. Additionally, it might also be that the chosen venue was not the best one for the paper: scientometrics papers there are only a small category of the journal. So, we were left with a rejected manuscript with the feeling that there was nothing fatally wrong with it. One of the reviewers of the manuscript chose to reveal his identity. I am glad to report his name: Christian Gumpenberger. Out of curiosity, I tracked him, and I discovered that he is a confident author in the field of scientometrics, with a background regarding open access publishing. That relieved us, because his review was very positive and applauding. Indeed, he appears by name in the acknolwedgements of our article. I like to thank reviewers by name. As non familiar with the field, we checked where he published in the past. This is how we found Scientometrics. The journal seemed well established for the reasons stated at the beginning of this review. Additionally, our paper was 100% in scope with the journal. Given that the papers can be made openly available there, we decided to submit with the confidence that experts in the field would evaluate it. Needless to say, we first revised the manuscript according to what we found valuable in the previous reviews.

The manuscript was submitted on 2013-11-08. I mentioned in the cover letter that the manuscript was already self-archived in arXiv as a result of a previous submission. That did not seem to bother anybody, at all. I also mentioned that one of their authors was involved in the previous review process.

On 2014-01-24, we received the editorial decision, together with the reviews. It was an accepted with minor revisions. Pretty curious and somehow funny, given the strong rejection received from the other venue. It has been the first time in my life that I received an almost immediate acceptance. It turned out, it was the case of my co-authors, too. We immediately jumped into reading the reviews. Either that was a bogus journal, or the reviewers and editor really appreciated the manuscript. It was the second case. We received two lengthy, detailed reports from the referees. Both of them were very positive but still presented insightful suggestions. Both of them also suggested to accept the manuscript in their reports. One of the two reviewers actually performed two levels of review. The reviewer provided the usual report, with the usual comments, concerns, and suggestions. However, the reviewer also provided a commented copy of our PDF, with several minor corrections, adjustments, and questions. There were also several compliments and encouragements in the commented PDF, for example near our reference of the open data sample for the study. We were so happy with the received reports, that we decided to also take into account most (if not all) suggestions provided in the “third” review. Just to approximate the detail of the received reports, our response to reviewer letter length was 3500 words.

On 2014-02-11, we submitted the revised version together with the response to reviewers. That quickly came back to us.

On 2014-03-04, the editor asked for minor revisions again. Literally, there were three sentences to be corrected because of wordiness. That was it. The day after, we submitted the revised version.

On 2014-03-06, the manuscript was accepted for publication.

Between the next day and 2014-04-02, there have been a couple of bounces between me and Springer’s production team. As we were (and currently are) in short of fundings, I tried to convince Springer to waive the article processing charges and go fully open access, or at least to discount the fees. 2200,00€ + VAT (22% in Italy) was way too much for a study not covered by any grant, in a university lacking funds for open access. Unsurprisingly, Springer denied any waive or discount. At least I tried. There was still the option of going green open access. I can report that the production team was professional and quick to respond. I received two proofs, and was given possibility to correct minor typos and introduce text embellishments. As far as I know, no errors were introduced by the production team.

On 2014-04-08, the article was published as “online first” and assigned a DOI. It was formally published. Scientometrics is also published on paper, and I like the idea to make the article available on line as soon as possible, instead of waiting for it to be formally published on paper.

I have to admit that, despite of the high professionalism, I could not feel a truly friendly atmosphere there. The “human” side I could grasp when I submitted to PeerJ or JORS was not present there. On the other hand, everything went smooth. The final, published article reads nice. It has a nice layout and looks professional in the typographic settings. However, this is Springer’s  “default” layout. Most, if not all Springer articles look that way.

Being in a traditional publisher, the journal has not helped in disseminating the article. This has bothered me, because we are in 2014 and social media presence and advertisement should be a standard basis. Take this as opposed to the strong disssemination offered by PeerJ to our article. The difference is abysmal. Another downside is that the journal is not gold open access, but that is to be expected from a traditional publisher. Springer Open article processing charges are high, too high.

Based on my experience, I would recommend submitting to Scientometrics. However, I strongly encourage potential authors to self-archive their postprints, in order to free up their research articles. Even better if they can afford paying the article processing charges, and subsequently open the published article.

How to open articles in Scientometrics

Ironically, our article in Scientometrics is a very open one, despite the fact that the publisher’s version is behind a paywall. The article contents have been freely available months before submitting the paper.  I self-archived the article on arXiv, 1308.2597. The current author-generated version in arXiv is a post-print. That is, its contents are the same as those of the published version. The dataset of the analyzed journals is available as open data. The data was released on figshare even before actually writing the article. How was that possible?

The journal (and its publishers) lets authors self-archive both preprints and postprints. For self-archiving the postprint (i.e., the after peer review version of a paper, which is generated by the authors), there are two ways. Either the authors wait 12 months after the formal publication before self-archiving as Springer tells, or they actually read the copyright transfer agreement (CTA) they sign. At section III of the CTA, before scaring the authors by telling them that the author “[..] has not granted any exclusive or non-exclusive right to the Article to any third party prior to the execution of the present Statement“, it is stated that (emphasis added):

The Author is entitled, however, to self-archive the preprint version of his/her manuscript. The preprint version is the Author’s manuscript or the galley proof or the Author’s manuscript along with the corrections made in the course of the peer review process

The CTA considers as preprints both what we commonly call preprint (before peer-review) and postprint (accepted version, after peer-review). This means that authors can self-archive preprints whenever they want. In my case, however, the preprint was already self-archived on arXiv. I just transparently mentioned this in the cover letter when I submitted the manuscript.

written by dgraziotin

Dr. Daniel Graziotin received his PhD in computer science, software engineering at the Free University of Bozen-Bolzano, Italy. His research interests include human aspects in empirical software engineering with psychological measurements, Web engineering, and open science. He researches, publishes, and reviews for venues in software engineering, human-computer interaction, and psychology. Daniel is the founder of the psychoempirical software engineering discipline and guidelines. He is associate editor at the Journal of Open Research Software, academic editor at the Research Ideas and Outcomes (RIO) journal, and academic editor at the Open Communications in Computer Science journal. He is the local coordinator of the Italian Open science local group for the Open Knowledge Foundation. He is a member of ACM, SIGSOFT, and IEEE.

Leave a comment