The revista española de pedagogía and the dialectic between caring and metrics

The month of July was very hot in Madrid. Tired of the heat, I decided, at the beginning of the month, to follow the example of King Philip V, who 300 years ago bought a farm from the Hieronymite monks of El Parral Monastery, situated on the north side of the Sierra de Guadarrama, around eighty kilometres from Madrid, which had originally been donated by the Catholic Kings in 1477. The reason for this purchase was clear: Philip V knew the site because he had often hunted in the area and noticed that, during the summer months, the air was fresh and very pleasant.

Following in these royal footsteps, I went to La Granja de San Ildefonso one Sunday with some friends, who also wanted to explore its famous gardens and the Royal Palace. The day was both refreshing and enlightening. We decided to join a tour of the Palace, led by a guide who told us some very interesting and detailed information. Out of everything we heard, it was a secondary topic that caught my attention, which was mentioned during an explanation of the extraordinary collection of Flemish tapestries made in honour of King Charles I. The guide explained that making the tapestries required the collaboration of four different kinds of people. First, a talented artist had to be commissioned to create a painting on the desired subject. If the artist was not also a cartoonist (as was the case of Goya), a separate cartoonist was appointed to adapt the painting into a cartoon the size of the future tapestry. Thirdly, there were the people who selected the threads according to their quality and colour. Finally, there were the weavers, who took the cartoons and thread to a loom. In order to successfully weave tapestries, they needed to be trained for three years, except for creating the faces, which required more extensive training. The majority of the weavers were women as they had more delicate hands than the men, as well as being more patient.

Our guide did not tell us some of the information which appears in other texts on tapestry-making techniques, which underlines the importance of the Upholsterer who invested a large amount of money, manpower and time and had to ensure that it culminated in the sale of a product, that it was, undoubtedly, an honour but also an extremely expensive practice.

This process shows the complexity of making tapestries, as money alone is not enough; it is essential to know how to coordinate people with different talents, always of the highest quality and with ideas on the work to be carried out that, at times, do not coincide with each other’s.

All of this made me think about the similarities between creating a beautiful tapestry and crafting a successful scientific journal.

Indeed, one could say that the person who takes on the role of artist is equivalent to an author of a journal, in such a way that it would seem the most important thing is to have quality authors. However, the creation of tapestries teaches us that without great cartoonists, the results could be mediocre at best. The cartoonist is equivalent to the editors of a journal, as well as its evaluators. The editors have to identify both the broad guidelines found in editorial policies and specific details on presenting works, which should be made public knowledge. Similarly, the evaluators have to determine whether the works presented sufficiently respect both the guidelines and quality requirements, within the scope of the international scientific discussion at that time. Equally, in the current climate, the evaluators must guarantee that readers are presented with an article that complies with the applicable ethical requirements.

Thirdly, there are the publication assistants of a journal, who, if comparable to the people who select the threads and colours for a tapestry, are concerned with a number of issues when making a journal, such as ensuring diagrams and tables are legible, that both the Spanish and English versions of a text are intelligible, that there aren’t any spelling mistakes, that the deadlines for publishing the journal are met, etc.

Finally, there are the weavers, who in a journal are those who work in printing and binding.

Combining the work of so many people requires great attention-to-detail on the part of the editors, detail that requires them to know how to say no to many authors, know the editorial policies of the time, introduce changes to improve the quality of the journal, select evaluators that are able to voluntarily dedicate their time to evaluating the work of others, not only in terms of making the right final judgement but also in terms of knowing how to suggest changes that prevent errors made by the authors. The editors also have to appoint people with special skills to be new production assistants in order to achieve perfect texts and choose companies to print the journal that diligently carry out their work. I could describe a whole host of situations that have required me to not only reject original articles but also dismiss evaluators, change publication assistants or stop working with a printing company. But these experiences are of relative interest: they are examples of the attention-to-detail needed from those managing a journal.

I have so far explained what I heard from the guide who showed us the tapestries in the La Granja Palace and I have said that the figure of the Upholsterer, who is responsible for the company finances, should not be forgotten. Here we find a topic which is being heavily debated at the moment and in which the Upholsterer, or Editor, has a special responsibility.

Indeed, we cannot forget that the so-called “welfare society” tends to promote a “free of charge” mentality, despite the claim by the English that “there is no such thing as a free lunch“, and the French saying that “la gratuité n’est pas gratuite”. Publishing a journal, even if only online, costs a fair amount of money, in spite of the fact that more than a few of the people who collaborate (and I speak to authors as an editor) do not necessarily charge. So, who provides the money? Tradition always was that major journals did not pay the authors, and that the costs of publication were passed on to the subscribers or those who occasionally bought an edition or article within an edition. But growing unease has now turned into uproar, both because of the “free of charge” mentality that leads some to claim that online journals should be free, and because of the internationalisation of metrics, which has led to a journal’s importance being defined by the number of times it is referenced during its first year of publication (the so called impact factor) especially if referenced by journals of a similar quality. In turn, this same internationalisation has led to requiring professors to publish in the journals with the biggest impact factor and more than a few nations or institutions think it gives them a “touch of distinction” to have their members appear in such journals: I have received many offers from various Asian countries offering large sums of money for each article published by one of their members, offers to which I have never responded. While the storm has been raging on, a number of journals have appeared (currently considered predatory if they are not strongly supported by relevant quality criteria) that are only available online, and offer you the chance to publish your work (or even be the editor of a special issue, if they have seen that you have edited a leading journal) for a sum of money.

All of this has led to more than a little confusion. The basic problem, as I said, is twofold: on the one hand, deciding who should pay, and on the other, discussing new quality criteria, which are not always entirely appropriate.

Previously, it was clear that the reader should be the one to pay. Now, the new mentality, as well as the growing desire for recognition among authors and the groups they are part of, which promises the opportunity to feature in a prestigious journal, has led to the authors or institutions concerned to more often than not be the ones landed with the costs.

On the other hand, the new quality criteria are in a constant state of effervescence as metrics are in reality around twenty years old, and the fight to impose certain parameters that defend various databases is now an all-out war in which a new front appears every day.

I don’t intend to outline “the one best system. Life is sufficiently complicated to feel moved to understanding the viewpoints of those who do not think like you, and to not expect anyone to think your ideas are the best, without question. But what I do want to do is propose some kind of rationale based on the current position of the revista española de pedagogía.

Firstly, I should mention that I do not agree with the “free of charge” mentality. It is reasonable to expect the recipients of anything which requires human effort to show gratitude, which often translates into monetary recognition. Consequently, I think that it is wrong to expect the authors of the articles to cover the costs of making the journal. I am well aware that there are journals which charge the author for every page published, just as there are some who expect the author to buy a large number of copies of the edition their work is published in. I regret to say that I don’t think this practice is appropriate, even if that means continuing to charge subscription fees for printed copies of the journal, and that online texts require anyone that wishes to read them to pay.

Naturally, that does not mean that everything on the Internet should be subject to a fee. It is logical that the unlimited knowledge we have inherited (by standing on the shoulders of giants) is made available to the world, carried forth by cultural interest, which must be a basic premise for imposing charges. On the other hand, large databases are now reaching agreements with a number of journals so that they advertise their texts as soon as they are published, or after an embargo period, for which they pay a small fee to the journals and charge a significant amount to the institutions subscribing to their service. But, leaving these experiences to one side, which practice is followed at present? Well, we should note that there are now thousands of journals and a variety of practices followed. As a result, I feel it is appropriate to provide some information about the first 100 journals that appear in the Journal Citation Reports (JCR), in their database of journals dedicated to Education and Educational Research. Of these, 74 charge for reading an article (at an average cost of around 37 dollars) and, of the remaining 26, only 17 continue to use a simple open access system. It seems that there is a clear conclusion: a significant number of major journals charge the readers and not the authors. Evidently, statistics can be calculated in a number of different ways, because a journal on natural science is not the same as a journal on social science. It is undeniable that the online journals which are purely “Open Access”, within the field of natural science, tend to charge authors significant amounts, for example Springer, which edits 1966 open access journals, charges an average of €2300 per article. This has led the San Francisco Declaration on Research Assessment (DORA), a text which was called for during a meeting of the American Society for Cell Biology, on 16 December 2012, and that has, to date, been signed by 1530 organisations and 14,690 scientists to insist, as its underlying thesis, that in order to evaluate the quality of scientific production, it is necessary to “assess research on its own merits rather than on the basis of the journal in which the research is published”. It also insists on “greatly reduc[ing] emphasis on the journal impact factor as a promotional tool, ideally by ceasing to promote the impact factor or by presenting the metric in the context of a variety of journal-based metrics (e.g., 5-year impact factor, EigenFactor, h-index, editorial and publication times, etc.) that provide a richer view of journal performance.”

We can see in this text that there is a crossover, of some sorts, between the two basic problems referenced above: who should pay for publishing new scientific knowledge and how should the new quality criteria be rated. As I said, I do not intend to provide a complete analysis of both questions, simply to propose some arguments relating to them by presenting some current data from the revista española de pedagogía (rep). I will briefly mention that, based on current reasoning, we believe that it is the reader who has the primary responsibility of economically sustaining a journal. Consequently, it is the subscribers of the printed Spanish copy of the journal, as well as those who subscribe to the online bilingual version, who should pay some kind of fee, in spite of the fact that the online version only has an embargo of one year, during which the readers can download embargoed content at a modest price of €12. In turn, there is no charge to authors for publication, neither for a printed copy nor the online version. However, the care that journal editors must take with regard to the quality of texts means that we do not accept English versions of a text that have not been translated by a native speaker. We instead ask for texts to be translated into English by a sworn translator, having agreed with a company that they directly charge authors a set fee, with the journal covering the remaining costs.

We now turn to the second issue, related to the emergence of metrics and new quality criteria. It seems to me that we should not begin to discuss this topic without having first reminded ourselves of the two quality criteria that have taken precedence for a long time: expert opinion and the number of readers subscribed to the journal, interested in keeping up to date with scientific progress. Of course, there were elements of subjectivity within these criteria, but that did not stop them from demonstrating the public appeal of a particular journal. Today, things have changed and it is years since I have read any statistics relating to expert opinions on journals. However, I feel that some very objective data must be provided in relation to previous practices. Instead of experts, we could focus on the number of authors that wish to publish in a specific journal. Thus, during 2018, the rep received 188 original articles of which 24, or 12.8%, were published. These numbers, aside from highlighting the considerable amount of editorial work that goes into the journal, show that it is widely accepted among researchers, as many submit work despite the obvious knowledge that, with maximum 600 pages printed per year, only a limited number of texts are published, always of the highest quality.

Likewise, the number of subscribers can be substituted by the number of people accessing the journal online, as well as those who go on to download some of its articles. It is, however, impossible to know exact figures as that would require access to a number of online repositories. We will therefore have to limit ourselves to two sources of data: Google Analytics, for the year 2018, and the data recorded by JSTOR. The first reveals that, in 2018, the revista española de pedagogía had 81,505 users living in 144 different countries. Naturally, the majority of countries (112) registered less than 100 users, while there were more than 20,000 in Spain and Mexico and between 1000 and 10,000 in Columbia, Argentina, the Dominican Republic, Venezuela and Guatemala.

JSTOR, meanwhile, who a few years back honoured us with an offer to be the first Spanish language education journal to appear in its archives (characterised by having included, from its founding, every original copy of the 2700 journals which it contains, and to which 11,343 institutions are currently subscribed from 178 countries) told us that in, 2018, we had 22,630 readers who downloaded 9006 articles.

Although these figures should be enhanced by statistics from other institutions, I think they already sufficiently demonstrate the significant interest that a great number of people have in the revista española de pedagogía.

Finally, we should mention the position that the journal currently holds in the two main international databases that follow several metrics currently in use. I have already warned that the current obsession with metrics is disputed by many. However, I understand that it would be wrong to abandon them altogether, and so accepted without question when, back in 2005, I received an offer from JCR to be the first Spanish language journal to appear in the Education and Educational Research section, where we are now one of six Spanish journals. I equally accepted the later proposal by Scopus to be part of its group of education journals, which currently includes 35 Spanish journals, of which 9 only appear in English.

The results published by the JCR on 20 June 2019 could not be more promising for the journal. It has gone from an impact factor of 0.547 in 2018 to 0.863 in 2019, which translates to a 21-point rise in comparison to the position we occupied last year. On the other hand, the position of our journal in the data provided by Scopus on 31 May 2019 was very surprising: last year we were 14th in Spain while this year we have dropped to 24th and in the international list we have gone from 631 last year to 599 this year.

Obviously, the contradiction between these numbers, in the metric provided by the same company in the same year, is a little disconcerting, even if the authors do have their reasons. Ultimately, this data reinforces my opinion that, in this dialectic between the care that goes into and can be seen in a journal, and the attention paid to the data provided by metrics, journals which are cared for should inspire more confidence. It seems to me that although we cannot forget metrics, we would be mistaken to follow a policy that deifies them.

José Antonio Ibáñez-Martín ■