Indexed out
Rankings, Scopus and the Indonesian research system
This event is part of a demonstration of what Lera offers for running knowledge events. Its content is written with care and draws on real sources, but the conference has not taken place, and apart from the convener its participants are fictional.
Indonesia has one of the largest collections of open access journals in the world. Thousands of journals are published by public and private universities, most of them in Indonesian and free to both author and reader. Yet when a university in Yogyakarta wants to promote a lecturer or graduate a student, the main question is not what that person has written or for whom; it is whether the article is indexed in Scopus.
In 2017 Indonesia's research ministry launched SINTA, a national index for ranking the country's journals and researchers. On the one hand SINTA was an answer to the dominance of commercial indexes, and a way for local journals to be seen. On the other, it repeated the same logic on a national scale: journals were sorted into six tiers, the tiers became points, and the points became salary and promotion. And at the top of that ladder, indexing in Scopus still carried the most weight.
The consequences were predictable. A market grew up for 'predatory' journals that sold indexing to anxious researchers. Local journals turned to English to get into Scopus and lost their readers. And research written for farmers, teachers or village councils came to weigh almost nothing in assessment. An index that was meant to make knowledge visible put a great deal of it out of sight.
The Leiden Manifesto of 2015, and the San Francisco Declaration on Research Assessment before it, warned that quantitative indicators must not replace qualitative judgement. In 2022 the European Coalition for Advancing Research Assessment turned this into an institutional commitment. But these reforms are mostly carried out in the wealthy universities of the North, where a university has enough standing to do without indicators. In the South, the indicator is still the only language that ministries and global rankings understand.
My argument is that assessment cannot be reformed without reforming infrastructure. As long as the only respected indexes belong to private, monolingual companies, every national index will either become a copy of them or stay on the margins. We need a public infrastructure that recognises a piece of research in whatever language it was written, records its social use, and returns assessment to human judgement.
Questions for discussion
- Are national indexes such as SINTA a resistance to commercial indexes, or a localised form of the same logic?
- What does European reform of research assessment mean for universities in the Global South, when global rankings have not changed?
- How should the social impact of research written in a local language be recorded and made visible?
References
Hicks, Diana, Paul Wouters, Ludo Waltman, Sarah de Rijcke, and Ismael Rafols. 2015. 'Bibliometrics: The Leiden Manifesto for research metrics.' Nature 520: 429–431.
San Francisco Declaration on Research Assessment (DORA). 2012.
Coalition for Advancing Research Assessment (CoARA). 2022. Agreement on Reforming Research Assessment.