myth

Myth-busting: all open access journals can be listed in DOAJ

This is a myth.

People think that DOAJ exists to index all open access journals. A journal can only be indexed if it passes all of our criteria.

The Directory of Everything Open Access

The Directory of Everything open access would be a wonderful thing but of how much use would it be? DOAJ understands that users want: advice on where to publish; guidance on the reputation and credibility of a journal; peer-reviewed open access content published by quality journals. We serve a community who are interested in finding places to publish or read research for DOAJ-approved journals.

Of course, there are arguments that seeing all the open access research is better than only seeing a selection of it, but for sites which do this, where are the quality filters? In fact Unpaywall, which does aim to show as much open access content as it can, suggests to keep using DOAJ as the quality filter.

What defines an open access journal anyway?

And herein lies the problem. There are many grades, shades and colours of open access but only some of those are recognised by DOAJ. DOAJ takes its lead from the definition of openness and open access as described by the Budapest Open Access Initiative (BOAI):

By “open access” to this literature, we mean its free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited.

 There are many journals which have open access content in them but not all journals, or the articles in them, adhere to this founding definition of open access. Therefore a user will not find these journals or articles in DOAJ. Equally, open access articles from ceased journals are not included. Hybrid publishing is another example. Hybrid publishing is full of problems and there are concerns of double dipping. Essentially, hybrid publishing goes against every statement from the BOAI definition above so, from the beginning, DOAJ decided not to include hybrid journals.

DOAJ compared to SCOPUS and Web of Science

In an attempt to understand the number of OA journals which are not in DOAJ, we have recently started a detailed analysis of the journal lists of the 3 most used indexing services: SCOPUS, Web of Science (WoS) and DOAJ. As all 3 services serve to list quality journals according to a set of similar criteria, we would have expected these lists to overlap to a large extent.

However, when looking at the list of OA journals listed in DOAJ, SCOPUS and WoS[1], we found huge differences. Where DOAJ listed 12,582 journals (as of January 2019), SCOPUS listed 5920 journals and WoS only listed 4485 journals.

Annotation 2019-09-17 070103

Image by Tom Ojihoek

The reason for these discrepancies lies in the differences between the criteria for acceptance into each index; the differences in inclusion processes–DOAJ relies very much on action by the publisher to provide data–and the differences in update frequencies of each index. For instance, we found that many of the journals listed in SCOPUS but not in DOAJ had either not submitted applications to DOAJ or had submitted applications that were rejected. In other words, many of the journals that DOAJ is apparently lacking (but which are listed in SCOPUS or WoS) do not fulfill the DOAJ criteria while others are not listed in DOAJ because they simply did not apply.

Of the massive number of journals in DOAJ but which are not in SCOPUS or WoS, many are non-English journals from the Global South meaning that DOAJ’s coverage is greater, more diverse and more truly represents the global nature of open access.

It is clear from these data that not all open access journals can or should be in DOAJ.

Reference
1. Bruns, A., et al., 2019. ISSN-Matching of Gold OA Journals (ISSN-GOLD-OA) 3.0, Bielefeld University. DOI: 10.4119/unibi/2934907 We are very grateful for the work done by Bruns et al and that their raw data was available for us to analyze.

Myth-busting: Journals must meet the DOAJ Seal criteria to be indexed in DOAJ

This is a myth.

There is a common misunderstanding that for a journal to have its application accepted and be indexed in DOAJ it must meet all the criteria for the DOAJ Seal. There is an assumption, born out of that misunderstanding, that journals in DOAJ without the Seal are of inferior quality. This is also a myth.

What ‘Indexed in DOAJ’ means
Being indexed in DOAJ means that a journal application has passed our editorial review. Editorial review consists of an investigation by the DOAJ Editorial team and our volunteers who have researched whether or not the journal and its publisher do what they claim to do on the journal site and in their application to us. The investigation consists of checking all of the 50+ answers in an application to make sure that the information on the website is easy to find, clearly and accurately presented and matches the application. The editorial board is scrutinised and sometimes members of the board are contacted and their institutional connections verified, their work on the board is confirmed and which other boards that member sits on.

Being indexed in DOAJ means that the journal adheres to high levels of quality in its publishing services and services to authors and users, including: peer review, licensing terms, a strong open access statement, a fully functional editorial board and more. Being indexed in DOAJ means that the journal is a trusted open access journal and, in fact, many do meet one or more of the Seal criteria.

 

What the DOAJ Seal means
The DOAJ Seal is a stamp of quality. Journals that are awarded the Seal have verified that they correctly fulfil all seven criteria that DOAJ has chosen specifically as indicators of a Logo for the DOAJ Seal.clear commitment to open access best practices, of high levels of commitment to publishing standards and best practices, and that the open access model they adhere to is the most open.

 

The Seal has been allocated to just over 10% of the journals accepted into DOAJ since 2014. In some cases, the Seal has been withdrawn because the journal can no longer fulfil all seven criteria.

Journals cannot apply for the DOAJ Seal. The Seal is awarded after careful assessment of the journal website. If you think that your journal qualifies for the Seal, you should contact us to have it assessed.

It is important to remember that, whether they have the Seal or not, all journals in the DOAJ have met our criteria for inclusion as trusted, quality, open access journals.

Myth-busting: DOAJ takes too long to reach a decision

This is a myth.

From about 2012 until 2017, DOAJ was struggling to keep on top of the amount of applications being received.

Implementing new acceptance criteria and making 9900+ journals reapply exacerbated the problem and suddenly we had many reapplications and new applications coming in at the same time.

Triage
All applications go through an initial review to filter out incomplete or substandard applications. We call this process Triage. (From March 2015 to November 2017, Triage rejected 3112 sub-quality, incomplete or duplicate applications.)  Today, the average turnaround on an application from submission to initial review is a few days at the most.

From submission to decision
To improve the time taken to review an application and reach a decision to accept or reject, a revised and improved editorial workflow was implemented. You can read a full explanation on each of the 7 points in our progress report for 2018. The effects of those changes, which we have been monitoring carefully since 2018, are significant. 

Today we have no outstanding applications that were submitted in 2018, and only a small number dating from the first quarter of 2019 remain to be completed. We aim to reach a decision on all applications submitted within 6 months (and are still working hard to reduce that time too) although many are now completed in 3 months or less.

Even so, why does reviewing an application take time?
There are over 50 questions in our application form. Much of the work involved in reviewing an application is correspondence with the applicant and manually checking each answer. Each answer is checked for 3 things: that the answer in the form is correct; that the URL provided contains the information required in the question; that the information on the site is complete and correct.

Incommunicado
A contributing factor to the myth that DOAJ takes too long to reach a decision is the perception that DOAJ never responds. In the past, we did reject some applications without contacting the publisher about this. Since 2018, we have sent emails out for all rejected applications.

One of the most common reasons that an application is rejected is because we do not hear back from the applicant. There can be technical issues at play here: we suspect that sometimes our system-generated alerts, informing the applicant of the progress of their application, don’t reach their recipient. This is often due to particularly sensitive institutional firewalls, messages ending up in Spam folders, or email addresses no longer being valid. But it is also true that long delays in responding to DOAJ’s queries, or in making requested changes, can mean that an application is rejected.

Myth-busting: DOAJ indexes “predatory” journals

This is, of course, a myth.

Some people are afraid to use DOAJ because they believe that it lists questionable (“predatory”) journals. DOAJ started to clean up its index in 2014. DOAJ was the first service to define the standards aimed at preserving the quality and trustworthiness of a database of open access journals. Today, DOAJ’s standards are the unofficial gold standard for open access journals.

Raising the bar
Early in 2013, DOAJ’s team decided that the problem of questionable journals—at that time proliferating from India in particular—needed to be tackled.  DOAJ’s inclusion criteria were not adequate enough to filter out journals of dubious character. By October 2013, the three countries with the most journals in DOAJ were the USA, Brazil and India. In December 2013, the number of journals in DOAJ passed the 10 000 mark for the first time: the number of journals from the “USA”* (1247) and India (652) had risen sharply and faster than any other country.

Work on a new set of criteria for inclusion in DOAJ started in early 2013; this was developed, reviewed by the Advisory Board and eventually sent out for public consultation in June 2013. After extensive development work, the new application form built around the criteria was made live in March 2014. The new criteria, a work that had involved the whole open access community, increased the DOAJ application form from just 17 questions to today’s 58.

At that point, every single journal in DOAJ was made to reapply under the new criteria, to be re-indexed. This was more effective at improving the level of quality in DOAJ than we could ever anticipate.

*A typical questionable publishing trait is to pretend that a journal is registered in Global North countries, particularly the USA.

Effective criteria
DOAJ has developed rigorous checks to ensure a very high level of quality of every journal in its index.

One of the most effective checks, which delivered considerable changes in the geographic distribution of journals in the database, is insisting that a journal is listed in the country that its business activities are carried out.

Compared to other indexes that are often cited in research on scholarly publishing and open access, it is safe to say that DOAJ is probably the cleanest and most reliable, especially in the context of questionable publishing. To back up that statement, DOAJ is carrying out its research, comparing some well-known indexes. More details on that will be published here very soon.

Joint initiatives
To highlight its approach to creating a list of quality journals, to reinforce its position on questionable publishing and to emphasise the importance of standards as an effective tool to helping to identify good journals, DOAJ co-authored the Principles of Transparency and Best Practice, first released in December 2013.

DOAJ is also a founding organisation of the innovative campaign, Think. Check. Submit.

Old stains are hard to wash away
The problem of questionable publishing is vastly exaggerated. For those who still insist that DOAJ is filled with questionable journals, we would ask you to take a closer look at the database today, review our criteria and read the research on both the problem of questionable publishing and how prolific it isn’t.

If you do think that a journal in DOAJ is questionable, however, please report that journal to us so that our Questionable Publishing team can review it.

Myth busting: DOAJ is not inclusive

This is a myth.

One of the most common criticisms levelled at DOAJ, particularly over the last 5 years, is that the index is not inclusive enough; that its coverage is poor; and that it lists only a fraction of the open access journals that exist. Our research shows that many journals reported as “missing” from DOAJ have a failed application or have been removed for not meeting DOAJ standards.

Diverse
It is true that DOAJ’s coverage is not complete but it does hold a very diverse set of journals from 130 countries. The top 10 countries with the most journals in DOAJ are a mixture from the global north and the global south.

Journals must apply to be indexed in DOAJ. They can be from any country, in any subject and in any language. We welcome enquiries from editors of journals who may not be sure whether or not they should apply to be indexed.

Thus far, we have rarely had to solicit applications, although that is what we will start doing in 2019. We will do this to: increase our coverage of non-English language journals, social science, humanities and arts journals, journals from the global south; and to increase coverage in key disciplines.

Apple_and_Orange_-_they_do_not_compare

Where are all the open access journals?
We are often labelled as unreliable because we don’t index all open access journals. We would remind the community that we are not an index of open access journals but an index of quality, peer-reviewed open access journals – journals which meet our basic criteria, and sometimes more.

The question of inclusiveness has to be seen in relation to other indexing services. DOAJ lists 7798 quality open access journals that are not listed in either WoS or SCOPUS. Full results of our research, comparing DOAJ against 3 indexes will be published here soon.

Some other databases seem to trawl the internet for any evidence of a “journal” without really doing any filtering or quality assessment.

Here is just a selection of the many reasons why DOAJ does not index the sometimes thousands of open access journals that can be found in other databases. DOAJ does not index journals that:

  • have ceased publication
  • haven’t published anything for up to two years
  • are hybrid
  • use a definition of open access that is not the BOAI definition
  • require people to register to access content
  • apply embargoes on full-text content
  • do not perform effective peer review (we allow editorial review only for some arts and humanities journals)
  • do not have a confirmed ISSN
  • do not have an up-to-date, transparent editorial board

Other journals may have once been in DOAJ but have been removed for various reasons, particularly because they did not reapply when DOAJ enforced stricter entry criteria, or because their website no longer functions.

We will always work with publishers to help them get their journals into DOAJ and we welcome new applications at any time!

Picture credit: By Michael Johnson – originally posted to Flickr as ‘Apples & Oranges – They Don’t Compare’ CCBY 2.0