Myth-busting: Journals must meet the DOAJ Seal criteria to be indexed in DOAJ

This is a myth.

There is a common misunderstanding that for a journal to have its application accepted and be indexed in DOAJ it must meet all the criteria for the DOAJ Seal. There is an assumption, born out of that misunderstanding, that journals in DOAJ without the Seal are of inferior quality. This is also a myth.

What ‘Indexed in DOAJ’ means
Being indexed in DOAJ means that a journal application has passed our editorial review. Editorial review consists of an investigation by the DOAJ Editorial team and our volunteers who have researched whether or not the journal and its publisher do what they claim to do on the journal site and in their application to us. The investigation consists of checking all of the 50+ answers in an application to make sure that the information on the website is easy to find, clearly and accurately presented and matches the application. The editorial board is scrutinised and sometimes members of the board are contacted and their institutional connections verified, their work on the board is confirmed and which other boards that member sits on.

Being indexed in DOAJ means that the journal adheres to high levels of quality in its publishing services and services to authors and users, including: peer review, licensing terms, a strong open access statement, a fully functional editorial board and more. Being indexed in DOAJ means that the journal is a trusted open access journal and, in fact, many do meet one or more of the Seal criteria.

 

What the DOAJ Seal means
The DOAJ Seal is a stamp of quality. Journals that are awarded the Seal have verified that they correctly fulfil all seven criteria that DOAJ has chosen specifically as indicators of a Logo for the DOAJ Seal.clear commitment to open access best practices, of high levels of commitment to publishing standards and best practices, and that the open access model they adhere to is the most open.

 

The Seal has been allocated to just over 10% of the journals accepted into DOAJ since 2014. In some cases, the Seal has been withdrawn because the journal can no longer fulfil all seven criteria.

Journals cannot apply for the DOAJ Seal. The Seal is awarded after careful assessment of the journal website. If you think that your journal qualifies for the Seal, you should contact us to have it assessed.

It is important to remember that, whether they have the Seal or not, all journals in the DOAJ have met our criteria for inclusion as trusted, quality, open access journals.

Myth-busting: DOAJ takes too long to reach a decision

This is a myth.

From about 2012 until 2017, DOAJ was struggling to keep on top of the amount of applications being received.

Implementing new acceptance criteria and making 9900+ journals reapply exacerbated the problem and suddenly we had many reapplications and new applications coming in at the same time.

Triage
All applications go through an initial review to filter out incomplete or substandard applications. We call this process Triage. (From March 2015 to November 2017, Triage rejected 3112 sub-quality, incomplete or duplicate applications.)  Today, the average turnaround on an application from submission to initial review is a few days at the most.

From submission to decision
To improve the time taken to review an application and reach a decision to accept or reject, a revised and improved editorial workflow was implemented. You can read a full explanation on each of the 7 points in our progress report for 2018. The effects of those changes, which we have been monitoring carefully since 2018, are significant. 

Today we have no outstanding applications that were submitted in 2018, and only a small number dating from the first quarter of 2019 remain to be completed. We aim to reach a decision on all applications submitted within 6 months (and are still working hard to reduce that time too) although many are now completed in 3 months or less.

Even so, why does reviewing an application take time?
There are over 50 questions in our application form. Much of the work involved in reviewing an application is correspondence with the applicant and manually checking each answer. Each answer is checked for 3 things: that the answer in the form is correct; that the URL provided contains the information required in the question; that the information on the site is complete and correct.

Incommunicado
A contributing factor to the myth that DOAJ takes too long to reach a decision is the perception that DOAJ never responds. In the past, we did reject some applications without contacting the publisher about this. Since 2018, we have sent emails out for all rejected applications.

One of the most common reasons that an application is rejected is because we do not hear back from the applicant. There can be technical issues at play here: we suspect that sometimes our system-generated alerts, informing the applicant of the progress of their application, don’t reach their recipient. This is often due to particularly sensitive institutional firewalls, messages ending up in Spam folders, or email addresses no longer being valid. But it is also true that long delays in responding to DOAJ’s queries, or in making requested changes, can mean that an application is rejected.

Myth-busting: DOAJ indexes “predatory” journals

This is, of course, a myth.

Some people are afraid to use DOAJ because they believe that it lists questionable (“predatory”) journals. DOAJ started to clean up its index in 2014. DOAJ was the first service to define the standards aimed at preserving the quality and trustworthiness of a database of open access journals. Today, DOAJ’s standards are the unofficial gold standard for open access journals.

Raising the bar
Early in 2013, DOAJ’s team decided that the problem of questionable journals—at that time proliferating from India in particular—needed to be tackled.  DOAJ’s inclusion criteria were not adequate enough to filter out journals of dubious character. By October 2013, the three countries with the most journals in DOAJ were the USA, Brazil and India. In December 2013, the number of journals in DOAJ passed the 10 000 mark for the first time: the number of journals from the “USA”* (1247) and India (652) had risen sharply and faster than any other country.

Work on a new set of criteria for inclusion in DOAJ started in early 2013; this was developed, reviewed by the Advisory Board and eventually sent out for public consultation in June 2013. After extensive development work, the new application form built around the criteria was made live in March 2014. The new criteria, a work that had involved the whole open access community, increased the DOAJ application form from just 17 questions to today’s 58.

At that point, every single journal in DOAJ was made to reapply under the new criteria, to be re-indexed. This was more effective at improving the level of quality in DOAJ than we could ever anticipate.

*A typical questionable publishing trait is to pretend that a journal is registered in Global North countries, particularly the USA.

Effective criteria
DOAJ has developed rigorous checks to ensure a very high level of quality of every journal in its index.

One of the most effective checks, which delivered considerable changes in the geographic distribution of journals in the database, is insisting that a journal is listed in the country that its business activities are carried out.

Compared to other indexes that are often cited in research on scholarly publishing and open access, it is safe to say that DOAJ is probably the cleanest and most reliable, especially in the context of questionable publishing. To back up that statement, DOAJ is carrying out its research, comparing some well-known indexes. More details on that will be published here very soon.

Joint initiatives
To highlight its approach to creating a list of quality journals, to reinforce its position on questionable publishing and to emphasise the importance of standards as an effective tool to helping to identify good journals, DOAJ co-authored the Principles of Transparency and Best Practice, first released in December 2013.

DOAJ is also a founding organisation of the innovative campaign, Think. Check. Submit.

Old stains are hard to wash away
The problem of questionable publishing is vastly exaggerated. For those who still insist that DOAJ is filled with questionable journals, we would ask you to take a closer look at the database today, review our criteria and read the research on both the problem of questionable publishing and how prolific it isn’t.

If you do think that a journal in DOAJ is questionable, however, please report that journal to us so that our Questionable Publishing team can review it.

The community has chosen: introducing the new DOAJ Advisory Board

Towards the end of last year, I wrote about the new governance model that DOAJ was implementing in 2019. The first step of that process is now complete and the community has chosen a new DOAJ Advisory Board. I am absolutely delighted to introduce them here:

Leslie Chan, University of Toronto, Canada
Jan Erik Frantsvåg, The University of Tromsø – The Arctic University of Norway, Norway
Mark Hahnel, Digital Science (Figshare), UK
Rolf Halse, NSD – Norwegian Centre for Research Data, Norway
Emma Molls, University of Minnesota, USA
Anja Oberländer, University of Konstanz, Germany
Solange Maria Dos Santos, SciELO, Brazil
Lisa Schiff, California Digital Library, USA
Steven Vidovic, University of Southampton, UK

Our website will be updated with the new names and a conflict of interest statement from each of them will be added there.

The inaugural Advisory Board meeting will be held in the autumn before which a Chair person for the Board will be voted in.

The DOAJ Council will be announced in September.

Finally, I would like to thank our outgoing Advisory Board for their dedication, input and wisdom over the years:

Kevin Stranack, PKP (Public Knowledge Project)
Caren Milloy, JISC
Cameron Neylon
David Prosser, RLUK (Research Libraries UK)
Iryna Kuchma, EIFL (Electronic Information For Libraries)
Stuart Shieber, Harvard University
Martin Rasmussen, Copernicus Publications
Paul Peters, Hindawi Publishing Corporation
Arianna Becerril-García, Redalyc
Susan Murray, AJOL (African Journals Online)

Myth busting: DOAJ is not inclusive

This is a myth.

One of the most common criticisms levelled at DOAJ, particularly over the last 5 years, is that the index is not inclusive enough; that its coverage is poor; and that it lists only a fraction of the open access journals that exist. Our research shows that many journals reported as “missing” from DOAJ have a failed application or have been removed for not meeting DOAJ standards.

Diverse
It is true that DOAJ’s coverage is not complete but it does hold a very diverse set of journals from 130 countries. The top 10 countries with the most journals in DOAJ are a mixture from the global north and the global south.

Journals must apply to be indexed in DOAJ. They can be from any country, in any subject and in any language. We welcome enquiries from editors of journals who may not be sure whether or not they should apply to be indexed.

Thus far, we have rarely had to solicit applications, although that is what we will start doing in 2019. We will do this to: increase our coverage of non-English language journals, social science, humanities and arts journals, journals from the global south; and to increase coverage in key disciplines.

Apple_and_Orange_-_they_do_not_compare

Where are all the open access journals?
We are often labelled as unreliable because we don’t index all open access journals. We would remind the community that we are not an index of open access journals but an index of quality, peer-reviewed open access journals – journals which meet our basic criteria, and sometimes more.

The question of inclusiveness has to be seen in relation to other indexing services. DOAJ lists 7798 quality open access journals that are not listed in either WoS or SCOPUS. Full results of our research, comparing DOAJ against 3 indexes will be published here soon.

Some other databases seem to trawl the internet for any evidence of a “journal” without really doing any filtering or quality assessment.

Here is just a selection of the many reasons why DOAJ does not index the sometimes thousands of open access journals that can be found in other databases. DOAJ does not index journals that:

  • have ceased publication
  • haven’t published anything for up to two years
  • are hybrid
  • use a definition of open access that is not the BOAI definition
  • require people to register to access content
  • apply embargoes on full-text content
  • do not perform effective peer review (we allow editorial review only for some arts and humanities journals)
  • do not have a confirmed ISSN
  • do not have an up-to-date, transparent editorial board

Other journals may have once been in DOAJ but have been removed for various reasons, particularly because they did not reapply when DOAJ enforced stricter entry criteria, or because their website no longer functions.

We will always work with publishers to help them get their journals into DOAJ and we welcome new applications at any time!

Picture credit: By Michael Johnson – originally posted to Flickr as ‘Apples & Oranges – They Don’t Compare’ CCBY 2.0

Regarding a Delta Think blog post analysing the DOAJ

Delta Think logo

In its series Open Access News & Views, Delta Think recently published an analysis of the DOAJ. We very much enjoyed the piece and found it to be one of the most well-informed articles written about us. We would like to add to the discussion by commenting on a few of the issues raised in the article.

Comparing apples with oranges

The main problem, when you compare ROAD, Web of Science (WoS), Scopus and DOAJ, is that all of these services have different definitions and criteria as to what constitutes a valid journal entry in their databases.

In general, one can say that DOAJ’s criteria are the strictest and therefore DOAJ is not an index of all open access (OA) journals but an index of gold standard, quality, peer-reviewed OA journals. So therefore not all OA journals meet our criteria.

Being indexed in DOAJ acts like a badge of quality. A quality stamp based on the business operations of a journal and its reliability, how closely that journal adheres to best practices and which standards it uses. Scopus and WoS are not in the business of measuring any of those. (We would take this opportunity to point out that DOAJ holds many more journals which aren’t in Scopus.)

The Scopus selection process is explained here: https://www.elsevier.com/solutions/scopus/how-scopus-works/content/content-policy-and-selection

Regarding WoS you can find more information here: https://www.clarivate.com/products/web-of-science/open-access

ROAD criteria are perhaps the most relaxed: open access to all content; no moving wall or embargo; no hybrid journals; mainly research papers; the audience is mainly researchers and scholars.

We hope that together, we can achieve more transparency on inclusion criteria and we think that this would be welcomed by the community. DOAJ has an excellent working partnership with ISSN (who run the ROAD database); we are in communication with Scopus regularly; and we would welcome the opportunity to work more with Clarivate, who produce Web of Science.

We know that all these services take DOAJ metadata, either directly via OAI-PMH, our journal CSV file or our data dumps, or indirectly by using data feeds from third parties. What we have yet to establish, in the case of Scopus and ROAD, is how often they take the data from us–we believe for Scopus this could be as seldom as every six months–and how they then filter it.

Another point worth highlighting is that, to date, DOAJ has relied on publishers applying to be assessed and indexed and we have done very little soliciting of journals. However throughout 2019/2020 we will start approaching relevant OA journals not currently indexed and working with them to help them submit a quality application. This initiative is part of a funding drive for this year.

Regarding the implementation of Plan S

DOAJ expects to be involved in the implementation of Plan S and this will lead to additional criteria for journals to become Plan S compliant. In any case, we expect that Plan S will make it even more relevant for journals to be assessed and listed in DOAJ.

DOAJ reaches its SCOSS funding target within 18 months and sets its sights on new work

(Repost from the SCOSS website.)

We are delighted to announce that the Directory of Open Access Journals (DOAJ) has reached its funding target to cover its operational costs as were outlined in its SCOSS application. Eight consortia and 175 institutions/organisations from 18 different countries have committed support to DOAJ. “We’d like to thank our supporters. That this many organisations stand behind this initiative and promised this amount of funding shows how important sustaining open access infrastructure is,” said Lars Bjørnshauge, DOAJ Managing Director.

There are organisations that have already made the decision to help fund DOAJ, but are still deliberating over the details of their financial commitment. Though the funding goal has now been met, these organisations may still commit financial support. Furthermore, the SCOSS Board has allowed DOAJ to present a new work package, both to these organisations and any additional organisations that come forward in 2019. The package describes additional activities that DOAJ would like to pursue with the extra funding, should it be granted.

The Board has supported the second work package on the conditions that DOAJ reports on progress made in a progress report on the activities delineated in the SCOSS application form, and that DOAJ describes the new work package in detail and includes a budget for the amount of funding needed. For details see the financial overview and the new work package.

SCOSS endorses this plan through the end of 2019.