Earlier this week, Nature News & Comment published a piece about the DOAJ under the heading ‘Open-access website gets tough‘.

We are happy about the exposure but there are a couple of things to address.

1) Coming to the end of a process started in December 2012

The process of drafting new, tougher criteria started way before the so-called “Science sting” which found problems in the peer review process of many of the questionable open access journals deliberately selected for the study.

IS4OA (www.is4oa.org) officially took over the responsibility for DOAJ on 1st January 2013 but it announced on 18th December 2012 that one of the most important things it would do was to ensure the implementation of stronger selection criteria for journals to be included—and to stay included—in the DOAJ. This was primarily to make it easier for authors to find a proper open access journal in which to publish their work. It makes it easier also for research funders, universities and managers of open access publication funds to make an informed decision on which open access journals comply with their policies in terms of licensing, archiving and APCs. In June 2013, after input from the DOAJ Advisory Board, I tweeted that a draft of the new criteria was available for public comment:

The draft received a lot of attention and constructive input from the community and the new application form finally went live in March this year, after a platform migration and a good deal of development.

All new journals wishing to be indexed in DOAJ and all journals indexed currently have to complete the form and then pass a much more rigorous and detailed evaluation. All the information provided by the journals will eventually be publicly available and searchable, further empowering the community to make better decisions re which open access journal to publish in and to help the DOAJ team monitor compliance. At time of writing, 231 journals have been accepted into the DOAJ under these new criteria.

Naturally, this change means much more evaluation is needed per application—multiplying current operations by a factor of 10—so DOAJ put out a call in January 2014 for voluntary editors to assist in the (re)-evaluation of the journals. The call generated 250 applications from researchers, PhDs, professors, librarians and academic publishing professionals from more than 30 countries mastering more than 30 languages. For DOAJ to achieve its vision of being a truly global service, extending coverage around the world, this is an important landmark.

With further development, DOAJ has also implemented a 3-tier evaluation process which will, as far as possible, filter out any questionable journals. This process will take time, especially since 99% of the 9939 journals have to be re-evaluated. We expect the process to be completed late 2015.

2) Even more vital to the community

The DOAJ Team knows for a fact that these efforts are taking DOAJ in the right direction that will ensure it continues to meet the needs of the public. It has already had excellent feedback on the new developments via social media channels, via training sessions held by Redalyc throughout South America and from its sponsors and supporters. It is also known that many university open access publication funds list inclusion in the DOAJ as being one of the criteria for a successful funding request. Furthermore, someone looking for a quality, peer-reviewed, open access journal in their field is more likely to start their search with the DOAJ list of journals, than with any other list; a curated list of reputable journals that uses a 3-tier review process, that harnesses the skills and expertise of the community and that requires 48 pieces of different information from an applicant before a journal can be considered for inclusion.

All the improvements above add up to a dramatic change for the DOAJ and the way it operates. It can only continue this approach if the communities that use and value DOAJ continue to support it. DOAJ has been operating entirely on financial support and is one of the oldest community funded, open access infrastructure services. Along with its sponsors, more than 100 university libraries, 15 library consortia and others already support DOAJ financially but more are needed! Become part of this impressive group of sponsors and supporters by going to http://doaj.org/supportDoaj and donating.

Leave a Reply

Your email address will not be published. Required fields are marked *


  1. Proactive DOAJ POINTS Measure Valuable Open Access Journal Characteristics Objectively

    DOAJ asks about many Open Access journal characteristics in its new “DOAJ Journal Application Form”. Many of these characteristics make a journal valuable in accordance to common belief in the Open Access community. Valuable, simple, and objective Open Access characteristics are selected from the application form to define the DOAJ POINTS. DOAJ POINTS try to measure best practice in Open Access publishing. The points have nothing to do with scholarly quality of the papers published in the journal, journal impact, prestige, or the like. I (Prof. Dieter Scholz) inspire readers, authors, editors, and publishers to calculate DOAJ POINTS for the journals of interest to them. I have contacted DOAJ representatives and DOAJ has currently no intention to implement DOAJ POINTS into their system. Maybe later? It would have been convenient, if DOAJ would do the calculation for us. However, it is not necessary, as everyone who can count to 20 can do it on her/his own.

    A journal indexed in the DOAJ can obtain between 0 and 20 DOAJ POINTS. In the same way as the DOAJ SEAL, the DOAJ POINTS follow from the entry in the DOAJ without any further human consideration. In a mathematical sense, the DOAJ SEAL as well as the DOAJ POINTS follow from a function (the DOAJ POINTS FUNCTION).

    Input to the DOAJ POINTS FUNCTION:
    Input are the entries to a journal that made it into the DOAJ based on the new application form.

    Definition of the DOAJ POINTS FUNCTION:
    DOAJ POINTS are added up according to this scheme:

    – DOAJ Question 13) 1 point, if the journal does NOT have article processing charges (APC).

    – DOAJ Question 16) 1 point, if the journal does NOT have submission charges.

    – DOAJ Question 21) 1 point, if the journal has a waiver policy for APCs and/or submission charges (for developing country authors etc). Or add an extra point here if both Question 11 and 12 are no (zero) charges.

    – DOAJ SEAL Question 23) 1 point for each archival arrangement the journal has in place with an external party (LOCKSS, CLOCKSS, Portico, …). Maximum 2 points.

    – DOAJ Question 25) 1 point, if the journal allows anyone to crawl the full-text of the journal.

    – DOAJ SEAL Question 26) 1 point for each permanent article identifier the journal has (DOI, URN, …). Maximum 2 points.

    – DOAJ SEAL Question 27) 1 point, if the journal provides, or intends to provide, article level metadata to DOAJ.

    – DOAJ Question 28) 1 point if the journal provides download statistics.

    – DOAJ Question 31) 1 point for every available full text format (PDF, HTML, …). Maximum 3 points.

    – DOAJ Question 39) 1 point, if the journal has a policy of screening for plagiarism.

    – DOAJ SEAL Question 43) 1 point, if the journal embeds or displays simple machine-readable CC licensing information in its articles.

    – DOAJ SEAL Question 45) 1 point, if the journal allows reuse and remixing of its content in accordance with a CC BY or CC BY-NC. No point is given for any other CC license (or equivalence).

    – DOAJ SEAL Question 48) 1 point, if the journal allows readers to ‘read, download, copy, distribute, print, search, or link to the full texts’ of its articles. (From the Budapest Open Access Initiative’s definition of Open Access.)

    – DOAJ SEAL Question 49) 1 point, if the journal has registered for at least one deposit policy (e.g. Sherpa/ROMEO)

    – DOAJ Question 50) 1 point, if the journal allows author(s) to hold the copyright without restrictions

    – DOAJ Question 52) 1 point, if the journal allows author(s) to retain the publishing rights without restrictions


    DOAJ POINTS are allocated to journals accepted in the DOAJ. This means DOAJ “MUST HAVE CRITERIA”, as reviewed and defined here, are already met when it comes to counting of DOAJ POINTS. “MUST HAVE CRITERIA” on their own do not produce DOAJ POINTS.

    No simple measure is provided for the additional DOAJ “SHOULD HAVE CRITERIA”, as reviewed and defined here, because “SHOULD HAVE CRITERIA” are either

    – not published in the DOAJ or
    – too fuzzy for simple counting.

    However, we learn from the rest of the questions in the new “DOAJ Journal Application Form” what else is good to have. These additional simple and objective characteristics are used above to define the DOAJ POINTS.

    A journal that has qualified for the DOAJ SEAL will have at least 6 DOAJ POINTS out of the maximum 20. In case of the DOAJ SEAL, it is either given or not (true or false). E. g. 5 points do not create a DOAJ SEAL. Advantage of the DOAJ POINTS is their higher resolution and their larger base of questions from which they are determined.

    DOAJ POINTS “Self Service”:
    DOAJ POINTS do not need to be counted officially by DOAJ or any one else. The objective nature in which DOAJ has set up its questions on the new application form combined with the strict definition of the DOAJ POINTS FUNCTION makes it possible for every one to count (or to check) DOAJ POINTS for every journal newly indexed in the DOAJ. If the DOAJ does not provide information on a particular question from above, the journal publisher has not answered the question and no point can be allocated for this characteristic. E. g. “DOAJ Question 13”: If APCs are not specified, no point is allocated. Positive characteristics have to be affirmed and stored during journal evaluation to count.

    DOAJ POINTS Reference:
    If anyone wants to publish DOAJ POINTS of a journal on the Internet, it should be done with this text:

    ‘The journal [NAME OF JOURNAL] included in the Directory of Open Access Journals has achieved [X] out of 20 DOAJ POINTS.The DOAJ POINTS have been counted by [YOUR NAME] on [yyyy-mm-dd].

    Please replace and link

    1. [NAME OF JOURNAL] with the name of the journal, linked to the journal homepage
    2. ‘Directory of Open Access Journals’ with the journal’s URL in DOAJ
    3. [X] with the number of DOAJ POINTS
    4. ‘DOAJ POINTS’ to http://doajournals.wordpress.com/2014/08/08/proactive-not-reactive/comment-page-1/#comment-41
    5. [YOUR NAME] with the name (or the organization’s name) that has counted the points and bears the responsibility for this count based on published DOAJ entries for the journal. Link it to your email address or homepage or the homepage of the responsible organisation
    6. [yyyy-mm-dd] with the date on which the DOAJ entries for the journal were counted in accordance with the DOAJ POINTS FUNCTION.

    If the reference is done in print, the same text should be used and reference with the links should be done as appropriate.

    DOAJ POINTS Background:

    – DOAJ Question 21: If there are no APCs and no submission charges there is nothing to waive. The waiver is in a way implicit in this case.

    – DOAJ SEAL Question 23: DOAJ is asking for “archival arrangements with an external party”. DOAJ is not asking for a “trusted archive”. This means that even the archive has a probability of failure. Having more than one archival arrangement in place could help, but having more than 2 of them in place may not noticeably reduce the probability of failure any more.

    – DOAJ SEAL Question 26: DOAJ is asking for permanent article identifiers. Standard seems to be the DOI. In some countries also other identifiers may be common and can ease handling. More “permanent” identifiers reduce the probability of failure of their “permanency”, but more than 2 of them may not noticeably reduce the probability of failure any more.

    – DOAJ Question 31: DOAJ is asking for available full text formats (PDF, HTML, …). PDF seems to be a must. HTML can be handy for quick viewing. Other formats will also have their advantages. More than a maximum of 3 points must not be allocated in order to avoid distorting the picture.

    DOAJ POINTS Compared to the Open Access Spectrum (OAS):
    The Open Access Spectrum – which is supported by SPARC and OASPA – embraces 6 core components related to:

    1. Reader Rights,
    2. Reuse Rights,
    3. Copyrights
    4. Author Posting Rights
    5. Automatic Posting
    6. Machine Readability.

    The OAS can be criticized (see my comments at OASPA: It confuses matters when you already deal with OA as defined by DOAJ. It is ambiguous in some of its definitions). OAS does not produce a simple number. DOAJ POINTS in contrast embrace all but one OAS components with sufficient detail and go far beyond the OAS:

    1. DOAJ “MUST HAVE CRITERIA” from Question 42)
    2. DOAJ SEAL Questions 45) and 48)
    3. DOAJ Question 50)
    4. DOAJ Question 52)
    5. not covered in DOAJ, not important
    6. DOAJ Question 43)

    DOAJ POINTS – Proactive not Reactive:
    Adding DOAJ POINTS to journals inspires our human drive and collector’s passion. As such DOAJ POINTS are positive and “white”. A reactive or “black” approach damns a journal for a single characteristic (e.g. use of a “wrong” e-mail address). The reactive approach may have many criteria, but it is binary in its outcome based only on one of its many criteria (which may not even be made public). The more criteria there are, the less likely it is to escape “blacklisting”. DOAJ POINTS, in contrast, carry the idea of the DOAJ SEAL forward. DOAJ POINTS can accompany a journal gradually as it matures and improves from minimum necessary standards.