API

Third update on changes to our API coming in Q3 2020

Progress report on the changes coming to the API in Q3 2020
This is the THIRD update about changes to the API which will go live in September 2020. The first and second updates are also available.

  1. We will be releasing a version 2 of the whole API. All the URLs that you currently use for the API are of the form “/api/v1/{endpoint}”, these will all become “/api/v2/{endpoint}”
  2. The API routes for “/api/v1/articles” and “/api/v1/articles/{article_id}” will continue to work, but they will respond with a permanent redirect to “/api/v2/articles” and “/api/v2/articles/{article_id}”. You should not have to change your article upload integration at all during this API upgrade, though you should strongly consider moving from the v1 to v2 URL routes.
  3. The data model for Journals and Applications is changing, quite significantly. This will affect all Application CRUD interfaces, Journal retrieve and Journal/Application search APIs.
  4. Attached are two files, OutgoingAPIJournal.md.txt and IncomingAPIApplication.md.txt, which show the data models of the Application and Journal models as they will appear in the new API. Full documentation is not quite done yet, but these provide a preview of the new models.

Any questions, please ask them here or email me: dom@doaj.org

Progress report on the changes coming to the API in Q3 2020

Here is the second update about changes that we are making to our API. If you submit applications via the API or are using the API for journals search and retrieval, then you will need to adjust your integrations.

Although development is still progressing, we want to get as much information out to our API users as we can. Here are some key points:

  1. The only thing that is changing is the data model because we are changing some of the questions that we ask in our application form.
  2. Only the Journal Search and Retrieval and Applications models are changing. CRUD Articles and article search are not changing.
  3. We will have a new URL for the API. The applications and journals API calls must be sent to the new URL.
  4. We will establish a redirection for Articles calls.

We have been exploring ways to allow users to test their integrations before the new API goes live. These might include a sandbox and a whole test site for you to upload to. We will have more details on the exact testing features we can offer as we draw nearer to launch.

Any questions?

Heads up! Our API is changing.

The DOAJ is changing its application form. This means that there will be downstream implications across all of our metadata output channels, including the API.

A new version of the API will be released in Q3 2020. This new version will not be fully backwards compatible.

Those of you submitting applications via the API, or who are using the API for journals search and retrieval, will need to adjust your integrations.

Not all functions will be affected: for those of you who are uploading article metadata to us, nothing will change; there is nothing for you to do.

If you have any questions, please post them here.

Further announcements will happen here and on our Twitter feed.

Article uploads: the importance of an ISSN

This is a reminder that in February this year we tightened up procedures for article uploads. This includes checks that are carried out on uploads particularly ISSN, DOI, and full text URL information.

This means that, whether uploading article XML to us or JSON via the API, the metadata must include the exact same ISSN value(s) as those we hold in the journal record in DOAJ. If you have two ISSNs on the website and we only have one, you should contact us to have the second ISSN added. Usually, this is a quick process but sometimes you might need to contact ISSN.org to have them update their information before we can continue.

If you have any questions on this, do contact us.

RESOLVED: DOAJ IN READ-ONLY MODE 30th January

UPDATE:
Our maintenance work completed successfully yesterday and read only mode for DOAJ has been disabled. All functions are working as normal.

The DOAJ will be in read-only mode from 20:00 to 22:00 CET on Wednesday 30th January. This will affect usual functionality of DOAJ in four main areas.

There may also be a brief interruption for all users while we make DNS changes during this time.

  1. Publishers will not be able to upload any article metadata to us during that time.
  2. Publishers will not be able to submit update requests to us.
  3. No-one will be able to submit new applications to us.
  4. The API will only allow search and retrieval. Updates, creates, and deletes will not work.

There may also be a brief interruption for all users while we make DNS changes during this time.

We will update you when our operations are back to normal or if this read-only period needs to be extended.

Please contact us if you have any questions or concerns.

Are you using an up-to-date version of DOAJ metadata? Apparently not.

Did you know that DOAJ’s metadata refreshes every 30 minutes? Given how many changes we, the DOAJ Team, make to the database on a weekly basis plus all the article metadata which is continuously being uploaded to us, the hourly update is absolutely necessary. DOAJ is seeing more use of its API than ever before (2018 total: 266,255,000 hits; 2017 total: 187,212,674) and most of that is by publishers uploading articles.

DOAJ metadata is EVERYWHERE. It is incorporated into all the major discovery services, other online databases and many library catalogues.

You might remember this post of mine from 2017, where I explained that third parties using our data may not have a recent copy of it. We know for a fact that some systems downstream of our data only update the copy every 6 months and, sadly, we get many emails from end users asking why they can’t find an article.

So if you are a researcher, reading this and wondering what happened to that article in DOAJ which is listed in your library catalogue or discovery service, contact your librarians and ask them to update their metadata. If you’re a librarian then do please reach out to sources responsible for your metadata and get them to update their DOAJ metadata.

If you have any questions about this or need some help in getting a more recent copy of our metadata then do leave a comment here or contact us at feedback@doaj.org.

RESOLVED: DOAJ in read-only mode 18th-20th December

Our maintenance work completed successfully yesterday and read only mode for DOAJ has been disabled. All functions are working as normal.

From 8pm (UK/UTC) on 18th December until 8pm on 20th December (UK/UTC) DOAJ will be in read-only mode for essential maintenance. This will affect usual functionality of the DOAJ site in four main areas:

  1. Publishers will not be able to upload any article metadata to us during that time.
  2. Publishers will not be able to submit update requests to us.
  3. No-one will be able to submit new applications to us.
  4. The API will only allow search and retrieval. Updates, creates, and deletes will not work.

The read-only mode period will enable us to do some essential system and stability upgrades.

I will repost this post on 21st December, once all operations are back to normal, or if the read-only mode period needs to be extended for any reason.

If you have any questions or concerns, leave a comment here or send me email.

DOAJ. Now with added PLOS.

Finally, after saying: “It’s coming” for almost a year, DOAJ has added the entire PLOS catalogue to its database. A total of 182,500 articles were added and the harvester is set to go out and collect new articles at 5.30am every morning. The harvester collects the metadata from Europe PMC.

This is a milestone for DOAJ: not only because it has been eager to get the PLOS article metadata for a long time; but also because the harvester can be re-tuned to go and search for other large bodies of content from other publishers on E-PMC. Eventually, with some extra development, we’ll be able to point the harvester at any body of content.

If you think your publisher would benefit from this approach, do please get in touch! Thanks to our technical partners, Cottage Labs, for their help in reaching this milestone.

New article on DOAJ available at ScienceOpen

We have just published an article in the open access journal ScienceOpen Research, giving an overview of the history and current status of the DOAJ. Please read and share the article! We would welcome your participation in the post-publication peer review of this article and/or any comments.

Criteria for open access and publishing
Authors: Tom Olijhoek, Dominic Mitchell and Lars Bjornshauge
DOI: 10.14293/S2199-1006.1.SOR-EDU.AMHUHV.v1

After a brief historical overview, DOAJ policies regarding open access, intellectual property rights and questionable publishers are explained in detail. The larger part of the article is a much requested explanation on how DOAJ uses its new set of criteria for the evaluation of open access journals and the rationale behind choosing the seven extra criteria that qualify for the DOAJ Seal. A final section is devoted to the extended possibilities that DOAJ offers to scholars and publishers for searching the database and for uploading metadata.

We are getting better at ingesting metadata. Until then: give us your metadata!

I just saw a tweet about an article published on PeerJ called ‘For 481 biomedical open access journals, articles are not searchable in the Directory of Open Access Journals nor in conventional biomedical databases‘. The article concludes:

DOAJ is the most complete registry of biomedical OA journals compared with five conventional biomedical databases. However, DOAJ only indexes articles for half of the biomedical journals listed, making it an incomplete source for biomedical research papers in general.

This made me very happy, especially when I read that the authors compare DOAJ with Medline, PubMed Central, EMBASE and SCOPUS to reach their conclusion. Those are some impressive databases to be ranked alongside.

Of course, the authors’ conclusion has a very large BUT in it: DOAJ contains article metadata for only half of the journals it indexes. Why is this and what are we doing about this?

The biggest issue for us is that since our launch in 2003 providing DOAJ with article metadata has been optional. DOAJ never made it compulsory for an indexed journal to upload its metadata. DOAJ encouraged it and made it easier for publishers to provide it but it has never been a criteria for being indexed. One of the reasons for this is that there is such a huge range of publishers in DOAJ and some of them simply don’t have the resources to create and provide metadata so by making this compulsory, a very large amount of small, open access journals would have been excluded.

Another issue is that, like PMC, DOAJ requires XML to be delivered to its own DTD. For journals that are not on the Open Journal System—OJS journals have a plugin that generates DOAJ XML automatically—or for those journals that have a huge content archive, processing and converting existing content into DOAJ XML is no small task. There are cost barriers, there is a lack of understanding around XML and there is a general reluctance, particularly by the larger publishers, to convert content to yet another DTD. DOAJ has toyed with the idea of accepting more XML flavours, the most obvious being JATS, since it is a standard DTD within STM publishing. What about SSH journals, or those journals that produce XML using their own proprietary DTD? As much as DOAJ wants to facilitate article upload, with limited finding there simply isn’t one solution that will work for everyone.

However, there is good news which I highlighted in an earlier post. This year, DOAJ will launch an API and a metadata harvester. Among other things, the API will enable the bulk upload of article metadata. The metadata harvester will seek out article metadata for journals that are indexed in DOAJ.

Until we get those two features on their feet, I would absolutely URGE publishers to give DOAJ their metadata. Those who do so already said that being in DOAJ increases traffic to their sites and increases visibility of content and we know from our Google Analytics that DOAJ is the starting place for many searches for open access content. If you need help in generating XML, then contact me and I will point you to some resources that I have used to troubleshoot XML problems in the past.

Who knows? At some point in the near future Messrs Liljekvist, Andresen, Pommergaard and Rosenberg might be able to publish an update to their paper with the conclusion that: ‘DOAJ is the most complete registry of biomedical OA journals compared with five conventional biomedical databases.’ Full stop. Or even better:

DOAJ is the most complete registry of OA journals.

Onwards!