Metadata

Third update on changes to our API coming in Q3 2020

Progress report on the changes coming to the API in Q3 2020
This is the THIRD update about changes to the API which will go live in September 2020. The first and second updates are also available.

  1. We will be releasing a version 2 of the whole API. All the URLs that you currently use for the API are of the form “/api/v1/{endpoint}”, these will all become “/api/v2/{endpoint}”
  2. The API routes for “/api/v1/articles” and “/api/v1/articles/{article_id}” will continue to work, but they will respond with a permanent redirect to “/api/v2/articles” and “/api/v2/articles/{article_id}”. You should not have to change your article upload integration at all during this API upgrade, though you should strongly consider moving from the v1 to v2 URL routes.
  3. The data model for Journals and Applications is changing, quite significantly. This will affect all Application CRUD interfaces, Journal retrieve and Journal/Application search APIs.
  4. Attached are two files, OutgoingAPIJournal.md.txt and IncomingAPIApplication.md.txt, which show the data models of the Application and Journal models as they will appear in the new API. Full documentation is not quite done yet, but these provide a preview of the new models.

Any questions, please ask them here or email me: dom@doaj.org

Progress report on the changes coming to the API in Q3 2020

Here is the second update about changes that we are making to our API. If you submit applications via the API or are using the API for journals search and retrieval, then you will need to adjust your integrations.

Although development is still progressing, we want to get as much information out to our API users as we can. Here are some key points:

  1. The only thing that is changing is the data model because we are changing some of the questions that we ask in our application form.
  2. Only the Journal Search and Retrieval and Applications models are changing. CRUD Articles and article search are not changing.
  3. We will have a new URL for the API. The applications and journals API calls must be sent to the new URL.
  4. We will establish a redirection for Articles calls.

We have been exploring ways to allow users to test their integrations before the new API goes live. These might include a sandbox and a whole test site for you to upload to. We will have more details on the exact testing features we can offer as we draw nearer to launch.

Any questions?

We can now fix article metadata ourselves

As part of our continued strategy to enhance our metadata offering, and looking forward to 2021 when our strategic focus will be on DOAJ’s place in the discovery chain, we are constantly looking for ways to improve the quality and recency of the DOAJ article metadata.

Errors in article metadata are usually reported to us by users of discovery services at libraries or by the discovery services themselves. The errors are often a broken URL or a missing article.

Since 2013, DOAJ’s stance on correcting metadata has been to contact the journal’s publishers and ask them to send an update to us. However, this approach is of limited success:

  • some publishers don’t respond,
  • some don’t know how to make the correction and,
  • since we do matching on the URL, the process often leads to duplicated articles.

As of yesterday, DOAJ can edit article metadata itself, including the URL, directly in the database. This makes changes immediate and goes a long way to improve the reliability of the DOAJ metadata. Importantly, it allows us to act quickly on queries sent to us by our discovery service colleagues.

The crux of the matter, when it comes to the accuracy of DOAJ metadata, remains unchanged however: all 3rd party services should update their copy of the DOAJ metadata regularly. DOAJ metadata updates every 30 minutes. Some 3rd party services only update their copy of our metadata every 6 months.

Heads up! Our API is changing.

The DOAJ is changing its application form. This means that there will be downstream implications across all of our metadata output channels, including the API.

A new version of the API will be released in Q3 2020. This new version will not be fully backwards compatible.

Those of you submitting applications via the API, or who are using the API for journals search and retrieval, will need to adjust your integrations.

Not all functions will be affected: for those of you who are uploading article metadata to us, nothing will change; there is nothing for you to do.

If you have any questions, please post them here.

Further announcements will happen here and on our Twitter feed.

Our application form is changing

The current application form is now 6 years’ old. It is very long. It needs an upgrade in terms of functionality and its content needs a review. A new form will go live in Q2 2020.

The editorial team and our UX designer have carefully reviewed each question using the following criteria:

  1. Is the question needed to determine a journal’s inclusion in DOAJ?
  2. Does the question text ask exactly what we mean to ask?
  3. Does the question’s functionality allow for the correct answers to be collected?
  4. Is the question redundant after the functional upgrade?
  5. Can the question text be reduced, simplified, made clearer in the context of the new UI?

The editorial team then considered if other questions needed to be added to the form. We will be adding three new questions, (none of which are part of the Basic Criteria for inclusion):

  • the country of the society/institution;
  • whether the journal allows for ORCID integration; and
  • whether the journals’ references adhere to the I4OC standards.

At this point, I’d like to underline two Very Important Things:

  1. Our form is changing, not our criteria. Some of the questions labelled for removal are outdated; for some of them, we check the information anyway so don’t need to ask for it on the form.
  2. Any additional changes that may be required for Plan S certification have no bearing on this work.

Once the new form goes live, journals will be encouraged to update their information with us as soon as possible.

From the form as it is today, the questions that are suggested for removal are:

Question number Question text
8 Platform, Host or Aggregator
9* Name of contact for this journal
10* Contact’s email address
11* Confirm contact’s email address
19 Amount (submission charges)
20 Currency (submission charges)
21 How many research and review articles did the journal publish in the last calendar year?
22 Enter the URL where this information can be found
27 Does the journal allow software/spiders to automatically crawl the journal content (also known as text mining)?
29 Does the journal provide, or intend to provide, article-level metadata to DOAJ?
30 Does the journal provide article download statistics?
31 Enter the URL where this information can be found
32 What was the first calendar year in which a complete volume of the journal provided online Open Access content to the Full Text of all articles? (Full Text may be provided as PDFs. Does not apply for new journals.)
33 Please indicate which formats of the full text are available
54** Will the journal allow the author(s) to retain publishing rights without restrictions?
55 Enter the URL where this information can be found
56* Your name (Applicant)
57* Your email address (Applicant)
58* Confirm your email address (Applicant)

*All applicants will need to sign in or register before they can submit an application, therefore these 6 questions become redundant.

**This question is being merged with the one before it and re-worded to: ‘Does the author(s) or their institution retain both full, unrestricted copyright and publishing rights?’

Once the new form goes live, the specific answers for those questions being removed will no longer be available to download.

We welcome the community’s feedback on these changes and look forward to discussing them with you.

DOAJ será compatible con Crossref

metadata-from-crossref-logo-200En una serie de mejoras de metadatos, DOAJ aceptará la carga de XML en formato Crossref a partir del 18 de febrero de 2020.

 

En 2018, preguntamos a nuestros editores qué facilitaría su interacción con DOAJ y el 46% dijo que les gustaría que aceptásemos Crossref XML. Hoy en día sólo aceptamos XML en nuestro formato DOAJ.

Hay algunas aclaraciones que nos gustaría hacer en torno a esta mejora:

  1. No vamos a desconectar DOAJ XML. Si ya nos están suministrando DOAJ XML, pueden continuar haciéndolo. No hay cambios en este sentido.
  2. Si quieren cambiar a Crossref XML, pueden hacerlo cuando quieran. Sin embargo:
  3. No se pueden mezclar XML de Crossref y DOAJ en el mismo archivo.
  4. El  ISSN no siempre está presente en los XML de Crossref. Cuando se envía un archivo XML de Crossref a DOAJ, se deben incluir los ISSN(s) para que la carga sea exitosa.
  5. El proceso de subida de metadatos de artículos en DOAJ no cambia: por ahora, todavía tendrán que seguir enviándolos. DOAJ no cosecha los artículos automáticamente.
  6. Los XMLs de Crossref se suben desde la misma página en la que se cargan los XML con formato DOAJ. https://doaj.org/publisher/  Ahora deben elegir qué formato quieren enviarnos:
    format
  7. Las revistas editadas en plataforma OJS que usan el plugin de DOAJ para generar XML, deberían continuar haciéndolo. No hay cambios en este sentido.

Si su revista tiene contenido en el formato Crossref que les interese enviar, sólo tienen que subirnos el archivo de manera normal. Crossref facilita varias formas de recuperarsus metadatos de ellos si aún no disponen de ellos.

Se harán más anuncios en los próximos días.

DOAJ to add Crossref compatibility

metadata-from-crossref-logo-200In a series of metadata improvements, publishers will be able to upload XML in the Crossref format to us from 18th February 2020.

In 2018, we asked our publishers what would make their interaction with DOAJ easier and 46% said that they would like us to accept Crossref XML. Today we only accept XML formatted to our proprietary DOAJ format.

There are a few clarifications that we’d like to make around this improvement:

  1. We are not switching off DOAJ XML. If you are already supplying DOAJ XML to us, you may continue to do so. Nothing changes for you.
  2. If you want to switch to Crossref XML, you may do so whenever you like. However,…
  3. You cannot mix Crossref and DOAJ XML in the same file.
  4. An ISSN is usually but not always present in Crossref XML. When sending Crossref XML to DOAJ, an ISSN must be included for the upload to be successful.
  5. The process of getting article metadata into DOAJ doesn’t change: you will still need to continue to send it to us, for now.
  6. Crossref XML is uploaded from the same page that DOAJ XML is uploaded to us: https://doaj.org/publisher/ but you must now choose which format you want to send to us:
    format
  7. Publishers on the OJS platform, using the DOAJ plugin to generate XML, should continue to do so. No change is needed.

If you have back data in the Crossref format that you want to send to us, just upload the file to us in the normal way. Crossref facilitate various ways for you to retrieve your metadata from them if you don’t have it already.

Further announcements will be made next week.

 

We have removed the Seal from more than 40 journals

DOAJ constantly reviews existing records in DOAJ to ensure that they meet DOAJ criteria, particularly those with the Seal.

Recently 1432 journals in DOAJ had the Seal. Today that number is 1339. Journals that have been awarded the Seal adhere to outstanding best practice and meet to our 7 carefully chosen criteria. These criteria are indicators of high commitment to open access best practices.

In the last few weeks, we performed an in-depth review of the journals with the Seal to check they still comply with these criteria. After the review, we found out that 45 of the journals no longer met one or more of the criteria. We contacted all the journals in this situation and we are happy to communicate that we have already restored some of the Seals.

In general, the question that most journals failed on was criterium 3: “Provide article level metadata to DOAJ (Question 29). ‘No’ or failure to provide metadata within 3 months do not qualify for the Seal.” We realise that the constant supply of article metadata to DOAJ can be tricky but there are tools to help you, and soon we will be adding support for Crossref XML.

Remember: if your journal has the Seal you MUST keep article metadata up to date in DOAJ!

 

Article uploads: the importance of an ISSN

This is a reminder that in February this year we tightened up procedures for article uploads. This includes checks that are carried out on uploads particularly ISSN, DOI, and full text URL information.

This means that, whether uploading article XML to us or JSON via the API, the metadata must include the exact same ISSN value(s) as those we hold in the journal record in DOAJ. If you have two ISSNs on the website and we only have one, you should contact us to have the second ISSN added. Usually, this is a quick process but sometimes you might need to contact ISSN.org to have them update their information before we can continue.

If you have any questions on this, do contact us.

We’ve added 2 new columns to the journal CSV.

In a series of enhancements to article metadata, we have released the third today.*

If you are familiar with DOAJ metadata then you probably already know that you can download a CSV file which contains the journal-level metadata for all the journals in DOAJ. It is updated every 30 minutes and is probably one of the most popular metadata services we have. It is certainly the best way to get an overview of all the journals in DOAJ.

Today, we’ve released a new version of that file which adds two new columns to it: the number of article records added to a journal in DOAJ and the date that the last article was added. (Columns BF and BG respectively)

The columns were added for two reasons:

  1. To give greater transparency to the information which we display on our homepage whcountere we state that 77% of the journals in DOAJ have article content loaded to them. This is slightly misleading because a journal which uploaded only 5 articles to us in 2013 is counted in that 77%. It is more interesting to know how recent the articles are from a particular title and how much content they have uploaded to us. While this information has always been available by selecting a journal ISSN and using the ‘articles’ filter in Search, we’ve never been able to show all the information in one place.I think that this development will be welcomed by all our users, especially publishers, librarians and those doing research on open access publishing developments.
  2. To allow us to review those journals which have been awarded the DOAJ Seal and remove that Seal from those which are not supplying article metadata to us. Supplying article metadata is one of the 7 Seal criteria and we haven’t yet been able to check, in an efficient manner, which journals are sticking to their promise. [In the application form, we ask if journals “intend” to supply metadata to us.] It’s going to take deeper analysis to get the final figure but I can see very quickly that 25 journals are going to lose their Seal.

I’d love to know what you think about this development so, as usual, do leave a comment or a question, or email me directly: dom@doaj.org.

P.S. The other development which I will post about soon is the removal of duplicated articles from the DOAJ database. More on that when I have it.

*The first and second developments are described in this blog post from February.