article metadata

Walt Crawford updates his analysis of DOAJ data

Walt Crawford is prolific! As if his first tranche of mega-analysis wasn’t enough work for him, he has released an update to The Gold Open Access Landscape 2011-2014, which I wrote about previously, that includes an initial analysis of the journals that were removed from DOAJ at the shut-down of the Reapplication project.* He completed the update in May 2016: this post is long overdue.

Gold Open Access Journals 2011-2015, sponsored by SPARC, ‘provides an empirical basis for evaluating Open Access sustainability models‘. The rest of the study is based entirely on DOAJ, refining and updating the previous work. To give you a taster of what was achieved, Walt visited the web site of every journal indexed in DOAJ, even doing a second pass at those sites who hadn’t posted any content the first time around. In the age of analysing large metadata sets programmatically, with computers, this is a huge feat and one that I admire greatly.

What did Walt’s analysis reveal? Well, you can buy the paperback, a snip at only $6, directly from Walt and you can download the PDF free of charge too.

What did DOAJ learn from Walt’s analysis? We hadn’t realised quite how heavily Japan had been hit by the removal: 73% of the Japanese journals indexed in DOAJ were wiped from the Directory. We have since determined that most of those are on J-STAGE and have started a targeted campaign to get as many of those journals to submit new applications as possible. Singapore, Bangladesh and Nepal were also heavily hit. We also learned that journals published by universities and colleges suffered heavy losses. We are looking into this area with our DOAJ Ambassadors to see if we can find out why these losses were so great.

I can thoroughly recommend Walt’s work, not only for the obvious reason that he focusses so greatly on the [great!] DOAJ data but because, to my knowledge, no-one else is doing this type of analysis with such granularity.

As I write, I see that Walt has released an analysis of Open Access by Country 2011 to 2015, the 3rd part of his Gold Open Access Journals trilogy. As always, you can purchase the paperback or download the PDF for free.

Now, get reading!

*The final figure was 2851 journals removed and not 3300 as we had originally anticipated. The discrepancy was because we included, in the original count, journals that had already been removed from DOAJ and therefore had a reapplication pending but which would never be submitted.

A lot goes a long way: data quality improvement at DOAJ

Since 2012, DOAJ has been on a path of data quality improvement. DOAJ metadata is used all over the world and all over the Web. Improving and fixing the quality of our metadata can be painstaking work but the effort goes a long way as changes propagate across the Web via search engines, aggregator databases, library portals and other databases.

Along these lines, the largest publisher (in terms of the number of journals) in DOAJ recently added missing abstracts to over 100,000 articles and fixed broken special characters in approximately 2000 more. This was a huge effort on their part and DOAJ is grateful for the work that has gone into this project. It is an achievement that will be welcomed by DOAJ metadata consumers.

To date, Hindawi has 161,334 articles loaded to DOAJ and until recently was the largest contributor of metadata to our index. That title was taken from them recently when DOAJ ingested the entire PLOS archive from Europe PMC.

DOAJ. Now with added PLOS.

Finally, after saying: “It’s coming” for almost a year, DOAJ has added the entire PLOS catalogue to its database. A total of 182,500 articles were added and the harvester is set to go out and collect new articles at 5.30am every morning. The harvester collects the metadata from Europe PMC.

This is a milestone for DOAJ: not only because it has been eager to get the PLOS article metadata for a long time; but also because the harvester can be re-tuned to go and search for other large bodies of content from other publishers on E-PMC. Eventually, with some extra development, we’ll be able to point the harvester at any body of content.

If you think your publisher would benefit from this approach, do please get in touch! Thanks to our technical partners, Cottage Labs, for their help in reaching this milestone.

NEW: continuations, improved tables of contents, feedback on a journal, APC granularity, and more…

Very soon, we will be releasing new functionality on DOAJ that will improve the detail in the journal record, increase DOAJ’s accuracy even more and will encourage greater input from the community. We have also abandoned our policy to remove journals that have ceased publication.

Continuations

Ever since we migrated to our new platform, we have been trying to find a slot in our development schedule to correctly display journal continuations on our site. It was important for us to get this correct, both in terms of the metadata and the display, so that users could easily understand how one, or several, journals transformed into a journal with a new title and ISSN. This new development will allow us to correctly display a journal’s timeline as it goes through name and ISSN changes.

Send us feedback on a journal

A decreasing amount of the data in DOAJ is old and DOAJ is always grateful to receive updates from users who get in touch to alert us to broken URLs or a change of title. With this in mind, and to make feeding back as simple as possible, we will add a ‘Tell us about this journal’ button to every journal entry in DOAJ. We hope this will encourage greater input from our users and lead to more updates to journal entries. This development is perfectly aligned with DOAJ being curated by the community and really facilitates an up-to-date resource.

Greater granularity for APC information

We know. The APC information on DOAJ was misleading. It lumps together 2 distinct groups of journals into one big ‘No’ group. We have broken that group out into: No, where we have up-to-date information from journals that they DO NOT charge APCs; and No information. ‘No information’ is the group of journals which has submitted a reapplication to us and we have yet to review it. The number of ‘No information’ journals will diminish in size as we work our way through the reapplications. I wrote about this inaccuracy previously.

Retaining the records – journals that have ceased publication

As long as they fulfil some basic criteria, we will no longer remove journals that cease publishing and we will even add back into DOAJ journals that were removed only because they stopped publishing. This change of policy could potentially add hundreds of articles back into DOAJ along with tens of archived journals.

Access from China

We have had several complaints from users in China that they were unable to use DOAJ. We realised that this is because the site uses Google javascript services and Google is banned in some parts of China, or by some institutions. We have now migrated most services off Google-related services (apart from Analytics) and are confident that the user experience for users in China will be vastly improved. This is perfectly times as we embark on a year-long project to increase awareness of open access in China.

Oh, and watch this space… In early June, we’ll be adding ~175 000 PLoS articles to DOAJ… More on that later.

 

DOAJ to remove approximately 3300 journals

UPDATE (11th May): the list of removed journals is here on the 3rd tab.

Today DOAJ will remove approximately 3300 journals for failure to submit a valid reapplication before the communicated deadline; a deadline which was extended twice to allow more time for reapplications. This batch removal is another step in DOAJ’s two year long project to increase the value and accuracy of the information provided in it.

Here are some details about the reapplication project from its launch in January 2015 to today:

  • The reapplication process is a necessary step towards ensuring that all journals in DOAJ (of which there were about 10000) met the higher criteria for indexing that DOAJ launched in March 2014. The criteria were produced as a response to the maturing open access arena, the greater demands made on open access publishing by questionable journals and publishers, and to retain DOAJ’s relevancy and importance in open access publishing.
  • Some journals have been in DOAJ since 2003 and have never refreshed their information with us.
  • As of today over 5000 journals have already submitted their reapplication to us and we are busy assessing those. Many reapplications have been accepted back into DOAJ.
  • The contact for every journal to be removed from DOAJ was emailed at least 4 times, informing them of our intention to remove their journals if they failed to submit a reapplication by the agreed deadline.
  • We send email via Mailchimp and took all the necessary precautions to ensure that our emails didn’t end up in Spam, get trapped in institutional firewalls, or failed to deliver for other reasons. We used the Mailchimp authentication options to “verify” that our emails were from a genuine source.
  • The first email, announcing the reapplication project and inviting people to reapply, was sent out in January 2015 and went to publishers with 11 or more journals in DOAJ. The second email went out to publishers with 10 or less journals in DOAJ in June 2015.
  • Reminders were sent out regularly, once a month as well as announcing the deadline to our largest communities: via this blog, Twitter and Facebook.
  • To ensure that our emails ended up with the correct contact, we spent a considerable amount of time tidying up our contacts database: we updated at least 1000 records.

Removed journals are welcome to submit a new application to DOAJ at any time. They will be placed in the queue along with other applications. We will add a third tab to our spreadsheet ‘DOAJ: journals added and removed‘ that will list all of the journals removed.

When a journal is removed from DOAJ, any article metadata will also become unavailable. This is standard functionality. We are confident that the majority of the journals removed have never supplied article metadata to us, or have done once but haven’t sent us anything for at least 2 years.

If you use DOAJ as a data source and would like to do your own analysis of the journals indexed,  download our journals CSV (https://doaj.org/csv) today before 11am BST, 12pm CEST. A copy of that spreadsheet is also available here.

Deduplication of article metadata

Housekeeping! We’re doing some important data cleanup work, improving the accuracy of our data, making it more reliable and relevant for you, our users:

DOAJ is working on cleaning up article-level metadata as we remain committed to maintaining a high-quality directory. We have recently developed tools to assist with the automated pruning of spurious records. As a result you may see the number of articles in DOAJ go up (new articles continue to be accepted) and down (as spurious records are deleted). We already have processes and checks in place to ensure that no new spurious records will enter the catalogue.

Let me know if you have any questions!

We are getting better at ingesting metadata. Until then: give us your metadata!

I just saw a tweet about an article published on PeerJ called ‘For 481 biomedical open access journals, articles are not searchable in the Directory of Open Access Journals nor in conventional biomedical databases‘. The article concludes:

DOAJ is the most complete registry of biomedical OA journals compared with five conventional biomedical databases. However, DOAJ only indexes articles for half of the biomedical journals listed, making it an incomplete source for biomedical research papers in general.

This made me very happy, especially when I read that the authors compare DOAJ with Medline, PubMed Central, EMBASE and SCOPUS to reach their conclusion. Those are some impressive databases to be ranked alongside.

Of course, the authors’ conclusion has a very large BUT in it: DOAJ contains article metadata for only half of the journals it indexes. Why is this and what are we doing about this?

The biggest issue for us is that since our launch in 2003 providing DOAJ with article metadata has been optional. DOAJ never made it compulsory for an indexed journal to upload its metadata. DOAJ encouraged it and made it easier for publishers to provide it but it has never been a criteria for being indexed. One of the reasons for this is that there is such a huge range of publishers in DOAJ and some of them simply don’t have the resources to create and provide metadata so by making this compulsory, a very large amount of small, open access journals would have been excluded.

Another issue is that, like PMC, DOAJ requires XML to be delivered to its own DTD. For journals that are not on the Open Journal System—OJS journals have a plugin that generates DOAJ XML automatically—or for those journals that have a huge content archive, processing and converting existing content into DOAJ XML is no small task. There are cost barriers, there is a lack of understanding around XML and there is a general reluctance, particularly by the larger publishers, to convert content to yet another DTD. DOAJ has toyed with the idea of accepting more XML flavours, the most obvious being JATS, since it is a standard DTD within STM publishing. What about SSH journals, or those journals that produce XML using their own proprietary DTD? As much as DOAJ wants to facilitate article upload, with limited finding there simply isn’t one solution that will work for everyone.

However, there is good news which I highlighted in an earlier post. This year, DOAJ will launch an API and a metadata harvester. Among other things, the API will enable the bulk upload of article metadata. The metadata harvester will seek out article metadata for journals that are indexed in DOAJ.

Until we get those two features on their feet, I would absolutely URGE publishers to give DOAJ their metadata. Those who do so already said that being in DOAJ increases traffic to their sites and increases visibility of content and we know from our Google Analytics that DOAJ is the starting place for many searches for open access content. If you need help in generating XML, then contact me and I will point you to some resources that I have used to troubleshoot XML problems in the past.

Who knows? At some point in the near future Messrs Liljekvist, Andresen, Pommergaard and Rosenberg might be able to publish an update to their paper with the conclusion that: ‘DOAJ is the most complete registry of biomedical OA journals compared with five conventional biomedical databases.’ Full stop. Or even better:

DOAJ is the most complete registry of OA journals.

Onwards!