progress report

2019 in review and what’s coming in 2020

It’s a new year and a new decade. This year, DOAJ turns 17 and much like a teenager, we have seen a major growth spurt over the last 4 years, peaking at a total of 14.25 million unique users at the end of 2019. 2020 will see further transformation into a maturer, more rounded service.

A chart showing the growth of DOAJ from 2015 to 2019 in millions of unique users

Usage of DOAJ, 2015 to 2019 (Source: Google Analytics)

2019

2019 was another busy year, focussed on preparing DOAJ for adulthood. We added muscle to the bones by increasing our capacity in the platform and in our editorial team and thereby increasing our throughput. Never before has DOAJ been such a lean operation on such a large scale.

Much of the process to get to where we are today was written about in September, in a guest post by Richard Jones from Cottage Labs, so I’m just focussing on the last quarter here.

Since October 2019, we updated the DOAJ search interface–a program called Edges–and upgraded to Python 3. We extended our automated testing scripts and fixed a variety of bugs or discrepancies which they subsequently uncovered.

I am pleased to say that these pieces of work have been so successful that in 2019 we had 99.999% uptime, a huge improvement on previous years.

On the front end, we added more translations of the COPE/DOAJ/OASPA/WAME co-authored Principles of Transparency and Best Practice in Scholarly Publishing which is now available in 20 different languages. We cleaned up our articles database by removing many old, malformed or duplicated entries.

We also improved our crawler which is pointed at Europe PMC’s API and added 13,200 PLoS articles which were missing from the database.

The start of the redesign project saw us focussing our efforts on design research:

  • getting in touch with users
  • doing exhaustive inventories of our content and information architecture
  • understanding DOAJ’s offers and services on a deeper level
  • setting realistic goals and metrics for the website’s design
  • benchmarking / looking at how other open access orgs and not-for-profits present themselves
  • finding visual inspiration from external sources / moodboarding
  • setting ourselves up for a more modular design mindset to facilitate development of new services on DOAJ.org

We also inaugurated our Advisory Board and Council.

2020

During the first quarter of 2020, the following pieces of work will be released live to the site:

  • Some additional language codes will be added to our XSD file.
  • A Spam reCAPTCHA module is being added to the feedback form.
  • The form for publishers to manually provide article metadata to us is getting an overhaul.
  • The addition of Crossref XML compatibility so publishers can send us Crossref XML if they wish.
  • A reconstructed, redesigned application form with improved functionality and usability.

Away from the platform, we are extending our partner program, after what is proving to be a successful pilot project with our friends at TSV in Finland.  More projects like this will result in an increase in the number of journals indexed in DOAJ, the goal of the work we wrote about when we reached our first SCOSS funding target in May 2019.

The aim of these projects is to:

  • Maintain the recency and relevancy of the database by targetting specific journals which are not yet indexed in DOAJ
  • Recruit back into DOAJ as many of the 2850 journals as possible which we removed during the Reapplication Project,
  • Increase our coverage of journals from the Arts, Social Sciences and Humanities.
  • Increase our coverage of non-English language journals.
  • Increase our coverage of journals from specific subject areas.

DOAJ will also continue its work with its partner program Think.Check.Submit which is developing it tools and resources.

If you have any questions about any of this work, do please leave a comment!

I wish all our readers a happy and healthy 2020.

DOAJ Progress Report for 2018

A very Happy New Year to all our readers. We wish you all the best for 2019. It’s going to be an exciting year for DOAJ and for open access. Much of the work which you can read about below will come to fruition in 2019 , as well as new opportunities presenting themselves.

With a revised set of Principles of Transparency and Best Practice and a new mission,  DOAJ started 2018 by publishing its strategy to show the community where DOAJ is focussing its efforts: a) funding and sustainability; b) functionality, stability and scalability; c) education and outreach. 

Financially, DOAJ has seen the benefits of the SCOSS initiative, with more than 60% of all monies being donated from the public sector:

Income 2017 and 2018 per category in £ (GBPs)  
 2017 2018
Libraries89,00026%282,86746%
Library Consortia86,00026%146,59024%
Research Funders31,0009%45,9008%
Smaller Publishers8,0002%4,4711%
Sponsors123,00036%129,55521%
     
Total337,000 609,383 

Below are details of the major projects that we have undertaken in 2018. The terms in brackets correspond to the three arms of the strategy. If you have any questions about them, or would like to know more, please get in touch: dom@doaj.org.The full details of DOAJ finances are always posted on the IS4OA website.

1. Faster evaluation of journals (funding and sustainability)

We have made excellent progress in this area and this will be of real interest to many of our stakeholders, particularly the publishers. For the first time since before 2013, we do not have a backlog of applications waiting to be triaged*. Here are the changes we made which led to this excellent result:

From mid-December 2017 to mid-December 2018, Triage rejected without review over 2000 poorly completed applications, removing them from the system so that they wouldn’t end up with the editorial teams.

  • We carried out a full editorial workflow review to identify bottlenecks and areas where we could become more efficient. The goal was to make the customer experience smoother and more uniform, and cut down turnaround times.   
  • We have created a monthly report which identifies applications which are stuck (we haven’t received a response from the journal contact), or which are reaching an age of 6 months. This has helped to focus activities on those applications which need it the most.
  • *We expanded our “Triage process” to include many more preliminary checks and have re-organised how Triage assigns applications to editors for review. (The purpose of Triage is to filter out all poor quality, incorrect or incomplete applications so that they do not clog up the editorial queues.) This has resulted in faster turnaround times, since fewer low quality applications are making their way through Triage.
  • We introduced a ‘Quick Reject’ email feature which allows journals rejected at Triage to get an instant notification that their application has failed Triage. The automated email contains the reason for rejection. Prior to this feature some applicants were having to wait before getting an answer that their application had fallen at the first hurdle.
  • We made it easier for quality applications to be allocated directly to a reviewer, rather than sitting in a holding pattern waiting to be processed. We did this by creating a method of grouping applications and enabling Triage to allocate all of them in the same way.
  • We have reduced the amount of people that an account holder comes into contact with. This improves transparency and also cuts down turnaround times.
  • We have strengthened API capabilities so that any account holder can send us applications in bulk automatically via the API. The same goes for uploading or correcting article metadata and updating journal metadata.

The primary goal of these changes is to free up resource, resource which is better focussed on reviewing and processing the “good” applications. This project has been incredibly effective and we are very happy with the results here.

2. Improved monitoring of journal compliance and systemic weeding campaign (funding and sustainability)

The introduction of an update function allowed us to make systematic journal entry reviews more focussed and more effective. These are undertaken as each update is submitted. Further reviews are taken across our larger multi-journal accounts where, as far as possible, we have tried to establish common metadata entries across all journals belonging to the same publishing entity.

As well as manual reviewing, we have undertaken 2 major automated reviews of our data: one for broken URLs and one for erroneous ISSNs. Both revealed problematic entries in the database which we were then able to target and either correct or remove. This was a new approach for us and one which is easily repeatable at any point in the future.

The result of this work is that we can categorically state that the metadata is purer and cleaner than it has ever been before, both at journal and article level. The metadata is now more correct and up-to-date than it has ever been. For example, an automated review of URLs in the database recently revealed that 121 journals in the database had broken URLs. That’s just 1% of all journals in DOAJ.

3. Enable greater collaboration with organisations to enrich metadata (education and outreach)

Much of the ability to enrich metadata must come after the refactoring work (see point 7 below) is complete. However, this hasn’t stopped us making a little progress in this area. We had already met with Altmetric in February 2017 to discuss the integration of alternative metrics into DOAJ. A sticking point then which limited the usefulness of such a feature is that a large amount (maybe almost 50%) of the article metadata had no DOI. We are working to improve this figure.

In 2018, DOAJ renewed its partnerships with Research4Life, ISSN ROAD, OCLC and Think.Check.Submit. DOAJ entered into new partnerships with Digital Science, Library Publishing Coalition, Creative Commons, Science Afrique and C4DISC. We are also working closely with the OJS team since so many of the journals in DOAJ use the OJs software.

4. Strengthen coverage and diversity in terms of language and geography, including an extended Ambassador programme and translated materials (education and outreach)

This is another area in which we have made excellent progress. We have undertaken the following initiatives:

  • Acquisitions: a common criticism of DOAJ is that it doesn’t include every open access journal. We have done work to identify groups of quality, peer-reviewed open access journals and are in the middle of assessing each journal and contacting them. We hope to entice new journals to DOAJ, or journals which didn’t remain in the database after the Reapplication project.

    One thing that has become clear very quickly is that a common reason for these journals to not be in DOAJ is that they simply do not meet our criteria. Another reason is that some non-English journals do not want to be in an English-based  website. See the Translation item below.

  • Video tutorials: we have published the first in a series of video tutorials. The videos are tailor-made, addressing areas and themes where we receive the most questions or feedback from users, including account holders. We have collected together ideas for at least 10 more tutorials.
  • Cognitive justice: DOAJ adopted the theme of cognitive justice into its strategy. We have attended several events dedicated to this topic. We commissioned a guest post on our blog, we have addressed the issue at a workshop and we have worked the theories of cognitive justice into our materials and modus operandi. We are also a founding partner in setting up a network for the promotion of French language journals and platforms called LIRAJ.  
  • Workshops in Indonesia and Korea: 3 countries were identified as target areas after our Reapplications project completed: Indonesia, Korea and Japan. DOAJ has held workshops in Indonesia and Korea to encourage quality applications from these countries. We have recruited and trained volunteers as Associate Editors and Ambassadors in both countries. We are deferring activities in Japan until 2019 to allow for the work in Indonesia and Korea to bed in.
  • Translated content to increase accessibility to DOAJ: in a coordinated effort with OASPA, WAME and COPE, DOAJ has led an exercise to translate the Principles of Transparency and Best Practice in Open Access Publishing. To date, we have received translations in 16 languages which will be published on the 4 partner websites. Nine languages are currently live and the first two translations, Spanish and Portuguese, were published to coincide with the 20 Year Anniversary of Scielo. We have also installed Google Translate on the DOAJ website as a first step to help localising some of the web site content. As with all automated translation tools, we know that it is far from perfect so we will begin working on further manual website translations in 2019.
  • Questionable publishing: after a coordinated campaign from several countries’ national newspapers to portray the problem of questionable publishing in an exaggerated light, DOAJ has been instrumental in putting the problem into the correct perspective with blog posts and courses. We have more work to do in this area!
5. Governance (funding and sustainability)

The development and implementation of a governance structure is almost complete. In December, DOAJ published its new governance model which was created with input from the existing advisory board. The new governance structure enables all of those organisations who fund DOAJ to nominate individuals from their organisation to sit on the DOAJ board. DOAJ will be announcing the details of nominations and elections later this month.

6. Account holder survey  (funding and sustainability)

We sent out a survey to the community of 6000+ DOAJ account holders asking for their opinions on what we should be developing next, benefits of being indexed in DOAJ, technologies that they employ etc. We had 1065 responses with a response rate of 25% which is excellent for an online survey. The results from the survey and some deeper analysis will be published later this month.

7. Refactoring Project (functionality, stability and scalability)

Last but certainly not least, we have undertaken a huge amount of work to stabilise the foundation that the DOAJ database sits on. It’s tecchy to a level which certainly won’t appeal to everyone but it is unarguably the most important piece of work that we have done this year and, it turns out, already puts us in an excellent position for whatever 2019 and 2020 will throw at us.

DOAJ receives more than 75,500 hits per day to the site. The API received 130,804,030 hits in the first 6 months of 2018 alone. The database holds 4.3 million pieces of article metadata

In 2014, when DOAJ was relaunched on its current platform, the database was receiving about 22,000 hits per day. Today, we are seeing figures upwards of 75,500 per day (averaging 1.2 million hits per month), plus an increase in traffic from referrals (75% of all DOAJ traffic comes via referrals), plus huge amounts of activity via our API (2018: 266,255,000 hits; 2017 total: 187,212,674). In 2014, the database held about 2 million pieces of article metadata; today it is about 4.3 million. The need for a holistic, strategic approach was very real to ensure that there would be no surprises along what is turning out to be a 3-year project.

In December 2017, DOAJ and its technical partner, Cottage Labs, finalised the list of tasks. To date, we have completed 20 of the 40 original tasks, although the remaining 20 tasks are bigger and meatier chunks of work which will take us well into 2019 and beyond. If you’d like details, let me know.

Some other things that we have done to ensure that we have a stable and scalable back-end:

  • Upgrades required to protect DOAJ from the Meltdown and Spectre vulnerabilities.
  • Machine snapshots, security updates, and resize operations across 4 servers
  • Increased our number of developer days at our technical partner to 20 days per month which was seen as a necessary increase if DOAJ was to achieve everything that it sets out to do in its strategy document.
  • Made data at DOAJ secure, following the recommendations laid out by the GDPR which came into effect in May.
  • Limited unintended access to the query endpoint.
  • Greater integration into Google Analytics.

NEW: continuations, improved tables of contents, feedback on a journal, APC granularity, and more…

Very soon, we will be releasing new functionality on DOAJ that will improve the detail in the journal record, increase DOAJ’s accuracy even more and will encourage greater input from the community. We have also abandoned our policy to remove journals that have ceased publication.

Continuations

Ever since we migrated to our new platform, we have been trying to find a slot in our development schedule to correctly display journal continuations on our site. It was important for us to get this correct, both in terms of the metadata and the display, so that users could easily understand how one, or several, journals transformed into a journal with a new title and ISSN. This new development will allow us to correctly display a journal’s timeline as it goes through name and ISSN changes.

Send us feedback on a journal

A decreasing amount of the data in DOAJ is old and DOAJ is always grateful to receive updates from users who get in touch to alert us to broken URLs or a change of title. With this in mind, and to make feeding back as simple as possible, we will add a ‘Tell us about this journal’ button to every journal entry in DOAJ. We hope this will encourage greater input from our users and lead to more updates to journal entries. This development is perfectly aligned with DOAJ being curated by the community and really facilitates an up-to-date resource.

Greater granularity for APC information

We know. The APC information on DOAJ was misleading. It lumps together 2 distinct groups of journals into one big ‘No’ group. We have broken that group out into: No, where we have up-to-date information from journals that they DO NOT charge APCs; and No information. ‘No information’ is the group of journals which has submitted a reapplication to us and we have yet to review it. The number of ‘No information’ journals will diminish in size as we work our way through the reapplications. I wrote about this inaccuracy previously.

Retaining the records – journals that have ceased publication

As long as they fulfil some basic criteria, we will no longer remove journals that cease publishing and we will even add back into DOAJ journals that were removed only because they stopped publishing. This change of policy could potentially add hundreds of articles back into DOAJ along with tens of archived journals.

Access from China

We have had several complaints from users in China that they were unable to use DOAJ. We realised that this is because the site uses Google javascript services and Google is banned in some parts of China, or by some institutions. We have now migrated most services off Google-related services (apart from Analytics) and are confident that the user experience for users in China will be vastly improved. This is perfectly times as we embark on a year-long project to increase awareness of open access in China.

Oh, and watch this space… In early June, we’ll be adding ~175 000 PLoS articles to DOAJ… More on that later.

 

DOAJ to remove approximately 3300 journals

UPDATE (11th May): the list of removed journals is here on the 3rd tab.

Today DOAJ will remove approximately 3300 journals for failure to submit a valid reapplication before the communicated deadline; a deadline which was extended twice to allow more time for reapplications. This batch removal is another step in DOAJ’s two year long project to increase the value and accuracy of the information provided in it.

Here are some details about the reapplication project from its launch in January 2015 to today:

  • The reapplication process is a necessary step towards ensuring that all journals in DOAJ (of which there were about 10000) met the higher criteria for indexing that DOAJ launched in March 2014. The criteria were produced as a response to the maturing open access arena, the greater demands made on open access publishing by questionable journals and publishers, and to retain DOAJ’s relevancy and importance in open access publishing.
  • Some journals have been in DOAJ since 2003 and have never refreshed their information with us.
  • As of today over 5000 journals have already submitted their reapplication to us and we are busy assessing those. Many reapplications have been accepted back into DOAJ.
  • The contact for every journal to be removed from DOAJ was emailed at least 4 times, informing them of our intention to remove their journals if they failed to submit a reapplication by the agreed deadline.
  • We send email via Mailchimp and took all the necessary precautions to ensure that our emails didn’t end up in Spam, get trapped in institutional firewalls, or failed to deliver for other reasons. We used the Mailchimp authentication options to “verify” that our emails were from a genuine source.
  • The first email, announcing the reapplication project and inviting people to reapply, was sent out in January 2015 and went to publishers with 11 or more journals in DOAJ. The second email went out to publishers with 10 or less journals in DOAJ in June 2015.
  • Reminders were sent out regularly, once a month as well as announcing the deadline to our largest communities: via this blog, Twitter and Facebook.
  • To ensure that our emails ended up with the correct contact, we spent a considerable amount of time tidying up our contacts database: we updated at least 1000 records.

Removed journals are welcome to submit a new application to DOAJ at any time. They will be placed in the queue along with other applications. We will add a third tab to our spreadsheet ‘DOAJ: journals added and removed‘ that will list all of the journals removed.

When a journal is removed from DOAJ, any article metadata will also become unavailable. This is standard functionality. We are confident that the majority of the journals removed have never supplied article metadata to us, or have done once but haven’t sent us anything for at least 2 years.

If you use DOAJ as a data source and would like to do your own analysis of the journals indexed,  download our journals CSV (https://doaj.org/csv) today before 11am BST, 12pm CEST. A copy of that spreadsheet is also available here.

Reblog: Top 10 publishers in DOAJ (by number of titles) 2014 to 2015

We’re a big fan of the work of the Sustaining Knowledge Commons team. The post below, published yesterday, takes a look at the change in the Top 10 list of publishers indexed in DOAJ from 2014 to 2015.

Source: Top 10 publishers in DOAJ (by number of titles) 2014 to 2015

In her post, Heather says: “The percentage of journals in DOAJ published by the top 10 publishers has increased slightly, from 14% to 16%.” I think there are 3 main reasons why:

1) We have seen a renewed interest in DOAJ from large publishers since we relaunched in 2013 and increased our visibility and transparency (expanded our communications and activities, launched the updated application form and started the Reapplications project).

2) We are now processing applications faster than ever before and part of that is due to the streamlining of the process for ‘bulk’ applications from multi-journal publishers, increasing our capacity but leaving us more time for the smaller publishers.

3) These publishers have launched more open access titles.

We are in regular discussion with 8 out of the 10 publishers listed in the blog, mostly as they update their information with us (Medknow to Wolters Kulwer Medknow) and submit their reapplications.

DOAJ adds two new members of staff to its team

On the back of my last blog post, I am delighted to announce two new members joining the DOAJ team: Judith Barnsby and Alejandra Manco Vega who both join us as Publication Specialists.

The employment of Judith and Alejandra is to a large extent a direct result of the substantial funding that has been facilitated by Austria’s FWF, BMWFW and several Austrian universities and research centres. This means that DOAJ has even more resource and this is specifically dedicated to reviewing reapplications. The benefits will be felt immediately: Alejandra will focus on Spanish and Portuguese reapplications, Judith on reapplications from multi-journal publishers.

Judith, based in the UK, comes to us armed with a great deal of publishing experience and is an expert in publishing technologies and standards, having previously been with HighWire, and is on the board for CLOCKSS. Alejandra, originally from Peru but now based in Sweden, comes with librarianship and language skills. They bring the total number of dedicated DOAJ Team members to seven, on top of the 100 or so volunteers who donate their time to us every week.

On behalf of the entire DOAJ Team, I welcome warmly Judith and Alejandra. We are delighted to have you with us!

Development plan for 2015/6

Recent discussions with a partner reminded me that we haven’t published a development plan.

I’ve put together a high level list of our developments: what we are currently doing and what we have next on the list. As it stands, it will take us through to mid-2016 but it is a very fluid document. We have the flexibility to add in items and move others so the 2016 items will be revisited in January 2016 and reviewed to make sure nothing else has cropped up in the meantime.

Some of these items I have already announced in earlier posts. I’ve collected those here.

As always, I’d welcome your feedback.

Updates to DOAJ will be switched off while we upgrade the site

Earlier this month I wrote about the twelve developments that we were working on. These are now ready to be released to the live site this week and we will be making the upgrade between Tuesday (21st April) evening and Wednesday (22nd April) morning British Summer Time (GMT+1). During this time, we will switch off all parts of the site that allow new data to be submitted to us:

  • the application form
  • the form for adding article metadata
  • the XML upload page for uploading article metadata
  • the CSV upload page for submitting bulk reapplications
  • the reapplication form

All other aspects of the site will continue to function as normal. We expect a full service to be reinstated by Wednesday 12 noon BST. I will post further updates here.

Greater visibility to APCs: amount, currency, URL

There has been a lot of focus in research on author processing charges (APCs) and submission charges, particularly in the last 16 months or so and DOAJ data is often used as a basis of that research.  Heather Morrison’s recent article in Publications and Walt Crawford’s research published in Cites and Insights are two very recent examples.

DOAJ wants to raise the visibility of charges information even further to facilitate future research and to make it easier for authors, researchers and funders to make informed decisions on where to publish. As part of our commitment to raising the level of quality of data in DOAJ, we released yesterday a small but important change to the display of charging information. All journals accepted into DOAJ after March 2014, or back into DOAJ after a successful reapplication, will have the following information displayed against them:

  • Does the journal have APCs or Submission charges?
  • If so, how much and what is the currency of those charges?
  • What is the URL where that information is clearly displayed and stated on the journal web site?
  • If there are no charges, what is the URL where that information is clearly displayed and stated on the journal web site?

During our review of applications we request that ‘no charges’ is stated explicitly on the journal’s site and we will ask publishers to add that information if they have not already done so.

You will find the new information on each journal’s table of contents page; that is to say the long, detailed view of all the information and metadata that we hold for a journal accessible by clicking a journal’s title in search results. Two examples would be here where the journal has no charges, or here where the journal has APCs.

There are further improvements in the pipeline: we will move the information above the [more detail] link on these pages; we will add charge information to all records in search results; we will include amount and currency in our downloadable CSV file; and we will point the Publication Charges facet in search to the new data. These changes are scheduled for completion in April.