development plan

Guest post: a technical update from our development team

This is a guest post by Richard Jones, founding partner of Cottage Labs and member of the DOAJ team. Cottage Labs has hosted, developed and managed the DOAJ platform since December 2013 and is responsible for keeping DOAJ available to the vast number of individuals using DOAJ every day.

To the public, it may seem that not a lot has changed at doaj.org for the past year or so but in the background, a lot of work has been going on to prepare for some major improvements.

January – August 2019

During this period, our technical focus has been on 3 major areas: the Application Form; the editorial workflow system, which underpins the application process; and the User Interface (UI). In addition, we have been carrying out the final bits of work to improve the stability and scalability of the system, with the net result that in those 8 months there was only 3 minutes of unscheduled down-time.

The team measures its throughput via the number of issues that are successfully dealt with per month in our GitHub issue tracker. On average we’re handling 20-25 issues per month, some of which are support questions. These questions come from a variety of sources including DOAJ team members, end users, or from technical users of the API and other machine interfaces.

We’ve also been working with a new performance monitor to identify bugs, and for the first time we are able to detect issues with the system that go unreported or even unnoticed by end users.

Here are some of the minor improvements we’ve made:

  • Improved API documentation
  • Further GDPR compliance: cookie consent banner; marketing opt in/out preferences for users; and anonymisation of data used in testing and development
  • Data about articles in the Journal CSV file
  • A preliminary overhaul of the site’s layout template and CSS, in preparation for a much larger UI upgrade next year.

Here are some major bits of work that we have carried out:

  • Enhancements to our historical data management system. We track all changes to the body of publicly available objects (Journals and Articles) and we have a better process for handling that.
  • Introduced a more advanced testing framework for the source code. As DOAJ gains more features, the code becomes larger and more complex. To ensure that it is properly tested for before going into production, we have started to use parameterised testing on the core components. This allows us to carry out broader and deeper testing to ensure the system is defect free.
  • A weekly data dump of the entire public dataset (Journals and Articles) which is freely downloadable.
  • A major data cleanup on articles: a few tens of thousands of duplicates, from historical data or sneaking in through validation loopholes, were identified and removed. We closed the loopholes and cleaned up the data.
  • A complete new hardware infrastructure, using Cloudflare. This resulted in the significant increase in stability mentioned above and allows us to cope with our growing data set (increasing at a rate of around 750,000 records per year at this point).

And here are some projects we have been working on which you will see come into effect over the next few weeks:

  • A completely new search front-end. It looks very similar to the old one, but with some major improvements under-the-hood (more powerful, more responsive, more accessible), and gives us the capability to build better, cooler interfaces in the future.
  • Support for Crossref XML as an article upload format. In the future this may also be extended to the API and we may also integrate directly with Crossref to harvest articles for you. We support the current Crossref schema (4.7) and we will be supporting new versions as they come along.

Finally, we welcomed a new developer to our team, Aga, who joined Cottage Labs and the DOAJ team in July of this year.

Taking a longer view, developments coming down the pipe in the next 6-8 months or so are:

  • A major overhaul to the UI, following extensive design and user experience work by DOAJ’s UX consultant.
  • A lot of work on the editorial back-end (so you might not notice much change on the public side) to improve the throughput and usability of the system for the editors and administrators.
  • A new, revamped application form, which will be easier to use and offer you better support in applying to DOAJ or updating your existing Journals.

If you have any questions or would like more detail on anything you have read here, do please contact us or leave a comment here.

Sponsorship model and pricing for 2019

Last year, we launched our new sponsorship model. This was after feedback from existing sponsors that we needed to be clearer about the benefits and costs of sponsorship.

For 2019, after feedback from users, we will no longer be displaying sponsors on our homepage. This will help our homepage to render better on smaller screens.

DOAJ sponsorship costs and benefits for 2019

DOAJ 2019 sponsorship costs and benefits

 

In 2019, a Gold sponsorship for commercial organisations is £15,000 and £7500 for non-commercial entities.  A Silver sponsorship is £10,000, and £5000 respectively; a Bronze is £5000 and £2500 respectively. If you would like to know what the money is spent on, you can read this publishers report from 2017 (2018’s is coming soon) or this post about our new mission statement which covers the areas on which we are focussing. Alternatively you can send me an email and I would be very happy to give you more information.

If you are interested in becoming a 2019 Sponsor for one of the most important online resources in academic publishing, and in joining our existing group of fantastic sponsors, then please contact me directly: dom@doaj.org. I look forward to hearing from you!

Development plan for 2015/6

Recent discussions with a partner reminded me that we haven’t published a development plan.

I’ve put together a high level list of our developments: what we are currently doing and what we have next on the list. As it stands, it will take us through to mid-2016 but it is a very fluid document. We have the flexibility to add in items and move others so the 2016 items will be revisited in January 2016 and reviewed to make sure nothing else has cropped up in the meantime.

Some of these items I have already announced in earlier posts. I’ve collected those here.

As always, I’d welcome your feedback.