by Xiaoyan Song

Introduction

The Acquisition & Discovery (A&D) department at North Carolina State University Libraries has three units: Monographs, Serials, and Data Projects and Partnerships (DPP). A&D’s core functions include the acquisition, licensing, cataloging and access management of materials for all NCSU Libraries, including the branches. The department employs a suite of commercial systems and home-grown tools to facilitate the core work. E-Matrix, developed in-house, is used to support e-resource life-cycle management for e-journals and databases. E-Matrix manages the complex relationships of e-resource titles, holdings, collections, organizations, contacts, licenses, administration information, and acquisition details. However, workflow support is not provided in E-Matrix. The department has developed a set of workflow maps for daily core work related to the resource selection and acquisition process, which involves multiple departments and staff. Purchase requests are often initiated by Collection Managers (CM), then sent to A&D, and finally routed to an appropriate workflow with corresponding staff. There were a couple of challenges with the process. First, the purchase requests came from a variety of methods including emails, phone calls, and shared Google docs, and staff found it challenging to keep track of requests from mixed channels. And second, orders were distributed to staff in the two units via emails or Excel files, but the sheer volume of firm orders was too great to keep track of. We needed a tool to support the diverse workflows and to address the needs for the resource selection and acquisition process.

CORAL was chosen due to its flexible and powerful workflow function support. It is an open source Electronic Resource Management (ERM) system which supports electronic resource life-cycle management. It was initially developed by Notre Dame University library and was released in 2010. It includes five modules [1]: Resources, Licensing, Organizations, Usage Statistics and Management. The Resource module stores resource records and provides support for e-resource life cycle management, including acquisition, access, cataloging, and workflow routing. The Licensing module stores licensing agreements, terms and related files. The Organizations module is used to manage organizations including names, contacts and accounts. The Usage Statistics supports COUNTER and COUNTER-like reports. The Management module is a place to store documents such as policies, process, and procedures, and to preserve the history of the documents. All five modules are interoperable, though independent from one another.

Since its first release of the Resources module in 2010, CORAL has been gradually adopted by libraries and other organizations globally. There are 61 confirmed institutions/organizations using CORAL presently[2]. CORAL users have shared their experience at a number of major conferences and these presentations have been gathered on the CORAL website. We noticed that many users have implemented and used CORAL but few performed significant functional enhancements as developers. To give back to the community, we want to share our journey to implement and develop CORAL, and the lessons we learned from our excursion.

From Users to Developers

To address the issues with the resource acquisition process, A&D implemented the CORAL Resources module to manage monograph firm orders in Dec. 2013. We created records for new resources to be ordered and assigned them to staff in CORAL. This module enabled us to keep the firm orders in one place and to keep track of such order requests. In conjunction with CORAL, a shared Google document between A&D and CM was created to centralize all firm order purchase requests. The Specialist in the Monograph Unit was notified whenever a new order was added to the spreadsheet, and she would then create a record in CORAL and use the workflow in CORAL to automatically assign it to a staff member, who would process the order following the corresponding workflow. This streamlined the acquisition process.

When we set up these workflows in CORAL, we immediately realized the limitation of CORAL’s workflow function to distribute orders among staff members. In CORAL, workflows are created as global templates at the Admin tab and they can then be associated with a resource record. Users can make changes to the global workflows at the Admin tab, but they cannot modify a workflow once it is linked to a local resource record. We identified two local workflow customization features to be added to CORAL workflow function. The first is to enable users to re-assign a workflow step to a different member if needed, and the second is to allow users to delete any unnecessary step in a local workflow. Figure 1 shows that a workflow is linked to a resource record in CORAL. Clicking on the pencil icon allows users to change the Group assignment. To delete a step, users can click on the red x symbol.

Figure 1. Workflow enhancement

 

These two new features enhanced CORAL’s workflow functionalities and led to the shared use of CORAL by both the Monographs Unit and the Serials Unit. The Monographs Unit expanded the use of CORAL to include firm orders, open access cataloging, and one-time purchases (a.k.a. end-of-year purchases). The Serials Unit established a set of continuing resources new order workflows in CORAL. See Figure 2 for an overview of all our CORAL workflows.

Figure 2. CORAL workflows

The Libraries were determined to contribute these modifications back to the community. The CORAL community requires that any submitted code changes be reviewed by the CORAL Steering Committee (SC). This Committee consists of several Electronic Resource Librarians (ERL)  and IT staff from libraries, who are also CORAL users. We reached out to the Committee Chair and inquired about the process. It was not a formalized process then because we were the first non-SC members who showed interest in making code changes to improve CORAL functions. As suggested by the Committee Chair, we created a workflow enhancement document and shared it with CORAL SC, who reviewed and approved the enhancement specs. The CORAL SC was looking for justification for the enhancement and a description of the changes to be made.We included the use cases for the requested changes and mockups of the interface change. This round of review and approval process ensured a smooth code merge to the master code at a later stage. Our IT staff completed the development based on the the approved enhancement spec within several weeks. The code merge request was created on Github and then reviewed and approved by CORAL SC before it was merged into CORAL codebase.

While we were pleased with the enhancements, one unexpected opportunity landed in front of us – NCSU Libraries announced an internal Good Idea mini-grant to encourage library innovation and collaboration in Spring 2015. To take advantage of the opportunity, A&D and CM collaborated on a proposal for further CORAL improvements that focused on streamlining e-resource selection and the acquisition process across the two departments. Excitingly, the proposal was approved by the Good Idea mini-grant committee and we received $21k for the project. Two groups were formed: the Admin Group and the Working Group. The Admin Group, including two department heads from A&D and CM and an Associate Head from A&D, was charged with providing admin support and managing the project budget. The Working Group was led by an E-Resource Librarian (ERL) and consisted of two specialists from A&D, one librarian from CM and one IT staff member. This group was tasked with creating the spec and interacting with developers to provide testing feedback.

Due to the unavailability of IT resources, the Admin Group decided to outsource the development work to provide some relief on internal competition for IT resources. This required a Request For Proposal (RFP) process. The key component of the RFP was a detailed development specification. After the grant was announced, the Working Group immediately started working on the spec. Within a couple of weeks, the group delivered the initial specs and shared it with the SC, who provided constructive feedback. Because of the complexity of the enhancements, it took several iterations of communications between the team and the CORAL SC until the final spec was agreed upon.

The Finance & Business department and the University Purchasing department facilitated the RFP process. To hire an experienced development team, we proactively reached out to the CORAL SC and a couple of potential candidates. We received two proposals after the RFP was sent out. Among the two, we chose our current development team, BibLibre, a French software company, who provided a reasonable quote and had significant experience with open source tool development in the library environment.

Grant-Funded CORAL Developments

For the grant-funded project, the team identified three areas for improvement.

The first one was to build a tightly integrated process for resource selection and acquisition between A&D and CM. Generally speaking, this could be a process involving acquisition staff and selectors in any library. The process had previously relied on emails and shared documents to help with decision-making. Our team aimed to streamline the process and to make it more efficient. We created a Propose New Resource API, which allowed users to submit a resource through the API. The API was based on Flight [3], a simple PHP framework enabling users to build RESTful applications. A client form, based on Unirest [4], was created for users to fill in and submit a resource to CORAL. Figure 3 shows the functions implemented in the API. For more information about the API and how to use it, refer to CORAL github Issue #7 Add an API to CORAL [5]. See screencast 1 to preview the completed development.

Figure 3. API functions

Screencast 1. Propose New Resource API. Best viewed in full screen mode.

The second area identified was management of the complex continuing resource renewal process in CORAL. This process involves a variety of activities and requires the authorization from our CM when there is significant price change for a renewal resource. We wanted to integrate the renewal authorization process to the workflow and to make the process transparent to all stakeholders. The team proposed a couple of enhancements to the workflow functions, allowing users to manage renewals:

• Restart new workflow for a completed resource. CORAL users could not restart a new workflow if a workflow was completed for a resource. This allowed users to restart a new workflow, for example a renewal workflow, after a previous workflow was finished. See screencast 2 for a preview.
• Archive a completed workflow. This enabled completed workflows to be archived by default, and users could show or hide the archived workflows. See screencast 2 for a preview.
  
  

  Screencast 2. Restart and archive a workflow. Best viewed in full screen mode.

• Add a new step to the in-progress workflow. For an on-going workflow, users could add a new step if needed. See screencast 3 for a preview.
• Reminders for workflow deadlines. Task reminders are often seen in project management applications or task management tools. This functionality would remind users to complete a step by a due date. See screencast 3 for a preview.
  
  

  Screencast 3. Edit the current workflow and email reminder. Best viewed in full screen mode.

• Email notes to a group/user regarding assigned resources. Users had complained that the workflow assignment emails sent from CORAL did not contain much useful information. This would allow us to embed a concise message in the email about the assignment rather than the generic message generated by the system. See screencast 4 for a preview.

  Screencast 4. Embed a note to reassignment email. Best viewed in full screen mode.

Last but not the least was to improve the import tool to support more diverse needs. At the moment, the import tool in CORAL only allowed a small number of fixed fields to be mapped, and this had limited the use of the tool. Users could use the tool for different purposes, like loading a KBART file into CORAL, batch loading some cost data, or loading resources to be ordered and triggering a workflow once resources were added. To facilitate the various uses of the tool, it required more flexible data mapping. SirsiDynix contributed some improvement to the CORAL import tool [6]. We reviewed SirsiDynix’s work, collected feedback from the CORAL SC, and modified our original proposal to incorporate the SirsyDynix contributions. This resulted in the following feature enhancements:

• Configure/create import templates. Allows users to create an import template by mapping incoming data elements to a desired field in the CORAL Resources module. The template could be loaded and applied during import.
• Preview the mapping before submit. The preview would provide a summary of data to be loaded, such as the total number of records loaded, the number of new records to be overlaid, etc.
• Review history of uploads. Users would be able to review the history of uploads, including the date of previous uploads, the file name, the link to the file, and the link to the imported resources.

The Import tool development is work in progress and the development started mid-September.

Challenges and Opportunities

As open source tool users, we are benefiting from community contributions to the project. However, as the community is growing, potential problems arise and challenge the community. During our journey, we experienced a few challenges, some may be unique to the NCSU libraries, but some may apply to others.

The biggest challenge was resolving conflicts between local needs and global needs. Since its first release in 2010, CORAL has gone through several evolutions. Community members can identify and implement improvements and contribute changes to the CORAL codebase. Enhancement requests originate from local instances. But to merge modifications to the codebase, the enhancements have to meet the community’s needs at large. It’s wonderful when local requests meet global needs. When they don’t, how do we resolve our local concerns? One way is to branch the code, which ensures local development fully responds to local needs. However, branching often means greater local tech support, and the possibility of being unable to use other desirable community contributed enhancements. In our case, our IT did not favor the branch option. This left the team with no choice except to resolve the conflicts and to negotiate with the community. This required more time and effort to deliver the final product, but we were able to deliver a product for all. The negotiation process was all about seeking to understand each other and to be understood by others.

Since community developers can all add code to the code base, it becomes a big challenge to ensure code consistency and to maintain high quality. Hurley [7] points out that “Establishing a standard for code submissions, requiring acceptance of a common license, and implementing peer review are three ways in which good open source projects help to mitigate the risk of problematic code.” The CORAL SC is working to establish version controls and best practices to address this issue.

Whether or not to outsource development can be a question for any software development. The answer to that question largely depends on the availability of internal resources and on the availability of third-party resources. For our grant-funded project, we chose to outsource the development work to a third-party due to the unavailability of internal resources. Networking and outreach ensured the success of finding the right outsourcing team. We realized, though, there were very few third-party developers in the US in this area of development. This may be because libraries have always relied on internal IT or other internal resources to do such development work, or simply purchase available commercial products. The software industry in the US should respond to the growing needs of software developments in libraries.

While working with non-US developers, it is important to take into account different cultures. The 12-week development cycle planned for our grant-funded project was extended several weeks longer because of a couple of “unknown” holidays and vacations that our French team took. It’s not a bad idea to clarify with your developers of upcoming holidays and vacations during the development cycle. This is especially important if the development is funded by state fund or other funds that have a strict spending timeline.

Conclusion

CORAL has a bright future as an open source ERM because of the strength of the software itself and of the strong community support. Either as regular users or developers, community members undoubtedly benefit from all valuable contributions. Challenges can become exciting opportunities for the software to evolve and for the community to exploit synergies.

References

[1] CORAL Modules. Retrieved from: http://coral-erm.org/modules/
[2] CORAL User Map. Retrieved from: http://coral-erm.org/about/user-map/
[3] Flight. Retrieved from http://flightphp.com/
[4] Unirest. Retrieved from http://unirest.io/php.html
[5] Add an API to CORAL (Issue #7). Retrieved from: https://github.com/Coral-erm/Coral/pull/41
[6] SirsiDynix Additions and Style Change. ER&L 2016 CORAL User’s Group Meeting. Retrieved from: http://www.slideshare.net/ScottVieira/er-l-2016-coral-user-group-meeting
[7] Hurley, David. 2014. 12 challenges for open source projects. Retrieved from: https://opensource.com/life/14/6/12-challenges-open-source-projects

About the Author

Xiaoyan Song is an E-Resource Librarian at North Carolina State University Libraries. She holds an MLS and a B.A. in Information Management and Information System.