Creating library tutorials to provide flexibility and customized learning in asynchronous settings

Abstract Online tutorials that provide flexible and up-to-date information on library resources and research concepts are a vital service in academic libraries. At the University of North Carolina Greensboro, librarians created a platform providing multiple options for creation and delivery of virtual and interactive tutorials. At this University, the tutorials are useful to train patrons on a variety of research concepts. When the COVID-19 pandemic began to impact courses, this already established workflow and set of library research modules was able to assist librarians with the increasing demand for asynchronous online information literacy instruction.


Introduction
Online learning and distance education are growing areas within higher education. According to the National Center for Education Statistics (2018), 35.3 percent of all undergraduate and graduate students at universities and colleges in the United States of America in Fall 2018 took at least one online course. Asynchronous, virtual tutorials teach students a variety of skills that can reinforce educational concepts throughout their college degrees, and academic librarians have long been creating tutorials to support students learning information literacy concepts. Due to the COVID-19 pandemic, many college students were quickly placed in online courses in order to support social distancing, and librarians had to adapt by creating or updating their virtual research services. This article presents a case study from academic reference librarians about creating flexible and accessible online information literacy tutorials through a team based approach.
In early 2019, a group of librarians at the University of North Carolina Greensboro (UNCG) University Libraries began creating a set of online and asynchronous library tutorials designed to better meet the needs of students and instructors. The Libraries' existing tutorial, PATH: Lighting Your Way From Research to Writing, was created in 2010 with the goal of providing asynchronous, chronological modules for patrons to enhance instruction, help distance students, or to replace a one-shot information literacy session. PATH allowed students to log in with campus credentials, which tracked and emailed progress to users taking online modules. This email could then be turned in as evidence of completion for course requirements. Due to the age of the PATH interface and the updated Framework for Information Literacy in Higher Education (American Library Association & Association of College & Research Libraries, 2016), the decision was made by the UNCG Libraries Research, Instruction, and Outreach (ROI) department to design a new suite of tutorials. The new tutorials were designed to give course instructors the ability to select what specific skills they wanted students to learn and create different combinations of modules custom to each course and its outcomes.
The new suite of tutorials directly align with new information literacy learning goals and outcomes, which were designed by UNCG Information Literacy Coordinator and approved in July 2018. The goals and outcomes were written based on feedback from UNCG instruction librarians and archivists from a variety of departments; ROI, the Harold Schiffman Music Library, and Special Collections and University Archives. There are five information literacy goals, which are supported by scaffolded outcomes. The information literacy goals are: Find: Students will feel empowered to locate, access, and select information sources appropriate to their information needs. Evaluate: Students will develop the habit of critically evaluating information and media sources in a variety of formats. Use: Students will synthesize information from multiple sources to support arguments, make decisions, or integrate multimedia content. Credit: Students will value the intellectual property of information creators and use sources ethically. Create: Students will see themselves as information creators contributing to scholarly or creative conversations.
UNCG librarians used the five categories of the information literacy student learning goals and outcomes as categories for the tutorials. During the Fall 2019 semester, several librarians in ROI created the initial suite of modules within the tutorials, 16 total, for release during the Spring 2020 semester. Each module was designed, written, reviewed, and published by librarians, then tested by graduate student reference interns for student usability and length of time required to complete the module. Additional modules were added and by Fall 2020, there were 21 modules available to the UNCG community. The main research questions that this article seeks to address are: How do these tutorials differ from online tutorials that UNCG has offered in the past? What virtual platforms and content delivery methods work best to facilitate information literacy outcomes? Is the content helpful in terms of enhancing library patrons' research skills?

Literature review
There is a wide range of literature on academic libraries creating online tutorials to assist patrons with research and information literacy. Online tutorials can come in a variety of formats, with most being multimedia-based by incorporating videos, images, text, interactions, infographics, and graphics interchange formats (GIF). There have been studies that compare what learning object type is most effective, such as a study comparing students' learning and satisfaction with static research tutorials versus interactive tutorials; in one study, it was found that students learned equally from both, but were more satisfied with the interactive platform (Sachs et al., 2013).
Regardless of the type of learning objects used in tutorials, librarians can design these platforms for specific audiences like undergraduates, graduates, first-year students, or faculty. Tutorials that are themed around "getting started with research" are often useful to first-year students, either to supplement information literacy instruction, integrate into an online course, or to refresh their knowledge (Boyd-Byrnes & McDermott, 2007;Chambers et al., 2013;Johnson, 2017). Subject specific research tutorials are also vital to information literacy student learning outcomes, such as online learning objects geared toward health science research (Walters et al., 2015). Graduate students often need advanced research help, such as Nursing or even Library Science students using medical databases (Homol, 2018;Lamb, 2017;Thomas & Gosling, 2009).
There is a need for library research tutorials for different patron types because of the increase in higher education online learners and courses. There is also demand for alternatives to face-to-face instruction to students, assistance for embedded librarians, supplements to one-shot research instruction, and help with flipped teaching approaches. Librarians have studied the positive effects of using online learning objects in flipped learning for information literacy one shot instruction in order to promote interaction and activities in the face-to-face session (Goates et al., 2017;Hawes & Adamson, 2016;Minuti et al., 2018;Obradovich et al., 2015). Virtual information literacy tutorials can also be course integrated, with some librarians creating online learning objects to work in the context of a specific subject or course (Johnson, 2017;Kayser et al., 2013;LeMire, 2016;Zakharov & Maybee, 2019). Even if an academic library does not have an information literacy course or integrations with first-year instruction programs, liaison or subject librarians can embed and use tutorials within research based online courses (Alverson et al., 2019;Olesova & Melville, 2017;Tumbleson, 2016;Tumbleson & Burke, 2013).
Collaborating within the library and beyond is key to creating a successful suite of research tutorials. Lo and Dale (2009) highlight the importance of collaboration between instructional design units and academic librarians to create tutorials to serve the research needs of an online course at Kansas State University. Appelt and Pendell (2010) study reveals the importance of collaboration with faculty members through focus groups; and at Georgia Southern University, librarians and teaching faculty teamed up to revamp their suite of research tutorials geared toward first-year students, showing the importance of working across different campus units in order to keep tutorials up to date (Chambers et al., 2013).
Assessment can highlight the effectiveness of online tutorials on patron research needs. Some libraries assess their tutorials by looking at a variety of data, including analytics and surveying patrons; Blummer (2007) evaluated academic library tutorials using this kind of mixed methods study. Using assessment to gauge the perceptions of students and faculty using tutorials through interviews and focus groups can also help improve design (Appelt & Pendell, 2010;Lantz et al., 2017;Sachs et al., 2013). Some studies interview patrons or use formative and summative in-person assessment in order to study the effect of online tutorials in a flipped or hybrid environment (Goates et al., 2017;Haber & Mitchell, 2017). Implementing tutorials through a learning management system (LMS) and within a course allows librarians and instructors to measure whether or not information literacy learning outcomes are met (Fontane, 2017;Henrich & Attebury, 2012). Being able to compare tutorial types, such as video versus interactive HTML5 based web pages, can help determine what method academic libraries should take when creating research resources (Lantz et al., 2017;Stonebraker, 2015). Assessing if knowledge was gained by patrons after taking their tutorials is useful when designing research resources, whether through post tests or usability studies (Fontane, 2017;Held & Gil-Trejo, 2016;Lindsay et al., 2006).

Methods
The new suite of UNCG Libraries Research Tutorials were built into the library's existing website by library web application specialists, who created a platform for interactive, multimedia content. The platform was built to be flexible, accessible, and sustainable over time, keeping in mind that research and information literacy practices change. When designing the tutorials, the instructional design theory of Universal Design for Learning (UDL) was referenced, with an emphasis on creating multiple means of representation, engagement, and action and expression in order to create a diversity of online experiences for all learners (CAST, 2018). The tutorials are modular, and UNCG patrons can log in to track their progress and take quizzes to generate a certificate for each tutorial. For example, the Evaluate tutorial includes four modules: "Evaluating Sources: ABCD & CRAAP," "Popular and Scholarly Sources," "Peer Review: Journal Articles," and "Anatomy of a Scholarly Article." Students would need to complete all four modules in the Evaluate tutorial in order to receive a certificate. As of the time this article is being submitted, there are eight tutorials divided among two categories: getting started with research and advanced research. Table 1 (see Results & Discussion section) contains a full list of categories, tutorials, and modules, including analytics.
These tutorials and modules are interactive, with integrated quizzes and quick checks. Quizzes come at the end of each module. Patrons must answer quiz questions correctly in all the modules within a tutorial to generate a certificate. Patrons can take the quizzes as many times as they need to in order to get all the questions correct. Since the platform works on a rich content editor platform, it works well with embed code from outside tools. Librarians were able to embed HTML5 "quick checks" of multiple choice, true or false, matching, drag and drop, and fill in the blank questions using the free, online tool H5P. Quick checks were integrated in order to engage patrons learning on each page of the modules, with no data tracking or consequences. A WordPress website was produced through UNCG Information Technology Services (ITS) to run the H5P WordPress plugin and allow those HTML5 interactions to be stored on UNCG servers.
Each module from these tutorials was also put into Canvas, UNCG's Learning Management System (LMS) platform, and shared in the Canvas Commons, Canvas's learning object repository, so that instructors could easily import each module into Canvas courses to share with students. In Fall 2018, UNCG librarians sent out a survey to UNCG students taking online courses about library resources and information retrieval strategies, and students had a strong preference for receiving course information within Canvas (Harlow & Hill, 2020). Based on this data, librarians knew that creating a Canvas access point for UNCG Libraries Research Tutorials would be crucial to their success. Most UNCG instructors use Canvas to host course content, assignments, announcements, and grading, so most instructors and students feel comfortable using this tool. Additionally, Canvas has an "accessibility checker" which informs creators about issues that need to be addressed. UNCG Libraries is heavily integrated in Canvas in many ways: there is a "Librarian" role in the People tab of all courses to allow librarians to become designers; there is a SpringShare LibGuides learning tools interoperability (LTI) that allows UNCG Libraries research guides to be embedded in Canvas as a Course Navigation tab called "Library Resources"; and Ã These modules were not tested by library interns because they were created after the Spring 2020 semester ended.
They were added after August 2020, therefore were not pushed out in the initial marketing of the tutorials and modules.
eReserves are embedded in Pages in Canvas through a secure, cloud based Box integration. Librarians at UNCG are also offered regular instructional technology training sessions, conducted by the Online Learning Librarian, which often cover Canvas-related topics; liaison librarians are prepared to embed in Canvas, as well as troubleshoot any potential issues with the platform. Canvas Commons allows for copyright designations by the creator of a module, and a CC BY-SA (Creative Commons Attribution-Share Alike) license was applied to these tutorials. This means that those wishing to use the material may do so as long as they give credit to the original creators and share their new materials using the same license. Canvas Commons also increases the discoverability of the modules through keyword tagging. Though the tutorial website was built by UNCG Libraries Information Technology department, the content and organizational structure was collaboratively created by UNCG liaison librarians. The Online Learning Librarian was trained on the backend of the website platform, and liaison librarians worked together to create the content for the modules using Trello, Google Docs, and Google Drive. The Online Learning Librarian and the Information Literacy Coordinator created a module template in Google Docs for liaison librarians to use to create modules. This template included places to input content for title, learning objectives, pages within the module, images, quick check interactions, and quiz questions. It also included suggestions and rules for module creation, such as number of words per page, how many pages per module, links to where to put images, and more. Liaison librarians then claimed modules on the Trello board, made a copy of the module template and created content in Google Docs. Once a module was complete and reviewed by the Information Literacy Coordinator, the module was inputted into the tutorial platform by the Online Learning Librarian.
Use of the Trello board to organize and keep track of the work related to the tutorial modules has been essential. This collaborative platform allows for multiple users in a "team" to tag themselves to claim modules that they are interested in creating. It also allows for commenting and color coded tracking related to the stage in the process of creation of each module. There are tags for "Research Basics" and "Advanced Research" to indicate which skill level the topic represents. Additional tags like "In Process," "In Platform," and "In Canvas," help to indicate the publication status of each resource. Since different librarians are responsible for different stages of the process, this tagging system has helped eliminate confusion and repetition of work. The Trello board also serves as a repository for logistical checklists and resources, as well as open educational resources on information literacy topics for inspiration. See Figure 1 for a screenshot of how the Trello board is organized for this tutorial creation workflow.
In order for the tutorials to be found and used, it is important to market them to a wide academic audience, particularly instructors and faculty. There was a soft launch of the new tutorial platform in Spring 2020 in order to gauge interest and collect feedback from teaching faculty, with liaison librarians recommending them in courses with relevant research projects. Throughout Summer 2020, the Online Learning Librarian featured the platform in virtual professional development opportunities. For example, a group of UNCG liaison librarians had a panel on the virtual meeting platform Zoom about online learning information literacy instruction examples, and these tutorials were featured; around 100 faculty were present in this virtual professional development session and the recording is available through the institution's University Teaching and Learning Commons (UTLC). The modules were promoted to course instructors around campus via their liaison librarians at the beginning of the Spring and Fall 2020 semester. In Fall 2020, many library services were moved online due to COVID-19, and the link to the tutorials was included in many of the flyers and webpages about library virtual instruction options. The Online Learning Librarian also featured the tutorial platform to UNCG Information Technology Services (ITS) and academic instructional technology consultants in order for them to spread the word to and gain feedback from UNCG instructors and faculty.

Results & discussion
The modules within the tutorials are designed to be able to be published quickly, but having a way to evaluate and check over the modules is vital; in order to check over the modules as they were published, a workflow was created with reference interns. Six graduate reference interns in UNCG's Library and Information Science (LIS) program were tasked with testing for usability and timing; two interns were assigned to test each one, with one intern using the Canvas version and the other using the library website version. A Google Form was used to record each person's feedback and results. The Google Form asked for each intern's name, whether they were in Canvas or the website, how long it took to complete the module, which module was completed, a Likert scale of if a module is helpful to a student wanting to learn a skill, and an open feedback field. Testing, which took place during the Spring 2020 semester, had mixed results. While most interns followed directions effectively and were able to complete the testing, several did not turn in feedback. Luckily, at least one intern was able to successfully test each of the modules that existed at the time. Results mostly included comments on minor spelling errors or clarifying directions. Interns were also asked to rate the helpfulness of their assigned module on a Likert scale of 1-5; the average response was a 4.76. Timing was also a focus of this initial evaluation; on average, it took interns 12.92 minutes to complete each module. Website clicks were pulled from UNCG Libraries Electronic Resources and Information Technology (ERIT) website analytic tool, which counts page, quiz, and tutorial level clicks. The dates for these analytics range from August 12th 2020 until October 20th 2020, and the Canvas Commons downloads show how many times modules have been downloaded since they were created; many modules were added to the repository in Spring 2020. Canvas Commons downloads do not account for the number of students who have used the tutorials within Canvas, only the amount of times downloaded by an Instructor, Course Designer, or Librarian role in Canvas. According to analytics, the most popular research module is "Plagiarism," followed closely by the "Popular and Scholarly Sources" module.

COVID-19 and asynchronous online instruction
During mid-March 2020, it was announced that most (over 90%) UNCG courses would be switching to an online format due to the COVID-19 pandemic. Instructors had approximately one week to make this change while students moved out of residence halls and campus emptied. Due to the sudden switch to remote learning, it became clear that these modules would be more important to instructors to better help their students with research. UNCG ROI liaison librarians made the decision to perform all of their information literacy instruction virtually, and simultaneously, librarians faced more demands on their time due to an increase in requests for online library services and resources.
When COVID-19 moved and kept library instruction online, librarians were searching and creating a variety of online asynchronous instruction objects. The tutorials were already established with categories and modules; it was quickly determined that liaison librarians would need more modules based on research course assignment needs. But how do research liaison librarians know if something goes into this suite of tutorials or if they should create a stand alone online learning object? In order to help answer this query, the Information Literacy Coordinator came up with a decision tree that asks a series of questions about the need for the module. The questions are: Is this content for a specific course and/or about a specific database (Academic Search Complete, Kanopy, or more)? Does the content align with the UNCG Libraries learning goals and information literacy outcomes? Do you want this tutorial to be interactive? Does the content need to include multiple forms of multimedia and quizzing? Will this content take users less than 15 minutes to get through? When going through the questions, librarians were offered alternatives to the tutorial and instructional technology tool recommendations (Figure 2).
In Fall 2020, the research module creation workflow was an advantage and more online resources were able to be quickly produced. For example, a professor needed an asynchronous resource for helping first-year students do research on a pro and con style topic analysis. At the same time, the University Libraries dropped a subscription to a research database that students were recommended to use for this sort of project. Because templates exist for organizing module content and because the workflow involves easy access to collaborative documents via Google Drive, two instruction and research librarians were able to quickly partner to create the module "Exploring Multiple Points of View." This module explains to students how to use some alternative resources that show a variety of perspectives on a topic in order for students to complete the assignment successfully. The module was then sent to the Online Learning Librarian, who posted it to the library's website for immediate use, while it was also easily integrated as a LMS module in Canvas Commons. The entire process took approximately two business days, and the module was ready for any course with argumentative essay assignments.
The flexibility and workflow of this suite of research tutorials lends well to creating modules based on assessment performed in online or face-to-face library instruction. UNCG librarians perform a variety of formative and summative teaching assessments. Many times instruction ends with a short survey to students, asking them about concepts learned from the information literacy learning outcomes, such as an activity asking them to find a permalink to a scholarly article. Based on these kinds of assessments, it was found that students struggle with citations, particularly the details of the style in their field and integrating it in their writing. During the Summer of 2020, librarians were able to finish two citation style tutorials: "MLA (Modern Language Association, 8th edition)" and "APA (American Psychological Association, 7th edition)," that links and connects to the preexisting modules on "Citation" and "Integrating your Sources in Writing." These modules blend citations, citation styles, paraphrasing, and in-text citations, while also working with the currently most used module on "Plagiarism." Other modules for specific classes have been created based on the established workflow of the tutorials and modules, including a library one-shot substitute module for CST (Communication Studies) 105, a course with more than 50 sections per semester, many of those being taught online. The module has been heavily used and many of the best practices established by the tutorial project were taken into consideration during the creation of this resource. Due to the adaptable nature of the content within Canvas Commons, the Information Literacy Coordinator was able to import and edit many of the research Pages into a large Canvas Commons module on "College Writing" for the high-enrollment course English (ENG) 101; this module has been downloaded into 18 ENG 101 courses (with around 25 students in each course) as of November 2020.
Assessment of tutorials is key to their success, and the library website tutorial platform is consistently assessed and reviewed to improve the content, even during COVID-19. The tutorials were checked for accessibility using the WAVE WebAIM online tool. An assessment form to measure the effectiveness of the tutorials for patrons was created. IRB (Institutional Review Board) approval was obtained for the purpose of collecting anonymous feedback from course instructors and students. This feedback was solicited via a Google Form which posed the following questions: What is your status? [Student, Instructor] Which module did you complete? This module met my needs [5-point Likert scale ranging from "strongly disagree" to "strongly agree"] Please share any feedback about this module.
As of December 2020, there were over 900 responses on this form, mostly from students. According to this survey, most students are accessing the tutorials through Canvas. Most students (approximately 90%)also reported that they agreed or strongly agreed with the statement "This module met my needs." So far the assessment shows a high satisfaction level, but a deeper analysis using coding will need to be deployed in order to look through all of the comments. See Figure 3 for ratings of the tutorials.

Future directions
The COVID-19 pandemic and its influence on higher education remains a pressing concern, and this evolving crisis will certainly impact future tutorials and modules. If more courses are placed online in the coming semesters, more asynchronous information literacy resources will need to be created, updated, edited, published, and shared widely with the campus community. This will require significant attention on the part of the Online Learning Librarian, as well as any librarian involved in the creation of these research learning objects.
As with all projects involving web-based interactive content, more assessment and testing will be needed in the near future. Interns and student workers in the library will likely remain a helpful group from which to collect feedback about content, timing, and possible usability improvements within the modules. A quantitative look at analytics over time will help librarians know what content is being used the most and how people are using the modules. Focus groups or interviews with individuals who are users of these modules have the potential to improve the platform and content. Usability testing can also help librarians understand how people are using the tutorials and what aspects might be confusing. And lastly, a comparison of Canvas Commons versus the website platform might help librarians know which version of the research tutorials to promote to instructors to use in their courses.
The tutorial platform is flexible based on the outreach and training demands of departments within this academic library; it was designed for any librarian or archivist to add asynchronous training modules based on their needs and the information literacy learning outcomes. There are plans to create more subject-specific tutorial modules, such as those covering research in the health sciences, scholarly communication, finding and using data, and primary sources in a variety of contexts (humanities, health sciences, and social sciences). These resources will be created by librarians in the ROI department, as well as in Technical Services. The Special Collections and University Archives (SCUA) department does a large amount of instruction, outreach, and modules on archival materials, primary source research, primary source analysis; this platform could significantly help them teach students.
Once the original set of tutorials was created, the need of the "Advanced Tutorials" category for graduate students, advanced undergraduates, and instructors became clear. These libraries have long had workshops on topics for graduate students, with publishing, scholarly communication, and citation management workshops being well attended. Therefore, a collaboration with Technical Services and librarians working with scholarly communication areas developed into a plan for a future tutorial. Additional modules have been added into this category in Fall 2020, including one on Open Educational Resources (OER) which is specifically designed for course instructors or teaching assistants who are interested in incorporating OER into their courses to reduce costs for students. UNCG Libraries has long managed an OER stipend program for instructors willing to replace costly materials with low-or no-cost OER alternatives. The Online Learning Librarian recently took over OER library initiatives with the Student Success Librarian; this new OER tutorial will eventually be required for instructors to take and produce a certificate in order to receive the stipend and help train them on how to properly find, use, and teach with OER in their course.
Though marketing of the tutorials did occur in Spring and Fall 2020, an ongoing marketing plan will need to be in place to allow patrons to continue to access these tutorials. UNCG Libraries runs two webinar series on Online Learning & Innovation and Research & Applications. The Online Learning Librarian presented on these research tutorials in an Research & Application session in December 2020. UNCG Libraries is also planning on turning off the old research tutorial platform in January 2021; this is noted on the old interface, but a more comprehensive messaging will need to be sent out to instructors who might still be using the links. In order to create a sustainable marketing plan, the Online Learning Librarian is working with all liaison librarians, as well as the libraries Director of Marketing and Outreach to create flyers and blurbs featuring the tutorials. UNCG liaison librarians work with a variety of orientations, or introductions to the library for new students, across departments.

Conclusion
UNCG Libraries have consistently been seeking ways to reach students beyond what face-to-face methods have to offer. Asynchronous online learning is an important content delivery method and allows students to work at their own pace to interact with information and take assessments. Instructors find value in these tutorials because they can access them in a variety of ways, either by downloading modules from the Canvas Commons or by sending students directly to the UNCG Libraries website. There are a number of significant takeaways from this project, and the team continues to learn as new content is created. Working together and having implemented a workflow that allows for collaboration from a distance has become more important due to the COVID-19 pandemic, and the online systems for creating, editing, marketing, and assessing these tutorials turned out to be exponentially more critical than ever imagined. As tutorial development continues, the key principles of teamwork, organization, and multiple modes of online delivery will make this project a continued success.

Disclosure statement
No potential conflict of interest was reported by the author(s).