(PDF) Rapid Development Studio: An Intensive, Iterative Approach to Designing Online Learning

1047574 research-article20212021 SGOXXX10.1177/21582440211047574SAGE OpenMei et al. Original Research SAGE Open Rapid Development Studio: July-September 2021: 1­–9 © The Author(s) 2021 DOI: 10.1177/21582440211047574 https://doi.org/10.1177/21582440211047574 An Intensive, Iterative Approach journals.sagepub.com/home/sgo to Designing Online Learning Bing Mei1,2 , Lawrence May2, Rena Heap2, Damon Ellis2, Sue Tickner2, Jacqui Thornley2, Jamie Denton2, and Richard Durham2 Abstract Given the increasing demand for online learning at the tertiary level, there currently exists a need to modify or develop instructional design (ID) models/approaches that can effectively facilitate the collaboration between learning designers and teachers, as well as to research the effectiveness of these models/approaches. Against this backdrop, adopting a design- based research approach, we tested a practical ID approach that is developed on two prior models: rapid prototyping and collaborative course development. Accordingly, a 2-week rapid development studio—an agile, intensive, iterative ID process—was arranged. Data from multiple sources were gleaned during the study to generate a comprehensive and in- depth understanding of the proposed approach. Overall, results suggest that the approach is effective for developing online courses in case of a limited time frame and was positively perceived by both course instructors and learning designers. Moreover, practical tips for replicating the process in other contexts are also shared. It is our hope that the study will stimulate further exploration of alternative ID models/approaches to improve online course design efficacy in other higher education institutions. Keywords instructional design, collaborative course development, rapid prototyping, online learning Introduction learning designers often struggle to deliver well-designed online learning due to the inflexibility of these linear ID With the expansion of digital cultures and access to informa- models (Adnan & Ritzhaupt, 2018). Such a dilemma is fur- tion and communication technologies (ICT), online learning ther exacerbated by the growing offering of online programs has been increasingly integrated into the educational sector by higher educational institutions from around the world as a catalyst to transform teaching and learning, especially at (e.g., Bates, 2018; McConnell, 2018). Thus, there is a grow- the tertiary level (Broadbent & Poon, 2015; O’Shea et al., ing interest in research on exploring alternative ID models/ 2015; Zhang & Worthington, 2017). Meanwhile, research approaches that allow for rapid production of online courses evidence (e.g., Clifton, 2017; George-Walker & Keeffe, and efficient collaboration between learning designers and 2010; Lai et al., 2017; Reiser & Dempsey, 2012) suggests teachers, with some recent examples including the ABC that as a multifaceted process, development of effective learning design (Young & Perović, 2016), the Carpe Diem online learning material benefits from design thinking in learning design (Salmon & Wright, 2014), and Course order to link potential affordances to pedagogy. Thus, explo- Design Intensives (Benfield, 2008). rations of effective instructional design (ID) for online learn- Against this backdrop, adopting a design-based research ing have started to gain momentum (Bonk et al., 2018; approach, we, as course developers, tested an ID approach Ifenthaler et al., 2018; Maybee et al., 2019; Richardson et al., 2016; Yang et al., 2016). However, current efforts are mostly scaffolded by traditional ID models, such as the Analysis, 1 Henan University, Kaifeng, China Design, Development, Implementation, and Evaluation 2 University of Auckland, New Zealand model (ADDIE; Branson et al., 1975), the Attention, Corresponding Author: Relevance, Confidence, and Satisfaction model (ARCS; Lawrence May, Faculty of Education and Social Work, University of Keller, 1987), and the Dick and Carey (1978) model. Further, Auckland, 74 Epsom Avenue, Epsom, Auckland 1023, New Zealand. when employing these models, course instructors and Email:

Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 SAGE Open that combines the advantages of rapid prototyping (Tripp & materials for e-learning grids. The results indicate that the Bichelmeyer, 1990) and collaborative course development process can reduce redundant human efforts significantly and (Hixon, 2008). According to Richey and Klein (2014), produce high-quality teaching materials. Moreover, Salmon design-based research refers to “the systematic study of and Wright (2014) reported findings from their use of the design, development and evaluation processes with the aim Carpe Diem workshop model, which is essentially a rapid of establishing an empirical basis for the creation of instruc- prototyping process. Their findings suggest that this approach tional and non-instructional products and tools and new or can enable teachers to produce online courses quickly and enhanced models that govern their development.” (p. 142) that it should be employed for preparing staff for e-learning Though being a relatively new research scheme, a growing design. body of research (e.g., Anderson & Shattuck, 2012; Barbera However, rapid prototyping for ID is not without its draw- et al., 2017; Gorard et al., 2004; Tracey, 2009) suggest that backs. Davidson-Shivers et al. (2018) pointed out that in a the participatory and iterative process can bridge the gap typical rapid prototyping scenario, course instructors are between theory and local practice by designing, testing, generally involved only at the early stage, while the full- modifying, retesting, and redesigning contextualized educa- scale development around the subject matter usually relies tional interventions. This paper reports how we, as course on collaborative efforts among instructional designers. Such developers, started the exploration from an initial concept, disconnection often leads to repeated cycles of revision at the linked theory to practice, proposed the hybrid ID approach, later stage of a project. and conducted a pilot operation within the context of a ter- tiary education institution. It is our hope that this develop- ment process might stimulate active exploration of alternative Collaborative Course Development ID models/approaches to further improve online course Before the advent of the internet, teachers were mostly design efficacy at the tertiary level and inform future research responsible for developing their own teaching resources and practice. (Clark & Angert, 1981). However, with the rise of online courses, the need for collaboration among teachers and learn- ing designers has become evident (Lowyck & Pöysä, 2001; Literature Review Mor & Winters, 2007). This is especially true given the increasing awareness of the significance of ID, the affor- Rapid Prototyping dances of ICT, and the variety of available digital pedagogies The term rapid prototyping was initially used in the field of (Garrison, 2003). As a result, collaborative development has computer software design and engineering (Tripp & increasingly been adopted as an approach for developing Bichelmeyer, 1990). It refers to a “process [which] involves online courses (Hixon, 2008; Xu & Morris, 2007). Recent quickly developing a prototype product in the very early studies suggest that collaborative course development could stages of an instructional design project and then going be a practical solution to combining learning designers’ tech- through a series of rapid tryout and revision cycles until an nical-pedagogical knowledge together with course instruc- acceptable version of the product is produced” (Reiser, 2001, tors’ expertise in practice, and Chao et al. (2010) found p. 63). Prior research (e.g., Brown & Green, 2015; Desrosier, this particularly evident in the development of new courses 2011; Nixon & Lee, 2001) suggests that when compared “from scratch.” Campbell et al. (2009) also determined that with traditional ID models, rapid prototyping exhibits the collaboration could bring about positive changes among following four advantages: learning designers and make them more active, while Martinez-Maldonado et al. (2017) found that face-to-face (1) It is especially suitable for designing computer-aided collaboration enhanced participants’ mutual understanding instruction; and facilitated fluid group interactions. Nonetheless, despite (2) Its expeditious process can deliver significant cost the growing research interest in collaborative course devel- savings in time, resources, and labor; opment approaches, less is known about effective ways to (3) It allows formative evaluation of the prototypes dur- substantiate such collaboration in practice. For example, ing the process; and, collaborative course development is, in practice, mostly (4) It facilitates a co-inquiry model integrating direct organized in the form of scheduled meetings with course criticism and feedback among multiple stakeholders. instructors and learning designers working in different loca- tions. This physical detachment often impedes the ID pro- For instance, adopting rapid prototyping for their projects, cess, and can exacerbate common practical constraints such Jones and Richey (2000) found that designers were able to as limited time availability and tight budgets (Hixon, 2008). spend less time on revision, deliver the product within a Our review demonstrates that both rapid prototyping and shorter time frame, and communicate better with customers. collaborative course development have proven effective for Further, Shih et al. (2008) conducted an experiment to test the online course development. Yet, neither approach offers a use of wiki-based rapid prototyping in designing teaching balanced solution to the joint constraints of time and space. Mei et al. 3 In view of this research deficiency, this study tested a practi- The Proposed Rapid Development Studio cal way to combine these approaches for online course development. The following three research questions guide Guided by prior research, the RDS was proposed. Its com- this study: pressed timeframe requires quick, but incomplete, articula- tion of the desired end-goal, followed by rapid prototyping (1) How can rapid prototyping and collaborative course of the different components of the course. Equally important development be combined for online course develop- is the presence of the teaching staff member(s) throughout ment as an ID approach? the RDS process. This can result in the creation of rapid pro- (2) How do participants of the study, as learning design- totypes with reduced need for revisions and rebuilds. ers and teachers, perceive the ID process? In terms of structure, the RDS is intended to consist of (3) What shared reflections can be gleaned from this col- three phases. Phase 1 starts before the RDS takes place. In laborative inquiry to inform the future team action? this phase, the course content is collated for redevelopment. Next, in Phase 2, multiple stakeholders collaborate to articu- late course design needs and desired student experiences, Method and to develop solutions and content. Phase 3 is the evalua- Considering its focus on course developers, the case study tion or quality assurance stage, wherein the final product is adopted a design-based research approach (Wang & reviewed and proofread before being delivered to students. Hannafin, 2005). Specifically, drawing upon prior research (e.g., Coetzee & Smart, 2012), we reported the process of Data Collection developing a course with the help of a rapid development studio (RDS), which is conceptualized to be an agile, inten- Following the recommendation of Wang and Hannafin sive, iterative ID process with a compressed timeframe and (2005), data from multiple sources were collected. During the thus enables course instructors and learning designers to col- RDS, observations and field notes were taken by the project laborate in close physical proximity. team and collated. After the RDS, an anonymous question- naire survey was conducted within the course redevelopment team. The questionnaire consisted of eight open-ended ques- Study Site and Participants tions designed to elicit participants’ reflections on the RDS. This study was conducted within a digital learning team The questions covered previous understanding of rapid devel- (DLT) from an educational faculty at a New Zealand univer- opment approaches, experiences of the pilot project, personal sity, in collaboration with two partner lecturers. opinions on the approach, and suggestions for future modifi- The DLT consisted, at the time of this study, of a group of cations (see Appendix for details). six learning designers and two project assistants, responsible for consulting and collaborating on the design of blended Research Ethics undergraduate and postgraduate courses and special projects within the institution. Research ethics were maintained during this study. The data Since the implementation of Canvas (a learning manage- were gathered as a review of departmental practice and then ment system) in 2017, the team has faced a challenging task handed over as deidentified secondary data which are exempt to migrate existing courses hosted on prior learning manage- from ethics approval by the University of Auckland. Further, ment systems as well as to adapt traditional face-to-face all participants were invited to contribute to the manuscript. courses for online or blended delivery. In recent years, amid They were not forced to participate in the writing and were competition from other tertiary providers, demand from the assured that their identity would be strictly confidential institution’s leadership and academic staff for online course under any circumstances. Hence, all participants were aware delivery as a mean to innovate learning and teaching has rap- that the results of this study would not lead to any punitive idly increased, aiming to diversify and increase student consequences. All participants also gave verbal consent for enrolments. their data within this dataset to be considered when they Against this background, the DLT, in partnership with two were invited to contribute to writing the article. lecturers of a course, carried out the study. The two instruc- tors were responsible for an existing face-to-face, postgradu- Data Analysis ate course on the subject of digital pedagogies for in-service teachers. Previously, the course bad been delivered in a When the RDS ended, the de-identified secondary data were 1-week, face-to-face block. In view of surging demand for analyzed within a thematic analysis framework. The data professional development and critical inquiry into the use of were coded into themes that captured an understanding of digital pedagogies by in-service teachers in New Zealand, participants’ perceptions of and experiences in the RDS. As the course was identified as one to be made available for Braun and Clarke (2006) point out, thematic analysis in qual- general enrolment for online study. itative research is widely used and offers rich opportunities 4 SAGE Open Figure 1.  The intended structure of the RDS. to identify patterns and understandings that may emerge Next, learning designers and course instructors met to dis- from subjective experiences of individuals. After analysis of cuss the current implementation of the course, reflect on stu- the data, important patterns that were corroborated by dent feedback, and instructor experience of prior delivery. respondents were identified. To ensure trustworthiness, the This was expressed in terms of what had worked well and results were uploaded to Google Drive and shared among the what could be improved, what students liked and disliked, participants for peer scrutiny and member checking (Nowell and the challenges that the new context for the course’s et al., 2017). delivery might bring. Furthermore, two whiteboards were set up in the shared workspace. The “Responsibilities” board kept track of which particular team member had been Results assigned responsibility for which overall task (e.g., “con- tent,” “assessments,” and “design”). The second board, Implementation of the RDS “Progress,” was established as a space for team members to Considering the practical constraints (e.g., both the instruc- update the current status of the project using post-it notes and tors and learning designers had considerable teaching, simple status categories (e.g., “to do,” “in progress,” and research, administrative, and other project commitments), “done”). the RDS was planned to be completed within 2 weeks. The In Phase 2 (Days 2–9), the project team and teaching staff detailed structure is presented in Figure 1. Before the RDS conducted agile prototyping of course components (assess- started, the course-related content including calendar, ments, activities, and resources) in semi-structured and fluid description, learning outcomes, weekly topics, assessments, small teams, using a tight iterative cycle of ideation, produc- reading list, and key activities were collected and dissemi- tion, peer evaluation, and revision. nated among the project team and stored in a shared Google Regular all-team check-ins, milestone monitoring, and Drive to facilitate ongoing access. peer evaluations maintained overall course design cohesion. Phase 1 (Day 1) of the RDS began with multiple stake- Each day of this phase started with a focused meeting to share holders (e.g., members of faculty leadership responsible for updates on progress, and to discuss obstacles. Meanwhile, the learning and teaching strategy, and for expanding postgradu- course content was outlined in a physical timeline on the floor ate enrolments) meeting with the project team and course of the shared workspace to visualize the course in its entirety. instructors to develop a shared understanding of the context Differently colored sheets of paper represented different of the course, identify strategic and practical needs impact- course components, and were laid out from left-to-right in ing the course design, and to brainstorm possible solutions. columns, with each column representing 1 week of study. The Guiding principles of the course were also discussed and scale of the map allowed for a snapshot view of the entire determined. course but left enough room to describe specific topics and Mei et al. 5 Figure 2.  The course map in the office. activities in more detail, and for multiple groups to interact of the RDS for online course development, all respondents with the map simultaneously. The choice of color to delineate agreed that the RDS demonstrated an effective approach for course components enabled quick identification of balance this type of online course development. For example, and variety in those elements, and the timeline sequence structure enabled identification of rhythm, repetition, and It is a collaborative engagement in an intensive project helps a progression of those elements. The ability to physically rear- team to commit to outcomes in a set timeline. (From Respondent E) range the elements enabled speedy communication and accel- erated the iterative design process. Figure 2 presents how Such value of the RDS is further reinforced by partici- mediating artifacts were used in the workspace. pants’ willingness to engage in future iterations of this practi- Finally, in Phase 3 (Day 10), quality control and inclusive cal approach. When asked about their willingness to design checks were conducted, and participants reflected on participate in another RDS, all participants responded posi- the overall process. Despite ongoing informal checks and tively and made relevant suggestions for managing a second meetings to ensure course continuity and cohesion, the speed deployment of the RDS. As some respondents clearly indi- at which activities or assignments were altered resulted in cated below, unanticipated impacts throughout the course. Thus, a cogni- tive walkthrough approach recommended by Mahatody et al. It can be a good technique to be added to a mix of approaches. (2010) was adopted to check that the student workload was (From Respondent E) feasible and that the aims were met and scaffolded into Repeating the process as it was, repeating the experiment with timely and relevant tasks. Furthermore, a member of the different stakeholders, repeating the experiment with different design team performed a walkthrough from the student per- durations and with different outcomes. (From Respondent F) spective, highlighting any inconsistencies and opportunities for further revision. As a result of these reviews, the decision Furthermore, all participants viewed the RDS as a signifi- was made to create a “Getting Started” document to orient cant contributor to promote motivation, mutual trust, engage- learners, and set out expectations and protocols prior to the ment and connection within the team. start of the course. Finally, a checklist for inclusive design was also applied and adjustments made where feasible. I enjoyed the parts where we came together to share ideas, check on progress, debrief and collaboratively design the next steps. (From Respondent A) Post-RDS Survey Results Overall, the open-ended survey results suggest that the RDS When responding to question about effective parts of the is generally well perceived by both course instructors and RDS, most participants identified with the physical visual- learning designers. Specifically, when asked about the value ization approach, in particular the timeline of the course 6 SAGE Open layout on the floor in the RDS. For them, it provided an Discussion opportunity for central point of reference, communication, alignment of content and re-contextualization of ideas as In order to combine rapid prototyping and collaborative they were discussed. For example, some respondents course development, we adopted a design-based research commented, approach and piloted the RDS in order to adapt a course for online delivery in 2 weeks. Overall, the outcome suggests that this agile and collaborative RDS can be an effective The collaboration around the locus of a physical artefact (the course on the floor was the most engaging part of the process. alternative model for ID, and worthy of further pursuit for (From Respondent D) course development projects with limited timeframes. Despite the overall positive feedback, there were participant I thought it was really useful having the visual timeline of the concerns regarding personal space and ability to focus, sug- course and being able to move activities around on it, to literally gesting that while the RDS might be productive and effi- see the effect on workload etc. (From Respondent E) cient as an ID approach, varied individual working styles, and personal needs should be considered as part of its This focus on the potentially transformative impact of implementation. making what are otherwise primarily digital developments First, according to the results, the co-working arrange- in online platforms (which require project participants to ment of the RDS had a positive impact upon the team pro- actively access and monitor platforms on their computing ductivity. Similar to prior research on space and productivity devices) constantly visible and accessible in physical form (e.g., Bednář et al., 2021; Clarke, 2016; Mariotti et al., 2021), is repeated numerous times in the data. One respondent fur- feedback from both learning designers and teachers indi- ther explains the significance of this element of the RDS’s cated their preference for the arrangement, as it facilitated design as relating directly to questions of disembodiment in their interprofessional communication. This positivity may a professional role largely dedicated to virtual artifacts and also be attributed to the fact that physical proximity allowed experiences: quick distribution of roles responsibilities, and direct com- munication between team members, and access to rapid peer Another highlight for me [is] to be able to crowd around a feedback. The finding has implications for learning design physical representation of a virtual component of the course and team managers. As team leaders, they may consider devising negotiate, test, critique and develop ideas with the group, in the innovative ways to put both learning designers and teachers moment. (From Respondent B) together either physically or virtually in order to accelerate the course development process. This is especially the case With regard to personal gains from the RDS, some par- when large amounts of existing courses might need to be ticipants pointed out that participation in the RDS helped adapted for online delivery during the COVID-19 pandemic increase their understanding of their colleagues’ strengths (Moorhouse, 2020), and flexible or remote working arrange- and skills from participating in the RDS. ments continue for university staff in many countries. Just as online teaching requires reconceptualizing the learning envi- One of the best things was working as a whole team on a project, ronment as a virtual space of community and collaboration, [which] helped me see where I can contribute and the areas of learning design practices will benefit in the post-COVID 19 others’ expertise. (From Respondent E) era by adapting to the possibilities offered by video-confer- encing tools and other digital collaborative platforms. It offers new insights into how my colleagues structure their own Next, our exploration extends the previous findings on the work, greater clarity. . . and improved connections. (From benefits of utilizing mediating artifacts in similar projects Respondent F) (e.g., Agostinho, 2011; Buckley, 2015; Quintana & Tan, 2021). The materialized visualization offered by the course Evidence from the survey data therefore suggests that map proves to be a cogent tool to facilitate collaboration and while the RDS might be well-suited to intensive, short-term promote efficiency. In our case, participants of the RDS projects, its impact on the internal culture and working style showed positive perception of the physical course map, of the team might be ongoing and longer-term in nature. which guided the structural design of the course and the proj- Nevertheless, participants also highlighted points that ect whiteboards, which captured the development’s progress. need to be considered in the future. As previously suggested by Kaufman and Flanagan (2016), It will be challenging to define this measure/requirement. (From the findings show that using tangible artifacts as a physical Respondent E) representation of a digital design can help people make sense of “intricate, multi-variable systems with a central problem Teachers came with a large body of excellent existing course to solve, rules to learn, and constituent components to nego- material . . .. Thus, the approach needs to be tested against a tiate” (p. 2). Thus, it could be helpful for a learning design development from scratch. (From Respondent F) team to develop their agreed protocol on the use of mediating Mei et al. 7 artifacts, as it could boost both productivity and creativity. Appendix Mediating artifacts could also have an important role to play outside of detailed redevelopment projects, and also be used Rapid Development Studio Pilot Reflection Survey effectively in more targeted learning design activities (e.g., Questions the redesign of a particular module or assessment). The 1. Please discuss what your understanding was of the insights these artifacts offer educators about their curricula rapid development approach to learning/instructional by making interdependencies, opportunities, and challenges design before participating in the pilot RDS? (e.g., more immediately legible suggests a wide range of contexts previous experience of this type of approach or for their use by teachers and learning designers. knowledge of this approach to learning/instructional In addition to the positive feedback, the challenges we design) met echo Van Rooij’s (2010) argument that effective project 2. Please identify any literature/journals/articles you management strategies are required within ID projects. For have read/remember about a rapid development example, before a similar ID project is started, its structure, approach to learning/instructional design? timing, and aim should be well-defined, and shared in acces- 3. Please discuss your experience during the RDS pilot sible formats with all participants in a development project. where you felt most engaged with what was happen- Also, as it is a labor-intensive sprinting process, activities ing? (e.g., Which part did you like, which action was (e.g., collaborative board games and paid lunches) may also useful?) be arranged to keep the team’s morale and motivation at an 4. Please discuss your experience during the RDS pilot ideal level. Further, a focus on project management dictates where you felt most distanced from what was hap- that teachers involved in such projects also need to prepare pening? (e.g., Which part did you not like, what effectively for the ID process by investing time and effort action was unhelpful?) prior to the project into collecting, arranging, and annotating 5. Overall, what is your opinion of the RDS approach to the course content that needs to be adapted. learning/instructional design? 6. On reflection, would you be willing to engage in the Limitations and Future Directions RDS process in the future? If so, please list any sug- gestions that would improve the process for further Although empirical support is evident for the RDS, a number iterations? If you are not keen on the process, please of limitations exist in the design of this study. First, this study advise why? was small-scale, largely qualitative, and the development was 7. What do you think you have gained personally from based upon an existing course, thus reducing the generalizabil- participating in the RDS pilot? ity of the findings to other Australasian tertiary environments. 8. Finally, (last question) .  .  . if you have any other sug- Next, due to its primary focus on course developers, the study gestions and opinions, please feel free to add. does not evaluate students’ feedback on the redeveloped course. Future studies that address these two limitations would provide Declaration of Conflicting Interests more insights on the effectiveness of the RDS approach in practice. Finally, considering the positive role of physical The author(s) declared no potential conflicts of interest with respect mediating artifacts in supporting collaboration and efficacy in to the research, authorship, and/or publication of this article. the iterative design process, research efforts can also be made to ascertain the guidelines on how to use such tools. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Conclusion publication of this research was funded by the Educational This design-based study explored the instantiation of RDS, an Department of Henan Province (grant number: 2021JGLX032). ID approach based on rapid prototyping and collaborative course development. Our results revealed its effectiveness in ORCID iD rapidly adapting existing offline courses for online delivery. Bing Mei https://orcid.org/0000-0001-9185-7509 Moreover, involving teachers in both designing and develop- ment process can facilitate effective collaboration between References learning designers and teachers. Additionally, given the com- Adnan, N. H., & Ritzhaupt, A. D. (2018). Software engineering pressed and accelerated nature of learning design practice design principles applied to instructional design: What can we within RDS, innovative ways may also be employed to encour- learn from our sister discipline? TechTrends, 62(1), 77–94. age collaborative innovation, particularly in the era of the https://doi.org/10.1007/s11528-017-0238-5 COVID 19 pandemic and a context of increased remote Agostinho, S. (2011). The use of a visual learning design repre- working. sentation to support the design process of teaching in higher 8 SAGE Open education. Australasian Journal of Educational Technology, Clarke, D. (2016). Architects as agents for organisational change in 27(6), 961–978. https://doi.org/10.14742/ajet.923 new generation learning spaces. In W. Imms, B. Cleveland, & Anderson, T., & Shattuck, J. (2012). Design-based research: A decade K. Fisher (Eds.), Evaluating learning environments: Snapshots of progress in education research? Educational Researcher, of emerging issues, methods and knowledge (pp. 65–74). Sense 41(1), 16–25. https://doi.org/10.3102/0013189x11428813 Publishers. Barbera, E., Garcia, I., & Fuertes-Alpiste, M. (2017). A co-design Clark, F. E., & Angert, J. F. (1981). Teacher commitment to process microanalysis: Stages and facilitators of an inquiry- instructional design: The problem of media selection and use. based and technology-enhanced learning scenario. The Educational Technology, 21(5), 9–15. https://www.jstor.org/ International Review of Research in Open and Distributed stable/44422550 Learning, 18(6), 105–126. https://doi.org/10.19173/irrodl.v18i6 Clifton, G. (2017). An evaluation of the impact of “learning .2805 design” on the distance learning and teaching experience. The Bates, T. (2018). The 2017 national survey of online learning in International Review of Research in Open and Distributed Canadian post-secondary education: Methodology and results. Learning, 18(5), 278–286. https://doi.org/10.19173/irrodl. International Journal of Educational Technology in Higher v18i5.2960 Education, 15(1), 1–17. https://doi.org/10.1186/s41239-018- Coetzee, S. J., & Smart, A. (2012). Rapid implementation of 0112-3 e-learning using a technology design model. In N. A. Alias & Bednář, P., Danko, L., & Smékalová, L. (2021). Coworking spaces S. Hashim (Eds.), Instructional technology research, design and creative communities: Making resilient coworking spaces and development: Lessons from the field (pp. 219–237). IGI through knowledge sharing and collective learning. European Global. Planning Studies. Advance online publication. https://doi.org/ Davidson-Shivers, G. V., Rasmussen, K. L., & Lowenthal, P. R. 10.1080/09654313.2021.1944065 (2018). Foundations of online learning and instructional Benfield, G. (2008). E-learning course design intensives: Disrupting design. In G. V. Davidson-Shivers, K. L. Rasmussen, & P. R. the norms of curriculum design. Educational Developments, Lowenthal (Eds.), Web-based learning: Design, implementa- 9(4), 20–22. https://radar.brookes.ac.uk/radar/file/dfe086e4- tion and evaluation (pp. 43–79). Springer. 79ec-93ab-dd47-b32ba604f3f6/1/benfield2008elearning.pdf Desrosier, J. (2011). Rapid prototyping reconsidered. The Journal Bonk, C. J., Zhu, M., Kim, M., Xu, S., Sabir, N., & Sari, A. R. of Continuing Higher Education, 59(3), 135–145. https://doi. (2018). Pushing toward a more personalized MOOC: Exploring org/10.1080/07377363.2011.614881 instructor selected activities, resources, and technologies for Dick, W., & Carey, L. M. (1978). The systematic design of instruc- MOOC design and implementation. The International Review tion (1st ed.). HarperCollins. of Research in Open and Distributed Learning, 19(4), 92–115. Garrison, D. R. (2003). E-learning in the 21st century: A framework https://doi.org/10.19173/irrodl.v19i4.3439 for research and practice. Routledge. Branson, R. K., Wagner, B. M., & Rayner, G. T. (1975). Interservice George-Walker, L. D., & Keeffe, M. (2010). Self-determined procedures for instructional systems development. https://files. blended learning: A case study of blended learning design. eric.ed.gov/fulltext/ED164745.pdf Higher Education Research & Development, 29(1), 1–13. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychol- https://doi.org/10.1080/07294360903277380 ogy. Qualitative Research in Psychology, 3(2), 77–101. https:// Gorard, S., Roberts, K., & Taylor, C. (2004). What kind of creature doi.org/10.1191/1478088706qp063oa is a design experiment? British Educational Research Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strat- Journal, 30(4), 577–590. https://doi.org/10.1080/0141192042 egies & academic achievement in online higher education 000237248 learning environments: A systematic review. The Internet and Hixon, E. (2008). Team-based online course development: Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc A case study of collaboration models. Online Journal of .2015.04.007 Distance Learning Administration, 11(4), 1–8. https://eric.ed Brown, A. H., & Green, T. D. (2015). The essentials of instruc- .gov/?id=EJ1065651 tional design: Connecting fundamental principles with process Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learn- and practice (3rd ed.). Routledge. https://doi.org/10.4324/978 ing design through analytics: Applying network graph analy- 1315757438 sis. Australasian Journal of Educational Technology, 34(2), Buckley, C. (2015). Conceptualising plagiarism: Using Lego to 117–132. https://doi.org/10.14742/ajet.3767 construct students’ understanding of authorship and citation. Jones, T. S., & Richey, R. C. (2000). Rapid prototyping meth- Teaching in Higher Education, 20(3), 352–358. https://doi.org odology in action: A developmental study. Educational /10.1080/13562517.2015.1016418 Technology Research and Development, 48(2), 63–80. https:// Campbell, K., Schwier, R. A., & Kenny, R. F. (2009). The criti- doi.org/10.1007/bf02313401 cal, relational practice of instructional design in higher edu- Kaufman, G., & Flanagan, M. (2016). Playing the system: cation: An emerging model of change agency. Educational Comparing the efficacy and impact of digital and non-digital Technology Research and Development, 57(5), 645–663. versions of a collaborative strategy game [Conference ses- https://doi.org/10.1007/s11423-007-9061-6 sion]. Proceedings of the First International Joint Conference Chao, I. T., Saj, T., & Hamilton, D. (2010). Using collaborative of DiGRA and FDG, Digital Games Research Association course development to achieve online course quality stan- and Society for the Advancement of the Science of Digital dards. The International Review of Research in Open and Games, Dundee, Scotland. https://tiltfactor.org/wp-content/ Distributed Learning, 11(3), 106–126. https://www.learn- uploads/2016/12/20160801-tiltfactor-efficacy-of-digital-vs- techlib.org/p/49142/ nondigital-game.pdf Mei et al. 9 Keller, J. M. (1987). Development and use of the ARCS model of TechTrends, 65(4), 562–575. https://doi.org/10.1007/s11528- instructional design. Journal of Instructional Development, 021-00592-x 10(3), 2–10. https://doi.org/10.1007/bf02905780 Reiser, R. A. (2001). A history of instructional design and tech- Lai, P. K., Portolese, A., & Jacobson, M. J. (2017). Does nology: Part II: A history of instructional design. Educational sequence matter? Productive failure and designing online Technology Research and Development, 49(2), 57–67. https:// authentic learning for process engineering. British Journal doi.org/10.1007/bf02504928 of Educational Technology, 48(6), 1217–1227. https://doi. Reiser, R. A., & Dempsey, J. V. (2012). Trends and issues in org/10.1111/bjet.12492 instructional design and technology (3rd ed.). Pearson. Lowyck, J., & Pöysä, J. (2001). Design of collaborative learning Richardson, J. C., Besser, E., Koehler, A., Lim, J., & Strait, M. environments. Computers in Human Behavior, 17(5–6), 507– (2016). Instructors’ perceptions of instructor presence in online 516. https://doi.org/10.1016/s0747-5632(01)00017-6 learning environments. The International Review of Research Mahatody, T., Sagar, M., & Kolski, C. (2010). State of the art on in Open and Distributed Learning, 17(4), 82–104. https://doi. the cognitive walkthrough method, its variants and evolutions. org/10.19173/irrodl.v17i4.2330 International Journal of Human-Computer Interaction, 26(8), Richey, R. C., & Klein, J. D. (2014). Design and development 741–785. https://doi.org/10.1080/10447311003781409 research. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Mariotti, I., Akhavan, M., & Rossi, F. (2021). The preferred loca- Bishop (Eds.), Handbook of research on educational commu- tion of coworking spaces in Italy: An empirical investigation nications and technology (pp. 141–150). Springer. in urban and peripheral areas. European Planning Studies. Salmon, G., & Wright, P. (2014). Transforming future teaching Advance online publication. https://doi.org/10.1080/0965431 through ‘Carpe Diem’ learning design. Education Sciences, 3.2021.1895080 4(1), 52–63. https://doi.org/10.3390/educsci4010052 Martinez-Maldonado, R., Goodyear, P., Carvalho, L., Thompson, Shih, W.-C., Tseng, S.-S., & Yang, C.-T. (2008). Wiki-based K., Hernandez-Leo, D., Dimitriadis, Y., Prieto, L. P., & Wardak, rapid prototyping for teaching-material design in e-learning D. (2017). Supporting collaborative design activity in a multi- grids. Computers & Education, 51(3), 1037–1057. https://doi. user digital design ecology. Computers in Human Behavior, org/10.1016/j.compedu.2007.10.007 71, 327–342. https://doi.org/10.1016/j.chb.2017.01.055 Tracey, M. W. (2009). Design and development research: A Maybee, C., Bruce, C. S., Lupton, M., & Pang, M. F. (2019). model validation case. Educational Technology Research and Informed learning design: Teaching and learning through Development, 57(4), 553–571. https://doi.org/10.1007/s11423- engagement with information. Higher Education Research & 007-9075-0 Development, 38(3), 579–593. https://doi.org/10.1080/072943 Tripp, S. D., & Bichelmeyer, B. (1990). Rapid prototyping: 60.2018.1545748 An alternative instructional design strategy. Educational McConnell, D. (2018). E-learning in Chinese higher education: Technology Research and Development, 38(1), 31–44. https:// The view from inside. Higher Education, 75(6), 1031–1045. doi.org/10.1007/bf02298246 https://doi.org/10.1007/s10734-017-0183-4 Van Rooij, S. W. (2010). Project management in instructional Moorhouse, B. L. (2020). Adaptations to a face-to-face initial design: ADDIE is not enough. British Journal of Educational teacher education course “forced” online due to the COVID-19 Technology, 41(5), 852–864. https://doi.org/10.1111/j.1467- pandemic. Journal of Education for Teaching, 46(4), 609–611. 8535.2009.00982.x https://doi.org/10.1080/02607476.2020.1755205 Wang, F., & Hannafin, M. J. (2005). Design-based research and Mor, Y., & Winters, N. (2007). Design approaches in technology- technology-enhanced learning environments. Educational enhanced learning. Interactive Learning Environments, 15(1), Technology Research and Development, 53(4), 5–23. https:// 61–75. https://doi.org/10.1080/10494820601044236 doi.org/10.1007/bf02504682 Nixon, E. K., & Lee, D. (2001). Rapid prototyping in the instructional Xu, H., & Morris, L. V. (2007). Collaborative course development design process. Performance Improvement Quarterly, 14(3), for online courses. Innovative Higher Education, 32(1), 35–47. 95–116. https://doi.org/10.1111/j.1937-8327.2001.tb00220.x https://doi.org/10.1007/s10755-006-9033-5 Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Yang, J. C., Quadir, B., Chen, N.-S., & Miao, Q. (2016). Effects of Thematic analysis: Striving to meet the trustworthiness crite- online presence on learning performance in a blog-based online ria. International Journal of Qualitative Methods, 16(1), 1–13. course. The Internet and Higher Education, 30, 11–20. https:// https://doi.org/10.1177/1609406917733847 doi.org/10.1016/j.iheduc.2016.04.002 O’Shea, S., Stone, C., & Delahunty, J. (2015). “I ‘feel’ like I am at Young, C., & Perović, N. (2016). Rapid and creative course design: university even though I am online.”: Exploring how students As easy as ABC? Procedia – Social and Behavioral Sciences, narrate their engagement with higher education institutions in 228, 390–395. https://doi.org/10.1016/j.sbspro.2016.07.058 an online learning environment. Distance Education, 36(1), Zhang, L.-C., & Worthington, A. C. (2017). Scale and scope econo- 41–58. https://doi.org/10.1080/01587919.2015.1019970 mies of distance education in Australian universities. Studies in Quintana, R. M., & Tan, Y. (2021). Visualizing course structure: Higher Education, 42(9), 1785–1799. https://doi.org/10.1080/ Using course composition diagrams to reflect on design. 03075079.2015.1126817