(PDF) Designing Mobile Learning Interventions for Language Learners

Educational design research – Part B: Illustrative cases Chapter 45 Educational Design Research: Designing Mobile Learning Interventions for Language Learners Agnieszka Palalas & Terry Anderson SLO • Netherlands institute for curriculum development Palalas, A., & Anderson, T. (2013). Educational design research: Designing mobile learning interventions for language learners. In T. Plomp, & N. Nieveen (Eds.), Educational design research – Part B: Illustrative cases (pp. 967-990). Enschede, the Netherlands: SLO Contents 45. Educational design research: Designing mobile learning interventions for language learners Abstract 969 1. Introduction and overview 969 2. EDR methodology and research design 974 3. Study results 984 4. Reflections on the EDR method 986 Key source 987 References 987 Credits 2013 SLO (Netherlands institute for curriculum development), Enschede All rights reserved. This publication may be reproduced in whole or in part by photocopying or any other means, provided the source is mentioned. 45. Educational design research: Designing mobile learning interventions for language learners Agnieszka Palalas & Terry Anderson Abstract This chapter presents a case of an EDR study completed at a Canadian community college and resulting in the development of an innovative educational intervention, Mobile–Enabled Language Learning Eco-System (MELLES), as well as corresponding MELLES design principles, which emerged from this interdisciplinary research experience. The first section of the chapter provides an overview of the educational problem targeted by the study, the purpose and outcomes of the research, as well as the overarching research question. The description of the EDR methodology then follows including its phases, cycles and micro-cycles. The MELLES study adopted the Integrative Learning Design Framework (IDLF) (Bannan, 2009) for design-based research and the corresponding nomenclature. Accordingly, we refer to the preliminary phase of conceptualization as Informed Exploration, followed by the design/development phase called Enactment, and the assessment phase referred to as Evaluation: Local Context. The Purpose and Outcomes and the Study Results sections of this chapter summarize the key outcomes of the study which included the development of a prototype MELLES educational intervention, replicable design principles guiding the creation of such an intervention, a refined theoretical framework of Ecological Constructivism and a comment on the professional development benefits reaped by the study participants and observers. With emphasis given to the praxis of the EDR approach, the Reflections section revisits the main features of the EDR method as distilled from our study, which demonstrates that EDR both enhanced the design and implementation of the study and was able to guide measurement of its efficacy in this context. 1. Introduction and overview Educational problem The increasing demand for global communications skills and especially capability to communicate in English has focused both demand and research on ways and technologies to enhance the efficiency and effectiveness of interventions designed to help learners achieve high levels of English language proficiency. Study has included a variety of teaching and learning interventions (Golonka, Bowles, Frank, Richardson, & Freynik, 2012) and more specifically a focus on studies that support learning both within and outside a formal language learning classroom (Underwood, Luckin, & Winters, 2012). Together these studies suggest that there is support for the notion that technological tools and especially those that are accessible, have potential to improve English language teaching and learning. Mobile-Assisted Language Learning (MALL) draws on the attributes of enhanced mobility, access and flexibility afforded by mobile technologies. It “offers learning that is potentially independent of location, time and space” (Palalas, 2012, p. 26) accommodating students’ preferences and schedules. At the same time, it enables language learning that can be  969 informed by the context in which it is occurring. MALL “affords exposure to authentic language samples and challenges in location-specific communicative situations and provides supports required for such situated learning” (Palalas, 2012, p. 26) for example in the form of glossaries related to the context-embedded tasks (Demouy & Kukulska-Hulme, 2010). The notions of mobility of the learner, space, technology, and time as well as informal and contextual learning have been interwoven into the latest definitions of m-learning (Brown, Börner, Sharpless, Glahn, De Jong, & Specht, 2010; Kukulska-Hulme, Sharpless, Milrad, Arnedillo-Sanchez, & Giasemi, 2011; Pachler, 2009). These aspects of mobile learning combined with the mobile network connecting resources, peers, teachers and other speakers of a language can potentially transform any context into a language learning space. Moreover, the environment in which language practice takes place is full of learning supports and affordances which can be optimized with the help of mobile devices, as described in more detail in the discussion on Ecological Constructivism. Interaction and communication with interlocutors and with the context is mediated by “cultural tools such as language and technology” (Pachler, 2009, p. 5). Mobile tools also afford a sense of ownership leading, through personalization of these tools, to individualized learning and learner agency (Pachler, Bachmair, & Cook, 2010). They also facilitate cognitive processes by making information available and presented in a way that avoids learner memory overload (Pachler, 2009). This study investigated how the affordances of mobile technology can promote language learning based on sound pedagogic principles. We also sought to investigate what particular capabilities of mobile technologies should be incorporated in designing MALL instruction. “Language is contextually contingent; therefore, the mobility of the learner across diverse authentic contexts potentially enables situated language practice” (Palalas, 2012, p. 29). The specifics of what design supports MALL and more specifically the acquisition of aural skills as well as the related design principles had to be thoroughly examined. The problem of inadequate aural skills amongst English as a Second Language (ESL) college learners has been identified as a key barrier to the students’ academic and professional success (CIITE, 2004; Palalas, 2009). Previous research at George Brown College, conducted prior to September 2009, determined a need for ESL intervention which would provide language learning instruction and practice going beyond the standard 52-hour course and facilitating flexible learning to accommodate the adult learners’ daily schedules and commitments. It was concluded that a blend of in-class and mobile-assisted aural practice situated in a real world language context would likely be the most effective approach to facilitate listening skills acquisition (Palalas, 2009). Moreover, findings of this previous research at the college and other educational contexts (Demouy & Kukulska-Hulme, 2010; Palalas, 2011; Rosell-Aguilar, 2007) highlighted student satisfaction with the language training offered on mobile devices experience and the need for interactive activities conducted outside classroom walls. Mobile technologies are perceived by students as helpful and appropriate for language teaching and learning (Demouy & Kukulska-Hulme, 2010). However, in response to the particular needs of our ESL students, it was vital to investigate questions pertaining to the design and delivery of a Mobile-Enabled Language Learning (MELL)1 solution enabling the development of aural skills by augmenting in-class learning through effective utilization of students’ own mobile devices outside the classroom. That necessitated an in-depth understanding of the interplay of technical 1 MELL is being used in place of a more widely-used Mobile-Assisted Language Learning (MALL). The modified term emerged from the findings of this study, emphasizing the role of mobile technology as an enabler of the learning process.  970 and pedagogical aspects of such an educational intervention and the context in which it was to be utilized. No guidelines or standards were available for creating this type of context-specific mobile-technology-enabled educational solution. Purpose and outcomes Hence, this 2009-2011 EDR study was designed to formulate replicable design principles and demonstrate their applicability through an innovative educational intervention which eventually evolved into a Mobile-Enabled Language Learning Eco-System (MELLES). These two primary outcomes also contributed to the broader purpose of optimizing the college’s ESL instruction through improved effectiveness and appeal. In addition, the study increased the awareness of the problem and the potential of the MELL approach both in the local and broader educational context. Lastly, the theoretical framework underpinning the study gradually evolved into Ecological Constructivism, melding Social Constructivism, Socio-cultural Theory and Ecological Linguistics. The key MELLES study outcomes are briefly characterized in the following paragraphs. Educational intervention: MELLES prototype This EDR study produced a functional prototype of a Mobile-Enabled Language Learning system created to support ESL practice and future studies of comparable MELL solutions. The MELLES intervention is a web of interlinked learning tasks and supports which can be accessed through the mobile website (Figures 4 and 5). It provides a language learning ecosystem incorporating mobile, web-based and face-to-face environments. Learners are asked to complete eight communicative tasks which are embedded in real-life language situations in the streets of Toronto. Following task directions, learners are encouraged to participate in authentic communicative challenges, negotiate meaning, and create their own language artifacts. These learning tasks combine individual focused practice with group communicative activities, rehearsed speech with ad-hoc discourse, and comprehension exercises with impromptu meaning-making challenges. As a result, learners, using their mobile phones, create authentic language artifacts (audio recordings, photos, video, and text) which they exchange through the MELLES website. It is the mobile technology that enables the interaction among the essential elements of the design, including pedagogy, content, context, actors, and digital communication channels. By offering access to information and to others, mobile devices not only mediate individual cognitive processes but also the co-construing of meaning through interaction and communication. The mobile website serves as a gateway to the MELLES network thus connecting people and the language learning resources. Learners can access the eight language tasks and related materials from anywhere at a time of their choosing to work on them either individually or in groups. The tasks are also designed to offer flexibility in what sequence and location the individual tasks or their components are completed. Such flexibility is combined with the appropriate amount of guidance and direction - learners are guided in their learning process through task prerequisites and contextualized collaborative assignments which serve as the learning process signposts. As motivational and scaffolding measures, learners are encouraged by reminders and notifications pushed to their mobile devices to complete language activities leading to the collaborative tasks. To ensure a seamless learning process and collaboration, the MELLES site serves as a hub an exchange and communication platform where all resources are aggregated and shared. All task-related materials, available through the hub, are selected and validated by experts to provide personalized information as well as to aid learners in managing the abundance of language resources available on the Internet. Similarly, the learner-generated artifacts and  971 information contributed by them are evaluated and rated by language experts. Evaluation and feedback exchange, expert scaffolding and other interactions are made possible by connection management tools (a blend of online and offline functions) which offer either on-demand or delayed communication. To this end web-based resources were integrated with customized apps residing on the user mobile device. The resulting mobile solution enabled access to the web of MELLES components providing improved situated aural-skill-acquisition experience. Like many EDR interventions, MELLES was iteratively designed, developed and evaluated to produce a pedagogically-sound m-learning solution addressing the need at hand with its specific purpose and target audience. By pedagogically-sound, we meant an intervention created and evaluated following the main theoretical framework of the study and designed to promote learning of listening skills. Building on findings of the Informed Exploration phase, a number of stand-alone mobile applications and mock-ups were initially proposed. Informed by the feedback collected through recursive Enactment and Evaluation phases these gradually evolved into MELL listening tasks before a more systemic MELLES framework eventually integrated these tasks into a complete solution. Accordingly, a prototype MELLES system (Figures 3-5) was created as an instantiation of the design principles, which Plomp (2009) refers to as intervention theory. The system interface, namely the project website, was accessed from students’ mobile devices for the tests and evaluation of both the prototype and the corresponding design guidelines. The iterative process of the design, development and evaluation of the MELL solution, as well as the evolution of thinking facilitated the creation of successive instantiations of the design theory. In turn, enhanced understanding of the essential features and functions of the educational intervention resulted in MELLES design principles which encapsulate the findings of this EDR study. Design principles While the first attempts at designing MELL models drew primarily from the design principles identified by current literature and the earlier studies at George Brown College, the conceptualization and development of the successive prototypes were driven by the EDR feedback and design guidelines emerging progressively from each cycle of the study. Ultimately, through iterative refinement, these principles evolved into what the MELLES pilots demonstrated to be pedagogically useful guidelines (elaborated in detail in Palalas, 2012). Reflecting the results of multiple feedback loops and insights of designers, programmers, teachers, and learners, the MELLES design principles refer to both the pedagogical and technological aspects of the intervention, and the interconnections between the two dimensions. The design principles encapsulate all the essential pedagogical characteristics of an effective MELLES intervention, including content, procedures, context, and actors (for instance, “Integrate learner-generated linguistic artefacts: audio, video, photos, images”). They also incorporate the technical dimension of the system pertaining to the functionality, tools, and technological context required (for example “Build in personalized user progress tracking capabilities”). To clarify, the design principle refers to how to design an intervention including the desired characteristics (as identified in the process of the DBR study). Accordingly, all MELLES design principles comprise three parts: (1) an essential characteristic of the MELL system (substantive emphasis), as extracted from the prototypes and the process of evaluation, (2) a strategy for realizing this unique feature (procedural emphasis), and (3) rationale for including the characteristic (see Table 2). The ensuing intervention theory aimed to inform future design of MELL listening practice, as well as provide an improved understanding of the praxis of mobile learning. The final MELLES design principles are thus formulated as heuristic statements and,  972 as recommend by van den Akker (1999), they include substantive and procedural knowledge. All in all, the resultant design principles were formulated to guide ESL practitioners and they are not “intended as recipes for success, but to help others select and apply the most appropriate substantive and procedural knowledge for specific design and development tasks in their own settings” (McKenney, Nieveen, & Van den Akker, 2006, p.119). Theoretical framework To ensure pedagogically-sound outcomes, the study was guided by current second language learning pedagogy and a constructivist theoretical framework. Socio-cultural Theory (SCT) (Lantolf, 2000), was initially selected as the framework for the study and subsequently reconceptualized to reflect an evolving understanding of the appropriate MELL intervention. The SCT paradigm derives from Vygotsky’s theory of Social Constructivism and as such, integrates the elements of mediation, goal-oriented learning, the Zone of Proximal Development (ZPD), as well as a strong emphasis on language, mediation and learning through interaction in a community. SCT also regards speaking (social interaction) and the internal cognitive process of thinking as being strongly interconnected and enabled through repeated interaction with the context and other people (Lantolf, 2000). In addition, for the learner to achieve independent performance, interactivity should be supported by scaffolding coming from facilitators or peers (Vygotsky, 1978). In the Mobile-Enabled Language Learning context that scaffolding is enabled by the mobile technology which affords access to others and to learning resources. To incorporate the important elements of constructivist learning theory coupled with the importance of a mobile context, the affordances of mobile ICT devices, and the contextembedded language learning, we applied the idea of Ecological Constructivist Learning (Hoven, 2008; Hoven & Palalas, 2011). Ecological Constructivism as a theory incorporates the multiple dimensions of Ecological Linguistics (Halliday, 1993; Lafford, 2009; Lam & Kramsch, 2003; Van Lier, 2000) and Constructivism. It encompasses “general learning and language learning more specifically, while also engendering an ecological approach to research methodology” (Hoven & Palalas, 2011). The key characteristics of the ecological nature of our research encompassed the emerging and evolving character of findings, research being done in the authentic teaching and learning context, findings being “ecologically valid”, i.e., reflecting and relating to real-life situations and exploring interventions preparing learners for real-world application of knowledge and language skills, the study applying a holistic lens to data collection and analysis including the interplay of the various aspects and actors involved in the teaching and learning relationship. The ecological perspective on learning adds a new dimension to the SCT emphasis on the interaction and co-creation of knowledge: the significance of the dynamic real-life context offering communicational challenges, potential supports and linguistic affordances. Thus our Ecological Constructivist framework highlighted the dynamic interconnectedness between (1) cognitive processes internal to individuals, (2) the linguistic dealings and supports from other human beings, (3) the mobile tools used to mediate the relationships and provide resources, (4) the authentic language situations that while challenging learners also provide support (linguistic affordances), and (5) the environment in which the parts of this system interact. The role of the context, in this case the English speaking real-world setting, and its affordances was the focus of this ecological metaphor. Thinking in ecological and systemic terms of connectedness, relationships, processes and context allowed to develop educational designs that addressed the need for “whole language learning, namely, practicing it as a whole system (as opposed to studying the parts of speech or only one language skill in isolation), learning it in a whole context of students’ life, as part of the whole learning community, and in the whole environment of the particular language situation students encounter” (Palalas, 2012). This more holistic  973 approach to the process of learning, evolved from the feedback gathered in the Informed Exploration phase. Professional development It was observed during the study that, while contributing their expertise, practitioners were able to enrich their understanding of the novel educational technology, its application in the context of language learning and, most importantly, the interplay of pedagogy and technology in practice. This two-year long study also benefitted from the progressive informal learning of all stakeholders. While students had an opportunity to design and develop real-life educational applications, practitioners enriched their understanding of the novel educational technology, its application in the context of language learning and the actual interplay of pedagogy and technology. Over time, this also produced better-informed input and research results. In addition, understanding and awareness of the practical applications of mobile learning increased across the college impacting a number of strategic and professional development initiatives. Research question The study was guided by the following research question: What are the characteristics of an effective, pedagogically-sound MELLES for students’ mobile devices, through which adult ESL students in a community college enhance listening skills, while expanding their learning outside of the classroom? Similar to other aspects of the EDR study, the question evolved replacing the original notion of a learning object with the MELLES intervention. The question inquired about the intended intervention, and what characteristics are vital for its design to be effective and pedagogically-sound, i.e., MELLES was created and evaluated following the main theoretical framework of the study and designed to promote learning of listening skills. The intervention was designed for ESL college students studying at the college programs in the area of business, accounting, hospitality, and technology. In terms of effectiveness, it was measured by participant feedback on perceived learning as well as their satisfaction with the design of the intervention and the learning experience. Using their own mobile devices students piloted MELLES in the streets and at landmarks of Toronto, where they interacted with the tasks and interlocutors (often native speakers of English), in a dynamic language environment, which both supported their language practice and challenged them to make meaning and communicate. Given the multiphase process, supplementary research questions, informed by and congruent with the various phases, emerged in the EDR process; they were then reflected in the surveys, focus groups and interview questions. Details of how the data were collected and analyzed are provided in the next section. 2. EDR methodology and research design Procedure Guided by the IDLF model (Bannan, 2009) (Figure 1), this EDR study progressed through four cycles of the phases of Informed Exploration, Enactment and Evaluation (IE-E-E); however it is worth noting that the phases did not always progress in a linear fashion, instead a particular phase of one cycle would overlap with a different phase of the next cycle; for instance when prototype 2 was being developed (Enactment), elements of prototype 1 were still being tested and evaluated providing feedback that was analyzed in a timely fashion and incorporated into prototype 2 (see Figure 2).  974 Figure 1: The ILDF model. Modified with permission (Bannan, 2009) In the dynamic educational context it was difficult to identify all the solution requirements up-front or exclusively at the evaluation milestones, instead the design gradually emerged trough cycles of the MELLES prototype evaluation and input from its end-user. Moreover, formative evaluation was conducted in an agile fashion throughout the project to manage the changing design requirements. Similarly, the various activities of the EDR process and their timelines were frequently adjusted to optimize the time and human resources available to the project. Accordingly, the three phases of the progressive IE-E-E cycles often partially overlapped (Figure 2). In total the study involved the development of three functional prototypes and implementation with 142 students (excluding the Informed Exploration focus group of 21 students and 191 students who completed the Mobile Device Usage Survey). Feedback was also collected from eight professors from the School of Design (2), School of Computer Technology (3), School of Business (1), Centre for Hospitality and Culinary Arts (1) and the Intensive English Program (1) - as well as two IT and mobile programming experts from outside the college. The data collection included 18 student and 26 expert interviews, 7 surveys and 12 focus groups. The figure below introduces the main study phases including their timelines, milestones, main activities, outcomes, as well as data and participants. The guiding questions addressed in each of the phases are presented in the table on the next page (Table 1).  975 Table 1: Guiding questions for each phase Informed exploration questions Enactment questions Evaluation questions 1. 1. 2. 3. 4. 5. 6. 7. 8. What is the specific ESP problem and corresponding educational needs to be targeted? What information can be gleaned from related previous studies at the college as well as existing data and research? What are the characteristics of the target population and the specific teaching and learning context (its unique requirements and limitations)? Are students and faculty ready to use mobile devices for learning? What is their mlearning experience? What are their mobile habits? What devices with what capabilities do they own? What comparable interventions have been identified in literature? What are the characteristics of their design? What preliminary design principles can be distilled from literature considering the specific context and target audience? What is the appropriate theoretical framework for the DBR study and the design of the intervention? What is the desired theoretical construct (ideal to guide the design)?  976 1. 2. 3. 4. 5. 6. 7. 8. What are the desired characteristics of the intervention and the rationale for including them in the design? What are strategies for incorporating those characteristics into the design? What are the consequential design principles? What are the sine-qua-non design requirements (the specific features and functionalities, e.g., technical specifications, instructional content, site architecture, etc.)? Does the design meet the learning objectives of the intervention? What are the practical (technical, social, cultural, and organizational) influences and constrains on design and development? Are the selected theoretical construct and framework feasible? What are the processes, procedures, and roles involved in the design, development, and implementation? 2. 3. 4. 5. 6. 7. 8. 9. What are the essential characteristics of the effective pedagogically-sound intervention and the rationale for including them in the design? Is the piloted design usable, valid and relevant in the specific educational context including its technology infrastructure? How is the design and the use of intervention impacted by the environmental factors, local cultures and policies, as well as the technology infrastructure? What are the specific requirements of the learning environment that impact adoption of design?” How effective is the design in terms of learning goals and target audience satisfaction? How effective are the distilled design principles? Does the design follow the selected theoretical framework? Is the intervention accessible to all end users? What’s the impact of the innovative design on the studied audience and its larger environment? Figure 2: EDR phases: Timeline, main activities, data source and outcomes Note. * See the list of questions in Table 1.  977 The following sub-sections further describe the activities, procedures, participants, and outcomes of all four cycles of each phase. With the reader in mind, the section is organized by IDLF phases despite the recursive character of the process. Pilot study Before the actual EDR research, a pilot study had been carried out to test the research approach and some of its constituent elements. From September 2009 to January 2010, the pilot progressed through one iteration of the Informed Exploration and Enactment phases. Fourteen School of Design (Digital Design) students were involved in the design of the same number of MELL conceptual models following an overview of m-learning and second language acquisition offered by the researcher. Students engaged in creation of a conceptual map for mobile learning objects as well as mock-ups and proposals for the design of such software. The initial conceptual framework was then mapped out based on the investigation of literature and refined through dialogue with Digital Design students and professors. The key finding of the pilot phase was the realization that the various mobile learning objects (MLO) created by the students did not constitute a solution to the education problem under investigation since they were disconnected MLOs and did not provide opportunities for sufficient practice of listening skills – a more holistic approach was needed. Moreover, some design and technological constraints of the m-learning intervention became apparent, for instance the cross-platform issue. Finally and most importantly, the EDR approach to the study was tested and streamlined to match the requirements of the context. The pilot thus provided an opportunity to observe the EDR process and participants in practice and to devise the research questions (Table 1) and strategies for the study. It prepared the stage for the full round of Informed Exploration, depicted in the following sub-section. Informed exploration phase This stage of research began with a careful formulation and analysis of the existing educational problem followed by in-depth exploration of the target audience and practitioner perceptions to further the understanding of the specific ESL student needs. The particular college ESL learner population had been previously studied by means of exploratory research, which identified using handheld devices as an appropriate approach for enhancing listening skills of the students. Nevertheless, more focused understanding of their needs in terms of learning listening with mobile devices was needed. Accordingly, results of the earlier studies and the EDR pilot were incorporated. To re-articulate learning targets, these findings were combined with the feedback from students and practitioners from the earlier m-learning research, the School of 2 3 4 Design , and the School of Technology as well as Communications/ESL professors. The data was collected via three student focus groups, practitioner interviews and meetings, and then analyzed for common themes. The additional analysis of the context highlighted the systemic social, cultural, and organizational influences as well as constraints on the intervention design. 2 Two School of Design programs were involved, namely (1) Digital Design - Advanced Digital Design Program (Postgraduate, 2 semesters) and (2) Digital Design - Game Design Program (Postgraduate, 3 semesters). 3 Participants represented two School of Technology programs, namely (1) Computer Programmer Analyst (6 semesters) and Wireless Networking (Postgraduate, 3 semesters). 4 These included ESL courses for Business and Communications courses for Business Administration, Business Accounting as well as Hospitality and Culinary Arts.  978 A survey (n=191 ESL students) of GBC students’ use of mobile devices was conducted college-wide to determine what mobile devices learners owned and how they used those technologies. Its aim was to understand participants’ mobile device usage patterns and mlearning experience in order to gauge their readiness for MELL solutions. Through student and practitioner feedback coupled with comprehensive review of comparable solutions and relevant literature on second language acquisition, m-learning and instructional design, the MELL listening intervention was conceptualized as a an ecological system (MELLES) requiring a mobile web hub which connects the main components of the whole flexible contextualized learning solution: learners, facilitators, other speakers of English, language and learning resources (including the mobile listening tasks), as well as the real-world environment in which learners complete the language tasks provided by MELLES. Accordingly, the abovementioned Ecological Constructivism framework was selected to guide the study. The key interconnected elements of MELLES are depicted in Figure 3. Figure 3: MELLES - Key interacting components of the system (Palalas, 2012) The educational technology under investigation was explored via on-going dialogue with the professors and students from the Programming, Wireless Networking as well as Digital Design programs. The existing data gathered through the pilot study helped to distil the preliminary design principles and thereby provide a sound theoretical base for the progressive development of the MELLES design framework. The twenty six initial design guidelines highlighted the significance of the pedagogical aspects of the design; for instance, they emphasized the features of the MELLES intervention supporting (1) active listening and engagement in the English language speaking environment outside the classroom, (2) communication through interaction mediated by mobile devices and the device inherent functions, (3) negotiation of meaning drawing on the affordances of the context, (4) applying creative effort to communication situations, (5) mobile access to others and to multimedia linguistic resources, learning tasks instructions and materials. A number of evaluation questions, including those concerning usability, validity, and relevance of the design, also evolved from the exploratory phase to guide the study and to form a base for the MELL surveys. On the whole, Informed Exploration resulted in an ideal which provided “a vision and a guide as well as a significant component of the measuring stick by which the ideal, as instantiated in actions within a real context, is measured” (Anderson, 2007).  979 Enactment phase In this highly observable production phase a number of prototypes were proposed and designed in cooperation with the School of Design and School of Technology students and practitioners. Multiple attempts at designing stand-alone learning objects and elaborate learning tasks gradually led to a more holistic solution that combined eight language tasks through a mobile website offering tools for collaborations and communications as well as scaffolding from peers and facilitators; it also served as a hub for the community of learners and an access point to linguistic resources. Hence, the ongoing evaluation of the iterative design ideas and models combined with the new ecological paradigm resulted in construction of the MELLES prototype (Figures 4 and 5). This mobile web-based system encompassed the properties and functions of the constituent MELL listening tasks, their interactions, the dynamic real-life context of the tasks, the website interface, as well as the enabling mobile technology components. Figure 4: Screenshot of final mobi-english.mobi audio tasks (mobile interface)  980 Figure 5: Screenshot of mobi-english.mobi (desktop interface) Central to the EDR process was the design of the evolving versions of the MELL intervention which were concurrently evaluated via Phase 3 pilots and ad-hoc formative evaluation exchanged by participants at various points of the IE-E-E process. The MELLES design, development, and evaluation cycles had to be relatively systematic to correspond with the academic schedules of the programs involved. The EDR activities were coordinated with the Digital Design, Computer Programmer Analyst (also referred to as Programming), Business, and Intensive English Program (IEP) courses being integrated in the study as part of curriculum and becoming the core assignment of the respective courses. For instance, the project-based 5 School of Design postgraduate course had been revised to include the design of a real-life MELL prototype as its course outcome. Consequently, all students in these courses participated in the design and evaluation activities as part of their program. In fact, some also volunteered to share their feedback on their own projects. In addition, four School of Technology students and one School of Design student invested many volunteer hours outside of the program to work on the MELL design. During Enactment, and to a lesser degree other phases of the study, the researcher was frequently invited as a Subject Matter Expert to share her knowledge of second language acquisition and m-learning with the students and practitioners. This ongoing exchange of 5 Interface Design, part of Digital Design - Advanced Digital Design Program (Postgraduate). In this project- based course students, in three successive graded projects, created proposal of prototypes for MELL solutions for various mobile platforms. They discussed, created and presented in class MELL concept maps, mock-ups, MELL proposals, system requirements chart and visual presentations of the user interface design.  981 expertise between the researcher and the participants formed the basis for the validity and applicability of the theoretical framework necessary for the design and implementation of the m-learning intervention. Subsequently, participants produced individual MELL listening prototypes. Their designs were circulated and critiqued by their professor, the researcher and the other practitioners involved. These ideas were documented through their assignments, design specifications, designer logs, wireframes and system requirements charts. This documentation, after students’ consent, was then analyzed by the researcher resulting in modifications to the proposed designs. Upon consultation with practitioners, the ensuing prototype designs served as a starting point for the development of corresponding digital constructs. Consequently, the Programming students and their professors developed the key components of the intervention prototype based on the detailed design documentation from the 6 School of Design course. In common with the Digital Design course, the Programming course included a mobile learning software development project as its core assignment. Formative feedback based on the student projects contributed to the final MELLES prototype design integrating eight m-learning listening tasks into a more complete solution. The mobile technology aspect of the system constituted another part of the Enactment phase. Research 7 and trials led to the adoption of the WordPress mobile web framework, thus optimizing cross-platform access to MELLES and its components. Concurrent with the product design and construction, subsequent versions of design principles were sketched and fed back into the system. All research decisions, processes, constraints and other usable knowledge were recorded and analyzed as qualitative data. This feedback combined with the Evaluation: Local Context data and formative feedback shared throughout all EDR cycles and activities formed the foundation for the research outcomes. The Evaluation phase of the study is presented below. Evaluation: Local impact phase This was the final phase of the EDR process - the Evaluation: Broader Impact phase suggested by the IDLF model was out of scope of the MELLES study. Interestingly, Herrington and Reeves’ (2011) revision of the Baynton model has substituted development of design principles for the “broader impact” phase and thus is consistent with our study. Data essential for evaluating the product and process of the innovation design were gathered during individual prototype tests and pilots of MELLES listening tasks and the website. This feedback was collected through project meetings, interviews, surveys, focus groups, as well as captured in researcher observations and memos. Ad-hoc formative feedback was incorporated in the analysis as well. Both online (Elluminate, Skype, Wiggio, Zoomerang) and face-to-face channels were employed to capture “the intended and the unintended consequences of the intervention” (Anderson, 2005, p. 3). The resultant mixed data captured in text-based documents, images and audio files were subsequently analyzed using the NVivo analytic software, Excel, and the SPSS statistical predictive analytics software. The aggregated data were analyzed at the five 6 Mobile Application Development is a last-semester course of the three-year Computer Programmer Analyst program. Through hands-on projects, students gain experience in developing and deploying wireless applications on mobile platforms. 7 WordPress Mobile Pack. A toolkit which helps mobilize any WordPress site (available via Internet from any mobile platform) and its components; it includes a mobile switcher which toggles between the desktop and mobile view, a selection of mobile themes, widgets, device adaptation and a mobile administration panel to allow users to edit the site.  982 project milestones (noted in Figure 2). The ensuing findings were reflected in the evolving MELLES design. 8 During the real-life pilots of the educational intervention, ESL students used their own devices to complete the out-of-class listening tasks and evaluate their effectiveness. The prototype pilots were conducted with five groups of ESL students from eight different college programs representing the target group. A purposive sampling strategy was employed to recruit these student-participants. Their feedback was collected through questionnaires distributed to 6 classes of students (n= 101), seven focus groups, and five interviews (Figure 2). In addition, one Digital Design and two Programming students, as well as two external mobile and IT programming experts, tested the prototype MELLES tasks and shared their observations through individual interviews. Practitioner input, based on their involvement in the MELLES design work and observations of the students, offered an invaluable perspective as well. The combined participant feedback addressed questions about the intervention usability, validity, relevance in the context of the learning process, and the specific needs of the ESL student population. Hence the designers’ and potential users’ responses helped to identify the essential characteristics and components of the MELL system under investigation. Participants also commented on the overall effectiveness of the MELLES approach and their perceived attainment of the learning outcomes targeted by the intervention. The MELL system and the corresponding design guidelines were modified after each round of pilots. With data collection points being staggered across the EDR process, sufficient time was allocated for the system redesign. The considerable amount of qualitative data gathered from these EDR cycles was rigorously analyzed, coded and recoded, and thus the design characteristics identified by participants as vital for the educational intervention were systematically worked into the design. All data sources (text, images and audio files) were integrated into the NVivo system. Codes were then generated and assigned in a cyclical fashion to phrases and sentences through repetitive thematic analysis driven by the main research question. Due to the amount of data accumulated, some pre-coding techniques were first employed and then elaborated by the Descriptive Coding (with codes identifying the topic of the respondents’ comments rather than their details) in the initial stages of the analytical work and followed by Focused Coding methods (Saldaña, 2009). Focused Coding allowed for astute questioning, exploration of meaning, re-conceptualization of concepts which led to reformulating of categories. After multiple iterations and analysis of all data, the researcher was able to see the results through a more systemic lens. The analysis of relationships of the categories, their properties and dimensions as elements of the MELL system and how they interplay resulted in the final collection of codes addressing the research question. The rigorous examination of participant comments and reoccurring themes, as well as the interdependencies of those themes, led to the generation of the theory put forth by this study. The feedback highlighted the multiplicity of required pedagogical and technological components of the MELLES system, which was reflected in the final coding themes (NVivo nodes). The main theme categories included the following: 8 Two students chose to borrow iPod Touches from the project.  983 Pedagogy 1. Pedagogic procedure – How? 2. Content – What? 3. Context – When and where? 4. Actors – Who? Technology 5. Functionality – How? 6. Tech Context – When and where? Moreover, quantitative findings validated these qualitative findings which provided a much more elaborated and informative perspective. At the same time, the effectiveness of the MELLES system built according to these requirements was measured through attitude evaluations, which indicated overall very high satisfaction with the final prototype of MELLES. The key study results are revisited in the next section to accentuate the key characteristics of the MELLES educational intervention. 3. Study results While the key outcomes of the study were described in the Purpose and Outcomes section, the MELLES design principles are revisited below (see Palalas, 2012 for a detailed discussion of the MELLES educational intervention and design principles). Design principles The key characteristics of the MELLES prototype are also reflected in the design principles which evolved from its consecutive versions. Ten pedagogical and seven technology-related principles were distilled from the design and its evaluation. As mentioned before, these design guidelines have three components: (1) they refer to the essential characteristics of MELLES (substantive emphasis) and (2) the strategies required to operationalize those features (procedural emphasis); (3) the rationale for the inclusion of the substantive and procedural recommendations is also included as demonstrated in the example below (Table 2).  984 Table 2: Evaluation: Design principle 1 Essential characteristic (Substantive emphasis) Strategy (Procedural emphasis) Rationale (In order to …) Balanced combination of individual and collaborative (group work) tasks. • • • • • • • • • Ensure communication with others in-person and via mobileenabled channels. Build in interaction with others in person and via mobile-enabled channels. Include discourse with diverse interlocutors. Incorporate language problems requiring negotiation of solutions. Inject fun and challenge. Ensure dynamic meaningmaking and negotiation. Maintain regularity of group/class activities. Build individual tasks to feed into the group tasks. • • • • • • • • Mediate communicative practice and communication (language usage). Allow for cognitive and collaborative knowledge creation. Enhance individual and group motivation. Offer peer scaffolding and support in problematic situation. Provide flexibility - time and place independent learning. Accommodate different pace of learning and levels of language proficiency. Support the learning network in and out of class. Support cognitive processes with social process. Glue the MELLES system together. Keeping the length of this chapter in mind, a brief synopsis of the design principles is presented below exclusive of the strategy and rationale discussion (see Palalas, 2012 for a detailed discussion). The following are the ten essential pedagogic characteristics of MELLES which form the basis of the design principles: 1. Ensure balanced combination of individual and collaborative (group work) tasks. 2. Integrate learner-generated linguistic artefacts (audio, video, photos, images). 3. Incorporate game-like real-life communicative tasks. 4. Build in expert facilitation: scaffolding, feedback, and coordination. 5. Include feedback mechanism (immediate and delayed). 6. Focus on authentic listening tasks in the dynamic real-world communicative situations. 7. Design self-paced individual audio tasks to support and feed into/prepare learners for the real-life language tasks. 8. Integrate all four language skills but focus on listening outcomes. 9. Incorporate linguistic resources (task-related): relevant vocabulary, dictionaries, pronunciation, clear task directions and explanations, examples of language usage. 10. Support out-of-class learning with in-class (f2f) instruction and practice (a blend of in-class and out-of-class context). These correspond to the following technology design principles: 1. Provide one-point access to all resources (mobile web). 2. Enable exchange of information and artifacts as well as communication through the mobile-web portal. 3. Ensure scalability, flexibility and adaptability of the system.  985 4. 5. 6. 7. 8. Build in a scalable rating system (from artefact to learning structures to the whole system). Incorporate multimedia (including text) artefact authoring, management and usage capabilities. Build in cross-platform and multi-technology support. Integrate technology support and tutoring /instruction. Include personalized user progress tracking capabilities. With 47% of students strongly agreeing and 45% agreeing (7% - neutral, 1% - disagree) that MELLES provided an effective way of learning listening skills, students reported a high level of satisfaction with the system. Based on their perceived learning and positive learning experience, all participants found the intervention highly effective. However no evidence of learning was collected through formal assessments of students’ progress. This limitation stemmed from the multiplicity of other variables under investigation and time constraints of the study. In addition to the above-mentioned outcomes, the EDR method was thoroughly tested and optimized though the MELLES study which is discussed in the following section. 4. Reflections on the EDR method In our ERD experience we co-generated new knowledge through enacting sound pedagogical theory through shared practice and reflection. The practice of producing the MELLES educational intervention based on its in-situ evaluation was made possible through collaboration and communication of stakeholders partaking in that activity - including designers, educators, students and teachers. The ensuing shared understanding of the effective practical design evolved into the MELLES design and theory. The EDR participatory design and reflective practice was augmented by the fact that many of the study participants were both system designers and its end-users contributing a unique understanding of language learner needs. Hence, the study benefited from a learning process in which designers and users learned from each other and exchanged feedback on an ongoing basis. Owing to the early feedback combined with continuous meaningful evaluation at each EDR iteration milestone and between these formal data collection points, the process adopted for the MELLES study produced outcomes which accurately reflected the changing requirements of the study participants as their understanding and thinking evolved. It is worth noting here that the close collaboration and communication of the students, experts and the researcher throughout the study were a sine-qua-non for the success of this project. With most of that teamwork being systematized by the project plan and the attached course curricula, it was actually the willingness to volunteer extra time and effort that made the attainment of the study goals possible. Solid coordination of the many activities, enthusiasm of the participants, and unceasing communication were the glue of this EDR process. Moreover, meaningful communication and collaboration was achieved by engaging and empowering students participating in the study. All things considered, it is the educational intervention meeting participant needs that is the primary measure of the progress and success of an EDR study. Accordingly, the EDR methodology worked well for the MELLES study as it combined elements of successful participative research, software development and second language learning processes. The framework shaped and evolved overtime to match our project goals and objectives, yet the following definition of EDR, proposed by Wang and Hannafin, remained the core of our research methodology: A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers  986 and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories (Wang & Hannafin, 1999). At the same time a need for currency of the contextually-sensitive feedback, agility of response and the resulting evolution of the MELLES design were enabled by the “interventionist” nature of the EDR approach. The study was grounded in a naturalistic setting and addressed issues of everyday practice, thus leading to the development of practical solutions and reusable design principles. Considering the complexity of the educational problem and its environment, developing and evaluating the intervention in-situ engendered a more holistic and realistic exploration of the problem. Likewise, the participation of learners and practitioners produced a practical and usable solution reflecting the dynamic character of contextualized language learning. To work through the complex network of notions and their connections did require a few iterations of feedback analysis and reflection, and examining things through a real-world lens. The EDR approach, thus, allowed describing educational practice holistically, notwithstanding its complexity and local idiosyncrasy (Kelly, 2006). We developed the design principles to expand theoretical understands (our and hopefully for others) and to guide the development of similar interventions in other contexts. We believe the design principles act as a whole, but given the iterative nature of EDR studies and necessity to adopt tools and methods to the unique characteristics in other contexts, we do not believe they are prescriptive, universal, or that they need to be enacted as a full meal. Rather they can be selected and, we believe, will be useful in other contexts, like separate items on a menu. Finally, the EDR method facilitated the concurrent and interdependent research and software development processes that fed into each other in a symbiotic relationship. The tangible results of the software development process served as stepping stones for the evolution of learning theory. In turn, the evolving theoretical framework guided all design decisions. Consistent with the ecological lens applied to the study, the EDR method provided a comprehensive and flexible plan for the design process as well as accommodated an in-depth investigation of the individual elements essential for the successful educational intervention. Key source Palalas, A. (2012). Design guidelines for a Mobile-Enabled Language Learning system supporting the development of ESP listening skills. Doctoral dissertation. Athabasca, Canada: Athabasca University. Retrieved from http://hdl.handle.net/10791/17. References Anderson, T. (2007). Design-based research: A new research paradigm for open and distance learning [Slideshare slides]. Retrieved from http://www.slideshare.net/terrya/design-basedresearch-new-research-paradigm. Bannan, B. (2009). The integrative learning design framework: An illustrated example from the domain of instructional technology. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 53-73). Enschede, the Netherlands: SLO. Brown, E., Börner, D., Sharples, M., Glahn, C., De Jong, T, & Specht, M. (2010). Locationbased and contextual mobile learning. A STELLAR Small-Scale Study. STELLAR European Network of Excellence in TEL (EU). CIITE (2004). The Colleges Integrating Immigrants to Employment (CIITE) Project. Implementing systemic change (Phase 3.1 - Final Report).  987 Demouy, V., & Kukulska-Hulme, A. (2010). On the spot: Using mobile devices for listening and speaking practice on a French language programme. Open Learning: The Journal of Open, Distance and e-Learning, 25(3), 217-232. Halliday, M.A.K. (1993). Towards a language-based theory of learning. Linguistics and Education 5(2), 93-116. Herrington, J., & Reeves, T. (2011). Using design principles to improve pedagogical practice and promote student engagement. Paper presented at the ASCILITE 2011, Hobart, AU. Retrieved from http://www.ascilite.org.au/conferences/hobart11/downloads/papers/Herringtonfull.pdf. Hoven, D. (2008). Resolving conflict in distance education situations: Changing roles to break down barriers. Presentation at the ALT-C 2008 Conference: Leeds, UK. 9-11th September. Retrieved from http://www.slideshare.net/debrah/alt-c-pres08hoven. Hoven, D., & Palalas, A. (2011). (Re)-conceptualizing design approaches for mobile language learning. CALICO Journal, 28(3). Kelly, A. (2009). When is design research appropriate? In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 73-89). Enschede, the Netherlands: SLO. Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo-Sánchez, I., & Giasemi, V. (2011). The genesis and development of mobile learning in Europe. In D. Parsons (Ed.), Combining e-learning and m-learning: New applications of blended educational resources (pp. 151-177). Hershey, PA: Information Science Reference. Lafford, B.A. (2009). Toward an Ecological CALL: Update to Garrett (1991). Modern Language Journal, 93(1), 673-696. Lam, W.S.E., & Kramsch, C. (2003). The ecology of an SLA community in a computer-mediated environment. In J. Leather & J. Van Dam (Eds.), Ecology of language acquisition (pp. 141-158). Dordrecht, The Netherlands: Kluwer Academic. Lantolf, J.P. (2000). Introducing sociocultural theory. In J.P. Lantolf (Ed.), Sociocultural theory in second language learning (pp. 1-26). Oxford: University Press. McKenney, S., Nieveen, N., & Van den Akker, J. (2006). Design research from a curriculum perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research: The design, development and evaluation of programs, processes and products (pp. 110-143). New York: Routledge. Pachler, N. (2009). Research methods in mobile and informal learning: Some issues. In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research designs (pp. 1-16). Bern, Switzerland: Peter Lang AG. Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. New York: Springer Science + Business Media, LLC.  988 Palalas, A. (2012). Design guidelines for a mobile-enabled language learning system supporting the development of ESP listening skills. Doctoral dissertation. Athabasca, Canada: Athabasca University. Retrieved from http://hdl.handle.net/10791/17. Palalas, A. (2011). Mobile-assisted language learning: designing for your students. In S. Thouësny, L. Bradley (Eds.), Second language teaching and learning with technology. Research-publishing.net: Voillans. Palalas, A. (2009). Using mobile devices to extend English language learning outside the th classroom. In D. Metcalf, A. Hamilton, & C. Graffeo (Eds.), mlearn2009: 8 World Conference on Mobile and Contextual Learning. Proceedings (pp. 179-183). Orlando, FL: University of Central Florida. Plomp, T. (2009). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 9-36). Enschede, the Netherlands: SLO. Rosell-Aguilar, F. (2007). Top of the pods: In search of a podcasting pedagogy for language learning. Computer Assisted Language Learning, 20(5), 471-492. doi:10.1080/09588220701746047. Underwood, J., Luckin, R., & Winters, N. (2012). Managing resource ecologies for mobile, personal and collaborative self-directed language learning. Procedia - Social and Behavioral Sciences, 34, 226-229. Van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1-14). Boston: Kluwer Academic. Van Lier, L. (2000). From input to affordance: Social-interactive learning from an ecological perspective. In J.P. Lantolf (Ed.), Sociocultural theory and second language learning (pp. 245259). Oxford: Oxford University Press. Vygotsky, L.S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.  989 Agnieszka Palalas is a Mobile Learning Specialist at George Brown College, Toronto, Canada, where she has been in charge of numerous m-learning initiatives and research projects. She has worked with students and faculty to explore and integrate innovative pedagogies and technologies in blended learning contexts. In her over 20 year experience as an educator, Agnieszka has also taught in the classroom and online, reviewed and designed curriculum, provided professional development, worked as a programmer and instructional designer, as well as led educational design research projects. Agnieszka holds a Doctor of Education degree from Athabasca University. In her doctoral work, she concentrated on the effective design of mobile-assisted language learning solutions. Her other research interests focus on emerging approaches to teaching and learning, as well as formal and informal learning practice using innovative technologies. Agnieszka is active in local and international mobile learning and second language learning associations and is a regular presenter at professional conferences. Agnieszka has published several articles pertaining to mobile learning and design-based research E-mail:

Terry Anderson is Professor and researcher in the Technology Enhanced Knowledge Research Centre at Athabasca University - Canada’s Open University. He has published widely in the area of distance education and educational technology and has co-authored or edited eight books and numerous papers. Terry is active in provincial, national, and international distance education associations and is a regular presenter at professional conferences. He teaches educational technology and research courses in Athabasca Universities’ Masters and Doctorate of Distance Education programs. His research interests focus on interaction and social media in educational contexts. Terry is the director of CIDER - the Canadian Institute for Distance Education Research (cider.athabascau.ca) and the editor of the International Review of Research on Distance and Open Learning (IRRODL www.irrodl.org). His homepage is at https://landing.athabascau.ca/pg/profile/terrya and his blog the “Virtual Canuck” is accessible at terrya.edublogs.org