(PDF) Computing Education Research as a Translational Transdiscipline

Computing Education Research as a Translational Transdiscipline Evan Cole Yoshi Malaise Beat Signer

WISE Lab, Vrije Universiteit Brussel WISE Lab, Vrije Universiteit Brussel WISE Lab, Vrije Universiteit Brussel Brussels, Belgium Brussels, Belgium Brussels, Belgium ABSTRACT 1 INTRODUCTION The field of Computing Education Research (CER) produces impor- Computing Education Research (CER) suffers from two divides that tant insights into learning and notable interventions, yet due to limit our ability to make Computing Education (CE) effective and in- the research/practice divide these do not have the desired impact clusive. The first divide is a research/practice divide whereby knowl- on learners or practitioners. Even within CER, Computing Edu- edge and artefacts from CER do not translate into practice [5, 21]. cation (CE) learning theories have limited influence on learning The second one is a theory/design divide within CER where ad- designs due to the theory/design divide, which is unfortunate given vances in theory do not always translate into improved learning that the goal of CER is to impact learners and broaden access to designs [10, 21, 25, 30, 47]. We have also experienced this first-hand computation. as computing educators interested in evidence-based practices, yet There is a lack of an overarching model defining CER as a unified teaching a learner profile that is overlooked by contemporary CER field and providing a framework for discussion. While there is dis- and in a context where available CER artefacts are difficult to adapt. cussion around many of the core activities and practices in CER, we These challenges are described in the literature, with a few pa- have yet to come across a holistic characterisation. We introduce a pers standing out for their actionable suggestions [21, 25, 30, 50]. model of Translational Computing Education Research (TCER) that However, we did not find any article clearly defining both di- helps to understand and discuss CER as a field, bridge the divides vides or suggesting a relationship between them. While the term and provide internal structure, while also making the field more łtranslationž appeared in several papers [4, 21, 48], those focus- approachable for interdisciplinary and non-academic collaborators. ing on the research/practice divide often use the word łpropaga- In our TCER model, theory and design are equally important but tionž [4, 15, 17, 19, 48]. weighted differently depending on an activity’s position along the In parallel, CER is actively defining itself as a field of study. While research/practice continuum. CER has a large body of research, it is still developing many of the In addition to the future exploration and exploitation of the conventions and features already present in more mature fields. We presented TCER model, we propose further characterising CER believe these challenges are easier to address once an overarching as a field, applying the TCER model to understand past and con- theory of CER is established. temporary CER, applying the model to address current challenges None of these challenges are unique to CER. Medicine has a in CER, imagining what the field can become, as well as exploring history of leveraging theory into better patient outcomes using a the potential for translational research programmes to maximise model called Translational Research (TR). Other fields have faced the broader impact of computing education research. the challenges of self-definition and recognition. To help address both challenges, we propose a unifying theory of CER: Translational CCS CONCEPTS Computing Education Research (TCER). After reviewing TR models • Social and professional topics → Computing education. used in medicine, we developed a model of TCER adapted to the realities and needs of CER and designed to be granular and action- KEYWORDS able. We later learned that other fields such as education [4, 28, 38], teacher training [6], reading education [45], STEM education [39], Translational Computing Education Research, Transdisciplinary łmind, brain and educationž [46], computing [1] and HCI [9] have Research, Translational Research Programmes, TCER Model also explored models of translational research. However, those ACM Reference Format: models were either less developed or not well-suited for CER. Evan Cole, Yoshi Malaise, and Beat Signer. 2023. Computing Education Translational Research is promising for bridging the two di- Research as a Translational Transdiscipline. In Proceedings of the 54th ACM vides but there are challenges in its implementation. We looked Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023), March 15ś18, 2023, Toronto, ON, Canada. ACM, New York, NY, USA, 7 pages. into criticisms of TR in medicine to better understand the risks in- https://doi.org/10.1145/3545945.3569771 volved [2, 3, 24, 34, 56, 58]. A criticism that stood out is that TR can impose a translational imperative [24, 58] pressuring researchers to Permission to make digital or hard copies of all or part of this work for personal or justify all their work with broader impacts (BI) [58]. This is harmful classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation to the long-term health of a field by discouraging exploratory fun- on the first page. Copyrights for components of this work owned by others than the damental research which only pays off in the long term. Another author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or criticism is that the term łtranslationalž can appear linear, giving republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from

. the wrong impression that there is a łpipelinež [2] moving from SIGCSE ’23, March 15ś18, 2023, Toronto, ON, Canada. research (theory) to practice (design). This creates counterproduc- © 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM. tive expectations since translation is a cyclical, unpredictable and ACM ISBN 978-1-4503-9431-4/23/03. . . $15.00 https://doi.org/10.1145/3545945.3569771 SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. Evan Cole, Yoshi Malaise, and Beat Signer multi-dimensional process [2]. There is also concern that a transla- However, we know from the literature [12] and experience that tional mentality sets false expectations for research programmes CER is multi-disciplinary at its core. Computing is infinitely varied that do not align with the history of medicine. Over the last 30 years, and cross-cutting, education is highly contextual and educational translational research in medicine has been wildly inefficient; mean- research requires diverse collaborations. while many of the longest-serving interventions in medicine were It feels too confining to simply call CER ła disciplinež, in par- developed without any underlying theoretical understanding [2]. ticular considering that CER involves disparate stakeholders and We are convinced that CER can successfully adopt a TR model addresses social and systemic challenges facing CE. CER has also in spite of these challenges. Not only can we learn from the past, been characterised as Discipline-based Education Research [37], but CER has a strong tradition of translation, is actively working but this definition [44] seems too narrowly focused on the research to improve translation, and there are features of CER and broader aspects of CER. computing communities that we believe set the stage for success. We then came across the term łtransdisciplinaryž [7, 36] which we believe best describes CER: 2 CHARACTERISING CER łTransdisciplinary involves scientists from different disciplines CER is still engaging in many activities typical of an emerging as well as non-scientists and other stakeholders and, through field, including curating a core literature [33], publishing first text role release and role expansion, transcends (hence łtransž) books [14, 41], debating methodology [21, 30, 50], defining domain- the disciplinary boundaries to look at the dynamics of whole specific methodologies [21, 25, 41], building domain-specific the- systems in a holistic way.ž ories [25, 47], discussing publication standards1 [14, 29, 50], char- This term encompasses both the theoretical and design activities acterising its research landscape [10, 12, 32, 47], creating its own within CER, while also including non-academic stakeholders and educational programmes [23], creating a global community [20] and recognising that true progress in education comes through broad defining career paths for CER graduate students2 [27, 55]. These engagement and systemic solutions. Thus, we characterise CER as advances are important, but still leave us wondering łwhat is CERž? łComputing Education Research is a Translational Transdisciplinež. In this section we characterise CER by first explaining why we believe it is a translational transdiscipline, then describing three key 2.2 CER Stakeholders stakeholders in CER and finally listing several defining features of CER that can encourage successful translational research. A field as broad as CER has a wide range of stakeholders with varying roles. Inspired by the immediacy axis of the Inclusion- 2.1 The Nature of CER Immediacy Criterion (IIC) described by Woodson [58], we first define three broad categories of CER stakeholders: While researching and designing our model of CER, we also thought about CER as a field. We concluded that CER’s main objective is to • Intrinsic Stakeholders: Anyone directly contributing to bodies improve the efficacy and inclusiveness of CE. That much is clear of knowledge in CER. and less controversial, but then things get fuzzier. • Direct Stakeholders: Anyone directly benefiting from the out- puts of CER. 2.1.1 CER is Translational. Discussions in existing literature have • Extrinsic Stakeholders: Anyone indirectly benefiting from circled around whether CER is a research field generating theo- progress in CER. ries of CE, or a practice field creating designs to improve learn- In the upcoming sections, we discuss three specific intrinsic or ing [12, 30, 50]. Choosing one or the other does not feel right be- direct stakeholders that are central actors in our TCER model, while cause both are valuable for impacting CE outcomes. They are even extrinsic stakeholders are discussed later in Section 4. Note that a stronger together than alone when practice informs theory and single person can fall under more than one category: research informs design. We then learned about the research/practice continuum [54] from 2.2.1 Researchers (Intrinsic Stakeholder). We define researchers as medical TR and it became clear that translation is more than a useful anyone who contributes to bodies of knowledge in CER following lens to understand CER; it is also a methodologically inclusive an appropriate methodology. łAppropriatež can mean different way to characterise CER as a field. Questions about theory versus things at different stages along the research/practice continuum. A design can be resolved by imagining a theory/design continuum. few examples include methodical theory development, controlled Some activities in CER produce theories, others generate promising experiments, design methodologies, qualitative research and data designs based on theory, and yet others focus on implementing and analysis. scaling successful designs. 2.2.2 Educators (Direct Stakeholder). We identify several types of 2.1.2 CER is a Transdiscipline. Next, we asked ourselves whether educators with different relationships to CER. The more łstereotyp- CER can be considered as a stand-alone discipline. At a basic level icalž practitioner is a teacher simply trying to teach their learners this position is simple enough as it has already been defended [43] with the best methods available within their means. Their relation- and it could be argued that contemporary CER meets more recent ship to CER is passive, they consume the products of CER but may criteria as well [53]. not be aware that it exists as a field of study. There is also the enthusiastic practitioner; an educator who has 1 https://faculty.washington.edu/ajko/cer#experience-reports the opportunity and interest to engage in design and experimen- 2 https://www.csedgrad.org tation, alone or in collaboration with researchers. If their work Computing Education Research as a Translational Transdiscipline SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. follows an appropriate methodology and is added to the body of We do not propose the TCER model as dogma. All research and CER knowledge, they can also be considered a researcher. intervention efforts should operate within constraints in a way that best benefits their stakeholders and objectives. We also recognise 2.2.3 Learners (Direct Stakeholder). A learner is anyone interested that CER is diverse and some activities may not fit cleanly in our in learning computing. This includes novices, experienced develop- model. We developed our model simply to start the conversation ers learning a new paradigm, experts in a different domain learning about a shared framework for discussing and understanding CER. a domain-specific language, conversational programmers [8], digi- tal artists or someone learning for fun. 3.1 Judging Our Model Self-guided learners are primarily learners; however they are effectively acting as their own teacher and could benefit from many A good model should not only help to understand the world as it of the guides and resources available to educators. is, but also help imagine a different future. We hope to show that discussing CER in terms of the TCER model presented in Figure 1 2.3 Defining Features of CER helps to structure a productive conversation about CER as a field. We judge our model’s success by whether it helps the community We now list some defining features of CE that from our experience to understand, share and discuss CER. We even believe that the lay the foundation for a successful translational practice. Taken TCER model should be measured by similar criteria to a łthresh- together, these considerations outline a field with the potential for old conceptž as described by Flanagan4 , on which we base the successful translational research: following criteria: (1) There are no legal or regulatory barriers to most research, • Transformative: Once understood, does TCER change the only ethical ones. way in which you view CER? (2) Health and lives are not at risk. A small unsuccessful exper- • Irreversible: Once you see CER as translational, is it hard for iment is not a risk as long as a better alternative is offered you to go back? after the experiment concludes. • Integrative: Once learned, does TCER bring together different (3) There is a wide variety of stakeholders who can all benefit aspects of CER that previously did not appear to be related? from small improvements, and many of these improvements • Bounded: Does TCER delineate CER, making it easier for you may be low hanging fruits. to understand the field as a unified entity? (4) Curricula can be updated incrementally. Isolated aspects of • Discursive: Does understanding TCER give you a richer vo- a learner’s experience can be improved before an entire new cabulary for discussing CE? solution is ready. • Reconstitutive: Does understanding TCER subjectively change (5) CER is a relatively small field, so one researcher or a small your interactions with CE? group can have both a broad and deep understanding of CER. • Liminal: Can you imagine learning about TCER being a łright (6) It is common for researchers to also be educators, giving of passagež for newcomers to the field? them first-hand insights to CE. (7) Educators often possess the technical skills necessary to Finally, does TCER help us to see how our work and that of our implement their own prototypes. peers are related? Does it help us to better understand CER from (8) Prototyping software and learning materials is relatively the perspective of our peers? Does Silver’s [42] characterisation of cheap and it is possible to iterate quickly. theory describe how you feel about our theory of TCER? (9) Tight feedback loops between theory and design or research and practice are possible because research often takes place łTo understand theory is to travel into someone else’s mind with real learners in an authentic setting. and become able to perceive reality as that person does. To (10) Many computing communities have a culture of teaching, understand a theory is to experience a shift in one’s own mental structure and discover with startling clarity a different way learning and making. of thinking. To understand theory is to feel some wonder that (11) There is strong public interest in improving and expanding one never saw before what now seems to have been obvious all Computing Education. along.ž 3 TCER MODEL 3.2 Description of the TCER Model The challenge of coordinating theory and practice are not unique Layers are the highest level of organisation in our model. Each layer to CER. To inspire our model, we reviewed models of translational spans horizontally through all phases of our model, and each layer research in medicine. The primary inspiration for our TCER model below provides a progressively finer-grained description of TCER. illustrated in Figure 1 was a review by Trochim et al. [54] and the In the following, we go through the different layers of our model more general concept of Translational Science.3 shown in Figure 1 and explain the reasoning behind each layer. We believe our model is relevant no matter what one thinks While the TCER model might seem to be linear with siloed łcomputingž means [49]. Our model is focused on the łEž and łRž progression from theory to intervention, we foresee strong feedback in łCERž; we are discussing how to generate and exploit the peda- loops between all research activities. This dynamic has also been gogical and technical insights to teach whatever computing content called the łtranslational science spectrumž.5 learners need to know. 4 https://www.ee.ucl.ac.uk/~mflanaga/thresholds.html 3 https://ncats.nih.gov/translation 5 https://ncats.nih.gov/translation/spectrum SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. Evan Cole, Yoshi Malaise, and Beat Signer Translational Computing Education Research Engagement and Outreach Identify challenges in CE and opportunities for CER Fundamental Research Synthesis Praxis Reflexive Analysis & Action Realms: developing & validating theories disseminating applying theory in practice Bodies of Theoretical CER Experimental CER State of the Art Applied CER Educational Impact Knowledge: Translational 1. Theory 2. Controlled 3. Research 4. Practice 5. Scaled Phases: Development Experiments Synthesis Research Interventions A* B A* B A B* A* B* A* B* Activities: General or Domain- Theory- Controlled, Researcher- Practitioner- Evidence- User Scalable Educational Borrowed specific based Empirical facing Litera- facing based Feedback Contextualised Impact Theories Theories Designs Experiments ture Reviews Guidelines Prototypes & Reports Interventions Analysis * TradingZones: "An area in which radically different activities could be locally, but not globally, coordinated." - Galison et al. 1996 [16] Research Practice Continuum: RT RT, RD PT, PD PD Theory Design Figure 1: TCER model overview 3.2.1 Theory/Design and Research/Practice Continuum. The foun- • Practice Theories (PT): These are useful theories for practi- dation of our model is the theory/design and research/practice con- tioners that may not be methodologically sound. Some PTs tinuum shown in Figure 2, a two-dimensional continuum classifying could be called a łheuristicž, though PTs may also be more contributions to CER based on their evaluation and contribution. general and less practical than a heuristic. • Research/Practice: This axis describes how an artefact is eval- • Practice Designs (PD): A design whose main purpose is to im- uated. Research artefacts are evaluated by their methodology, pact CE practice, it can be developed with any methodology while practice artefacts are evaluated by their effectiveness. and may or may not be evidence-based. • Theory/Design: This axis describes an artefact’s contribution Our continuum is different from Pasteur’s Quadrant [26] in that to CER. For example, a lesson plan designed for a controlled we use differently labelled axes, we do not simply use binary values experiment might contribute to both theory and design, on both axes, and we draw value from all four quadrants. while a new lesson plan for schools may only contribute 3.2.2 Engagement and Outreach. Computing education is every- to CE design. where. It is online tutorials, it is meetup groups, it is children, adults Research Practice and everything in between, it is professional and recreational, it is for novices and experts, it is endless. Fortunately, many comput- Theory RT PT ing communities have a culture of learning, teaching and making. Unfortunately, many computing communities do not know CER exists. When it comes to prevalent educational approaches there is Design RD PD still a lot of inertia for non-evidence-based practices, and plenty of łit worked for mež. The CER community has a tradition of outreach and direct en- Figure 2: Research/practice and theory/design continuum gagement with learning communities, and this must be an ongoing effort. If CER is to stay relevant it will need to make a concerted ef- Overall, no contribution is more important than another and it fort to identify new challenges in CE, reach CE communities where is a question of context, with different contributions being more they are, and work productively with a variety of stakeholders. valuable to different activities. Below are the four broad categories of contribution, but keep in mind that in reality this is a sliding 3.2.3 Realms. The next layer of our model breaks CER into three scale rather than absolute categories: realms without any order or hierarchy: • Research Theories (RT): Methodologically and empirically • Fundamental Research: Developing and validating theo- validated theories of CE. ries for CE. • Research Designs (RD): Designs created with a strict method- • Synthesis: Consolidating and communicating knowledge ology, often intended as a contribution to the CER literature from Fundamental Research and Praxis. or used to validate RTs. • Praxis: Designing evidence-based solutions for CE. Computing Education Research as a Translational Transdiscipline SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. 3.2.4 Bodies of Knowledge. Bodies of knowledge are distinct but 2.A: Theory-based Designs: Designing small-scale focused in- interrelated groupings of knowledge in CE. Imagine one researcher terventions to test or develop a specific theory. choosing to specialise in experimental CER across many contexts 2.B: Controlled Empirical Experiments: Use controlled ex- and another researcher specialising in one CE context and being periments to investigate theories from 1.A/B and designs familiar with relevant knowledge from all bodies. from 2.A. Separating 2.A and 2.B can help in establishing • Theoretical CER: Theoretical understanding of how people replication standards. teach and learn computing. 3.A: Researcher-facing Literature Reviews: Perform system- • Experimental CER: Bodies of empirical evidence used to atic literature reviews of activities in all phases, targeted at develop CE theories and inform practise. an academic audience. • State of the Art: Syntheses of all bodies of knowledge, 3.B: Practitioner-facing Guidelines: Produce evidence-based communicated differently depending on the target audience. practical guidelines or suggestions for CE practitioners. • Applied CER: Prototypes and heuristics inspired by the 4.A: Evidence-based Prototypes: Design evidence-based proto- state of the art to support specific educational outcomes in types for targeted educational outcomes and contexts; share specific contexts. ongoing designs and process. • Educational Impact: Practical insights gained from scaling 4.B: User Feedback & Reports: Understand what works and educational interventions and analysing their impact. what does not work in different settings and share these insights to progress theory and design. 3.2.5 Translational Phases. Translational phases are about what 5.A: Scalable Contextualised Interventions: Develop and de- you do, not just what you know. Compared to bodies of knowledge, ploy scalable interventions based on promising prototypes. translational phases are active and more directional, giving struc- 5.B: Educational Impact Analysis: Conduct follow-up studies ture to the theory/design and research/practice continuum. Each to analyse an intervention’s impact and publish reports to translational phase is intimately tied to its corresponding body of share your findings. knowledge, each growing from and building on the other. 3.2.7 Trading Zones. We have also considered which activities are 1: Theory Development: Generating and developing theories more łinternalž to CER and which activities invite transdisciplinary of how people learn and teach programming. collaboration. The latter we call trading zones (marked with an 2: Controlled Experiments: Testing and exploring theories asterisk in Figure 1). Galison defined trading zones as łan arena in with controlled experiments. which radically different activities could be locally, but not globally, 3: Research Synthesis: Synthesising results from all phases coordinatedž [16], and Draper and Maguire [12] discussed the im- and communicating the relevant results to each stakeholder. portance of trading zones in CER. We believe that being explicit 4: Practice Research: Designing and validating prototypes about which activities are trading zones can help to foster transdis- that apply CE theory in diverse educational contexts. ciplinary collaborations while still maintaining a distinct identity 5: Scaled Interventions: Exploiting successful designs to have for CER. large-scale impact in computing education. Translational research can only succeed if researchers and prac- 3.3 Reflexive Analysis and Action titioners engage in active collaborations and when all stakeholders This final feature of our model is the most important one because have reliable channels of communication and shared knowledge. even if our model were perfect today, it might become obsolete in For example, impact reports from phase 5 can help to guide a phase 1 the future. A field without the culture and mechanisms to support researcher’s work, and recent experiments from phase 2 can directly self-reflection and re-definition cannot stay relevant. inspire a phase 4 practitioner. To aid in communication, our model CER currently has this self-reflective culture as evidenced by has defined a phase of research (phase 3) dedicated to synthesis- recent publications about CER [12, 21, 30, 49, 50], a recent special ing and disseminating the state of the art to both academic and issue on theory in CER [52], conference proceedings6 and discus- non-academic audiences. sions taking place in social media and blogs.7 If CER is to flourish, this culture needs to be hard-coded into how we define CER, how 3.2.6 Activities. Within each of the five phases, we have defined we collaborate with other fields, interact with non-academic stake- two primary activities. Researchers in each activity ask different holders and train new members of our field. Each new generation questions and answer them with different methods. These ten ac- of CE researchers should feel that they can still help to (re)define tivities are not isolated or always in order; a single publication the field. may contain a focused literature review (3.A), present a new theory (1.B), design a theory-based intervention (2.A), conduct a controlled 4 TRANSLATIONAL RESEARCH experiment (2.B) and conclude with advice for educators (3.B). See PROGRAMMES Nelson and Ko [30] for a discussion of balancing design, explanation and experiment in one study. One important implication of our TCER model is Translational Research Programmes (TRPs). By coordinating diverse research 1.A: General or Borrowed Theories: Explore and borrow learn- projects dealing with different aspects of the same problem, TRPs ing theories from other domains. 1.B: Domain-specific Theories: Develop domain-specific learn- 6 https://www.ukicer.com/#keynotes ing theories for computing education. 7 https://twitter.com/NALooker/status/1549780567683653633 SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. Evan Cole, Yoshi Malaise, and Beat Signer can address challenges in CE that are too complex or diffuse for The overall success of a TRP should not only take the BIs into a single researcher or institution. The key features of TRPs could account, but the entire process from initial scoping to final delivery. include (in no particular order): We are not aware of any work providing such a model adapted to CER. However, we think a combination of the Inclusion-Immediacy • Broader Impacts (BIs): A specific and meaningful impact in Criterion, the Translational Science Benefits Model [22], healthy CE that the research addresses. collaboration with practitioner partners [11, 31], and assessments • A Translational Team: A team of researchers who together or feedback from stakeholders could serve as a starting point. possess the broad range of skills and experience necessary for a TRP to succeed [35]. 5 FUTURE WORK AND CONCLUSION • Non-Academic R&D Partners: Ideally a TRP involves the enti- ties it would like to impact. If not possible, stand-ins with Our characterisation of CER, the TCER model and TRPs are just a similar constraints are acceptable. starting point. They open the door to a variety of future research • Resources: A TRP will likely be larger and longer than a both theoretical and applied, including but not limited to: standard research project. It may be necessary to secure • Debating and refining our characterisation of CER. substantial resources. • Testing and developing the TCER model by using it to de- • Feasibility: A TRP should have a reasonable chance of success scribe CER activities and literature. given its constraints and the challenge it is addressing. • Build on existing CER literature classifications [12, 43] using the lens of TCER. TRPs are iterative and unpredictable projects that require a di- • Using the TCER model to approach existing challenges within verse and flexible łtranslational science teamž [18, 45]. The sep- CER and imagine possible futures for CER. aration of activities in our TCER model is intended only to help • Structuring research/practitioner partnerships with TCER. discuss and plan, not to silo. For example, imagine a TRP for which • Develop translational research methodologies suited to CER. there is enough theory available, but the theories have not been • Exploring how TRPs could best operate in CER. validated for your contextÐyou will need to plan some phase 2 • Identifying suitable challenges for TRPs in CE. activities. There can be as much łreturning to the drawing boardž • Exploring the use of TCER and TRPs to secure more substan- as necessary and as many feedback loops as possible. tial funding for CER.8 TRPs may also stand a higher chance of securing funding [57]. The TCER model can help communicate your research’s BIs, your We discussed two important challenges in CER that are limiting methods and activities, which profiles you will need, and your our ability to impact CE: the research/practice divide and the the- relationship to non-academic partners. ory/design divide. Before imagining solutions to these challenges, More insight into what it could mean for CER to adopt trans- we tried to understand CER holistically and concluded that CER is lational science can be found in Solari et al. [45] where the same a transdiscipline, and that CER’s nature is translational. The overar- question is discussed in the context of reading education. ching premise of CER is to work with all stakeholders to translate theoretical and empirical understanding of CE into broader impacts 4.1 Inclusive and Immediate TRPs for learners, educators and society. Our model of Translational Computing Education Research cap- For a TRP to succeed, it will not only need a rigorous approach to re- tures CER as a field. Based on the discussed research/practice search and design (supporting efficacy), but also a rigorous approach and theory/design continuum, the presented TCER model recog- to diversity and inclusion. There are already several approaches nises the systemic complexities of education, accommodates trans- available to CER including culturally responsive computing edu- disciplinary collaboration, encourages engagement with broader cation (CRCE) [13, 40], universal or accessible design, engaging computing communities, and foresees methodological introspec- openly and frequently with stakeholders, and defining success mea- tion. We finally explored the use of Translational Research Pro- sures in cooperation with direct and indirect stakeholders. grammes to address large-scale challenges in CER and discussed However, what CER is missing is a way to characterise how a different ways to plan for and measure the effects of TRPs on effi- TRP will impact diversity and inclusion in computing. We propose cacy and inclusiveness in computing education. adopting and adapting the Inclusion-Immediacy Criterion (IIC) [58] Over the last 70 years CER has passed through several peri- to qualify BIs in CER. The model has nine categories of BIs defined ods [51] and the next period might be translationalÐa period where by a 3x3 grid. The two axes of the grid are inclusiveness, defining we discover new ways to structure our understanding of Computing who will benefit from the impacts, and immediacy, expressing how Education Research and translate it into a more equitable future. direct the impacts are. REFERENCES 4.2 Quality Assurance [1] David Abramson and Manish Parashar. 2019. Translational Research in Computer To ensure the quality of a TRP’s ongoing activities and subse- Science. Computer 52, 9 (2019). https://doi.org/10.1109/MCSE.2021.3109962 [2] Christopher P. Austin. 2021. Opportunities and Challenges in Translational quent BIs, there must be conventions for evaluating the research Science. CTS 14, 5 (2021). https://doi.org/10.1111/cts.13055 process and outcomes. To track the progress of a TRP, Trochim et al.’s [3] Andre Barkhordarian et al. 2013. Assessment of Risk of Bias in Translational process marker model (PMM) [54] can be a starting point. The PMM Science. Journal of Translational Medicine 11, 1 (2013). functions in uncertainty by identifying measurable markers that 8 https://medium.com/bits-and-behavior/dagstuhl-trip-report-educational- can help understand the health of an ongoing TRP. programming-languages-ad9117ee615b#e2a9 Computing Education Research as a Translational Transdiscipline SIGCSE ’23, March 15–18, 2023, Toronto, ON, Canada. [4] Bryan A Brown, Phillip Boda, Catherine Lemmi, and Xavier Monroe. 2019. Mov- [31] Elena Novak and Javed I Khan. 2022. A Research-Practice Partnership Approach ing Culturally Relevant Pedagogy From Theory to Practice: Exploring Teachers’ for Co-Designing a Culturally Responsive Computer Science Curriculum for Application of Culturally Relevant Education in Science and Mathematics. Urban Upper Elementary Students. TechTrends (2022). https://doi.org/10.1007/s11528- Education 54, 6 (2019). https://doi.org/10.1177/0042085918794802 022-00730-z [5] Neil Brown, Quintin Cutts, Maria Kallia, Joseph Maguire, Fiona McNeill, and Leo [32] Zacharoula Papamitsiou, Michail Giannakos, Simon, and Andrew Luxton-Reilly. Porter. 2021. Supporting Computing Educators to Create a Cycle of Teaching 2020. Computing Education Research Landscape Through an Analysis of Key- and Computing Education Research. In Proceedings of UKICER 2021. https: words. In Proceedings of ICER 2020. https://doi.org/10.1145/3372782.3406276 //doi.org/10.1145/3481282.3483527 [33] Arnold Pears, Stephen Seidman, Crystal Eney, Päivi Kinnunen, and Lauri Malmi. [6] Kevin Burden, Sarah Younie, and Marilyn Leask. 2013. Translational Research 2005. Constructing a Core Literature for Computing Education Research. ACM Principles Applied to Education: The Mapping Educational Specialist Knowhow SIGCSE Bulletin 37, 4 (2005). https://doi.org/10.1145/1113847.1113893 (MESH) Initiative. Journal of Education for Teaching 39, 4 (2013). https://doi.org/ [34] C. Petrini. 2011. From Bench to Bedside and to Health Policies: Ethics in Transla- 10.1080/02607476.2013.801216 tional Research. Clinica Terapeutica 162, 1 (2011), 51. [7] Bernard C. K. Choi and Anita W. P. Pak. 2006. Multidisciplinarity, Interdisci- [35] Yaacov Petscher, Nicole P. Terry, Nadine Gaab, and Sara A. Hart. 2020. Widening plinarity and Transdisciplinarity in Health Research, Services, Education and the Lens of Translational Science Through Team Science. https://doi.org/10. Policy: 1. Definitions, Objectives, and Evidence of Effectiveness. Clinical and 31234/osf.io/a8xs6 Investigative Medicine 29, 6 (2006). [36] Cyrille Rigolot. 2020. Transdisciplinarity as a Discipline and a Way of Being: [8] Parmit K. Chilana, Rishabh Singh, and Philip J. Guo. 2016. Understanding Conver- Complementarities and Creative Tensions. Humanities and Social Sciences Com- sational Programmers: A Perspective From the Software Industry. In Proceedings munications 7, 1 (2020). https://doi.org/10.1057/s41599-020-00598-5 of CHI 2016. https://doi.org/10.1145/2858036.2858323 [37] Anthony V. Robins, Lauren E. Margulieux, and Briana B. Morrison. 2019. Cogni- [9] Lucas Colusso, Ridley Jones, Sean A. Munson, and Gary Hsieh. 2019. A Trans- tive Sciences for Computing Education. The Cambridge Handbook of Computing lational Science Model for HCI. In Proceedings of CHI 2019. https://doi.org/10. Education Research (2019). https://doi.org/10.1017/9781108654555.010 1145/3290605.3300231 [38] Henry L. Roediger III. 2013. Applying Cognitive Psychology to Education: Trans- [10] Tom Crick. 2017. Computing Education: An Overview of Research in the Field. Tech- lational Educational Science. Psychological Science in the Public Interest 14, 1 nical Report. Swansea University. https://cronfa.swan.ac.uk/Record/cronfa43589 (2013). https://doi.org/10.1177/1529100612454415 [11] Jill Denner, Steve Bean, Shannon Campe, Jacob Martinez, and David Torres. 2019. [39] Christine Rosicka. 2016. Translating STEM Education Research Into Practice. Negotiating Trust, Power, and Culture in a Research-Practice Partnership. AERA Australian Council for Educational Research. Open 5, 2 (2019). https://doi.org/10.1177/2332858419858635 [40] Kimberly A. Scott, Kimberly M. Sheridan, and Kevin Clark. 2015. Culturally [12] Steve Draper and Joseph Maguire. 2022. The Different Types of Contributions Responsive Computing: A Theory Revisited. Learning, Media and Technology 40, to Knowledge (in CER): All Needed, but Not All Recognised. ACM TOCE (2022). 4 (2015). https://doi.org/10.1080/17439884.2014.924966 https://doi.org/10.1145/3487053 [41] Sue Sentance, Erik Barendsen, and Carsten Schulte. 2018. Computer Science Edu- [13] Ron Eglash, Juan E. Gilbert, and Ellen Foster. 2013. Toward Culturally Responsive cation: Perspectives on Teaching and Learning in School. Bloomsbury Publishing. Computing Education. Commun. ACM 56, 7 (2013). https://doi.org/10.1145/ [42] Paula F. Silver. 1983. Educational Administration: Theoretical Perspectives on 2483852.2483864 Practice and Research. Harpercollins College Division. [14] Sally Fincher and Marian Petre. 2004. Computer Science Education Research. CRC [43] Simon. 2015. Emergence of Computing Education as a Research Discipline. Ph.D. Press. Dissertation. Aalto University. [15] Jeffrey E. Froyd. 2018. Special Session: Developing an Effective Propagation Plan [44] Susan R. Singer, Natalie R. Nielsen, and Heidi A. Schweingruber. 2012. Discipline- for Educational Innovations. In Proceedings of FIE 2018. https://doi.org/10.1109/ based Education Research. The National Academies Press. https://doi.org/10. FIE.2018.8659319 1002/sce.21091 [16] Peter Louis Galison and David J. Stump. 1996. The Disunity of Science: Boundaries, [45] Emily J. Solari et al. 2020. Translational Science: A Road Map for the Science of Contexts, and Power. Stanford University Press. Reading. Reading Research Quarterly 55 (2020). https://doi.org/10.1002/rrq.357 [17] Edward F. Gehringer. 2009. Propagating Software-based Educational Innovations. [46] K. Brooke Stafford-Brizard, Pamela Cantor, and L. Todd Rose. 2017. Building the In Proceedings of ASEE Southeast. Bridge Between Science and Practice: Essential Characteristics of a Translational [18] C. Taylor Gilliland et al. 2019. The Fundamental Characteristics of a Translational Framework. Mind, Brain, and Education 11, 4 (2017). https://doi.org/10.1111/ Scientist. ACS Pharmacology & Translational Science 2, 3 (2019). https://doi.org/ mbe.12153 10.1021/acsptsci.9b00022 [47] Claudia Szabo and Judy Sheard. 2022. Learning Theories Use and Relationships [19] Raina Khatri. 2018. A Model for Propagating Educational Innovations in Higher in Computing Education Research. ACM TOCE (2022). https://doi.org/10.1145/ STEM Education: A Grounded Theory Study of Successfully Propagated Innovations. 3487056 Ph.D. Dissertation. Western Michigan University. [48] Cynthia Taylor et al. 2018. Propagating the Adoption of CS Educational Innova- [20] Sophia Krause-Levy and Melissa Perez. 2022. Building a Multinational Com- tions. In Proceedings of ITiCSE 2018. https://doi.org/10.1145/3293881.3295785 munity Among Graduate Students in CS Education Research. In Proceedings of [49] Matti Tedre. 2018. The Nature of Computing as a Discipline. Computer Science SIGCSE 2022. https://doi.org/10.1145/3478432.3499195 Education: Perspectives on Teaching and Learning in School 2 (2018). https: [21] Anthony A Lowe. 2020. The Theory of Applied Mind of Programming. Ph.D. //doi.org/10.5040/9781350057142.ch-002 Dissertation. Purdue University. https://doi.org/10.25394/PGS.12743717.v1 [50] Matti Tedre and John Pajunen. 2022. Grand Theories or Design Guidelines? [22] Douglas A. Luke et al. 2018. The Translational Science Benefits Model: A New Perspectives on the Role of Theory in Computing Education Research. ACM Framework for Assessing the Health and Societal Benefits of Clinical and Trans- TOCE (2022). https://doi.org/10.1145/3487049 lational Sciences. CTS 11, 1 (2018). https://doi.org/10.1111/cts.12495 [51] Matti Tedre, Simon, and Lauri Malmi. 2018. Changing Aims of Computing [23] Stephanie Lunn, Maíra Marques Samary, and Alan Peterfreund. 2021. Where Education: A Historical Survey. Computer Science Education 28 (2018). is Computer Science Education Research Happening?. In Proceedings of SIGCSE [52] Josh Tenenberg and Lauri Malmi. 2022. Conceptualizing and Using Theory in 2021. https://doi.org/10.1145/3408877.3432375 Computing Education Research. ACM TOCE (2022). https://doi.org/10.1145/ [24] Jane Maienschein, Mary Sunderland, Rachel A. Ankeny, and Jason Scott Robert. 3542952 2008. The Ethos and Ethics of Translational Research. The American Journal of [53] Malcolm Tight. 2020. Higher Education: Discipline or Field of Study? Tertiary Bioethics 8, 3 (2008). https://doi.org/10.1080/15265160802109314 Education and Management 26, 4 (2020). https://doi.org/10.1007/s11233-020- [25] Lauri Malmi, Judy Sheard, Päivi Kinnunen, and Jane Sinclair. 2022. Develop- 09060-2 ment and Use of Domain-specific Learning Theories, Models and Instruments in [54] William Trochim, Cathleen Kane, Mark J. Graham, and Harold A. Pincus. 2011. Computing Education. ACM TOCE (2022). https://doi.org/10.1145/3530221 Evaluating Translational Research: A Process Marker Model. CTS 4, 3 (2011). [26] Robert M. Mason. 2001. Not Either/Or: Research in Pasteur’s Quadrant. Com- https://doi.org/10.1111/j.1752-8062.2011.00291.x munications of the Association for Information Systems 6, 1 (2001). https: [55] Ethel Tshukudu et al. 2022. Bridging the Gap Between Academia and Industry //doi.org/10.17705/1CAIS.00616 in CSEd to Promote Opportunities for CSEd Grads and Research in Practice. In [27] Alejandra Mendez, Ethel Tshukudu, Carolina Moreira, Weena Naowaprateep, Proceedings of SIGCSE 2022. https://doi.org/10.1145/3478432.3499187 Alan Peterfreund, and Brianna Johnston. 2021. Identifying Opportunities and [56] Ellis F. Unger. 2007. All is Not Well in the World of Translational Research. JACC Potential Roadblocks for CSEd Professionals. In Proceedings of ICER 2021. https: 50, 8 (2007). https://doi.org/10.1016/j.jacc.2007.04.067 //doi.org/10.1145/3446871.3469789 [57] Cameron Wilson and Mark Guzdial. 2010. How to Make Progress in Computing [28] Pru Mitchell. 2016. From Concept to Classroom: What is Translational Research? Education. Commun. ACM 53, 5 (2010). https://doi.org/10.1145/1735223.1735235 Australian Council for Educational Research. [58] Thomas Woodson and Sophia Boutilier. 2022. Impacts for Whom? Assessing [29] Gregory L. Nelson. 2021. Teaching and Assessing Programming Language Tracing. Inequalities in NSF-funded Broader Impacts Using the Inclusion-Immediacy Ph.D. Dissertation. University of Washington. Criterion. Science and Public Policy 49, 2 (2022). https://doi.org/10.1093/scipol/ [30] Greg L. Nelson and Amy J. Ko. 2018. On Use of Theory in Computing Education scab072 Research. In Proceedings of ICER 2018. https://doi.org/10.1145/3230977.3230992