(PDF) Building Capability and Communities In Engineering Education Research

Building Capability and Communities in Engineering Education Research A report from the workshop facilitated at the XXXVI Conferencia Nacional de Ingeniería Asociación Nacional de Facultades y Escuelas de Ingeniería, ANFEI Mérida, México • June 19th, 2009 Prepared by Rocío C. Chavela Guerra Monica F. Cox Executive Summary This report summarizes the results from a four-hour workshop aimed to (1) Introduce engineering faculty interested in developing or extending their capability to engage in educational scholarship how to approach and to move their work in the direction of educational research. (2) Share with interested participants strategies to build and to sustain a community of engineering education researchers and to help them network with other global communities. Facilitators presented information to 42 participants representing Mexican and Columbian universities. Participants completed a workshop evaluation form that assessed overall aspects of the workshop (i.e., fulfillment of workshop expectations, relevance of the information discussed, usefulness of the materials and sources presented, and opportunities to interact with others), and the four workshop objectives. Quantitative results confirmed that the majority of participants enjoyed the workshop and wanted to engage in future conversations about engineering education research. Qualitative findings confirmed (1) a high level of interest by faculty in conducting and in promoting engineering education research, (2) a need for more opportunities and support to engage in engineering education research, (3) concerns about faculty rewards for conducting engineering education research, and (4) language barriers between Spanish-speaking workshop participants and English materials and facilitation. Based upon the responses from the participants, facilitators present the following suggestions for advancing engineering education research efforts in Mexico and, more generally, Latin America: The organization of longer workshops, seminars, or certificates informed from educational research capacity-building models in the U.S.; the development of engineering education research certificate programs for Latin American E E R - A N F E I W O R K S H O P • 2 faculty; and the inclusion of Latin American faculty in multi-day engineering education workshops at national and international engineering education conferences. The publication of an indexed engineering education research journal for Spanish-speaking audiences. The Revista ANFEI could be repositioned to become the premier engineering education research journal in Mexico and in other Spanish-speaking countries. The creation of Spanish electronic communication resources (e.g. websites, blogs, newsgroups) and networking venues (e.g. local, national, and regional conferences) to discuss and to disseminate information relevant to engineering education research. The development of international funding (e.g., within the National Science Foundation‟s Office of International Science and Engineering) and of educational partnerships between engineering education researchers in Latin America and in the United States. A N F E I - E E R W O R K S H O P • 3 About the Workshop The workshop, Building Capability and Communities in Engineering Education Research (BCCEER), is a joint effort by the Annals of Research on Engineering Education, the Journal of Engineering Education (JEE), and the global initiative on Engineering Education Research to: (1) Introduce engineering faculty interested in developing or extending their capability to engage in educational scholarship how to approach and to move their work in the direction of educational research. (2) Share with interested participants strategies to build and to sustain a community of engineering education researchers and to help them network with other global communities. In collaboration with diverse international partners, the workshop has been held in countries such as Malaysia, Taiwan, India, and South Africa. Building Capability in Mexico Recognizing the increasing global interest on engineering education as a research field, the Asociación Nacional de Facultades y Escuelas de Ingeniería (ANFEI) hosted the first BCCEER workshop in Mexico, as a result of its strategic partnership with the Journal of Engineering Education. The workshop was held in June 19th, 2009, during the XXXVI Conferencia Nacional de Ingeniería (a space devoted to promote the integral development of Mexican engineers and the academic exchange among ANFEI member institutions since 1958). The Mexican engineering community enthusiastically responded to the workshop invitation that was published in Spanish on ANFEI‟s website. During the four- hour workshop, forty-two participants actively engaged in individual work, group discussions, brainstorming sessions, and collaborative poster production. The workshop was conducted in English and Spanish given facilitators‟ diverse backgrounds (see Appendix A). The information discussed during the session1 was aimed to help participants to 1) identify the principal features of engineering educational research and how they compare with technical engineering research; 2) frame research questions, situate them within appropriate theoretical frameworks, and consider alternative research methodologies; 3) gain familiarity with Web-based and print resources; and 4) become more aware of emerging global communities and their networks and ways to connect with them. 1 Workshop slides are available at http://www.ce.umn.edu/~smith/docs/ANFEI Workshop RC v2.pdf A N F E I - E E R W O R K S H O P • 4 During the four- hour workshop, participants actively engaged in individual work, group discussions, brainstorming sessions, and collaborative poster production. Participants A total of 42 (12 women and 30 men) participants attended the workshop, and two Latin American countries were represented (Mexico with 41 participants and Colombia with 1 participant). Participants from Mexico represented 13 different entities and 21 higher education institutions across all eight ANFEI regions (see Error! Reference source not found.). During the group introductions, attendees indicated diverse experiences academically and administratively. Table 1. Workshop participants’ origin. Region State Participants from Mexico came from Attendees (Attendees) (Institutions) 13 different entities, representing 21 I (3) Sonora (2) 2 higher education institutions, across Baja California (1) 1 all eight ANFEI regions. II (1) Tamaulipas (1) 1 Durango (2) 2 III (3) San Luis Potosi (1) 1 Jalisco (1) 1 IV (2) Guanajuato (1) 1 V (3) Morelos (1) 3 Tabasco (2) 2 VI (5) Mérida Veracruz (1) 3 Quintana Roo (1) 1 VII (13) Yucatan (2) 12 VIII (11) Mexico D.F. (4) 11 Workshop Evaluation Facilitators developed workshop evaluation forms (WEFs; see Appendix B) and obtained approval from Purdue University‟s Institutional Review Board (IRB) to collect data from workshop participants via notebooks, posters, and field notes. All 42 participants retrieved their anonymous forms at the end of the workshop. WEFs consisted of ten Likert-scale questions and three open-ended questions. All A N F E I - E E R W O R K S H O P • 5 Likert-scale questions provided additional writing space, allowing participants the opportunity to further explain their responses. In the first section, participants assessed overall aspects of the workshop (i.e., fulfillment of workshop expectations, relevance of the information discussed, usefulness of the materials and sources presented, and opportunities to interact with others). In the second part, the four workshop objectives were evaluated. The information discussed in the following subsections represents the themes that were consistently emphasized by participants in the WEFs and workshop artifacts (participants‟ notebooks). The data analysis was complemented with facilitators‟ field notes. High level of interest in conducting/promoting EER Facilitators perceived an audience highly motivated to learn more about EER. This impression was confirmed by 36 affirmative answers when participants were asked (in the WEF) whether they were interested in conducting EER. Three participants did not respond to this question, and three more responded negatively (two of them explained that their current positions were too demanding to pursue EER actively). Participants interest in conducting EER was also evident when the Levels of Inquiry in Engineering Education (i.e., teacher, effective teacher, scholarly teacher, scholar of teaching and learning, or engineering education researcher) were discussed (adapted from Streveler, Borrego, & Smith, 2007). Presenters prompted the group to situate their actual and desired states along the continuum. Fifteen participants drew the continuum in their notebooks, with an average of level 2 (scholarly teacher) as their current state. Twelve participants marked level 4 (engineering education researcher) as the level they wanted to reach. The group was prompted to situate their actual and desired states in the Levels of Inquiry in Engineering Education continuum. Twelve out of fifteen participants marked level 4 as the level they want to reach. Level 0 Teacher Level 1 Effective Teacher Level 2 Scholarly Teacher Level 3 Scholar of Teaching and Learning Level 4 Engineering Education Researcher In addition to conducting EER, participants also expressed a desire to promote it at their home institutions. Comments such as “this is a new area to me and I want to develop it at my institution,” “I have a broader landscape to organize EER at my institute,” and “I will use them [the references] to involve other departments [in EER]” were written in the WEFs. The interest to promote EER even materialized in the form of an invitation to the Spanish-speaking facilitator to lead A N F E I - E E R W O R K S H O P • 6 a talk about EER at a Teaching and Research Week in a large public institution in Mexico. A need for more opportunities and support to engage in EER The most frequent comment made in the WEFs across all questions was related to length of the workshop. Eighteen participants made reference (at least once) to the “short duration” of the workshop, and described it as a “good introduction,” a “baseline,” or “frame of reference” for future work. However, they expressed the need for more opportunities that would allow for deeper discussions on how to design and to carry out the research studies. One of the questions on the WEF specifically asked how ANFEI could best support participants‟ interests in conducting EER. The two most popular answers (out of 31 comments) were related to training and dissemination. Participants suggested that ANFEI could 1) organize longer workshops (seminars, or certificates), perhaps in each region; 2) publish an indexed EER journal, potentially the Revista ANFEI; and 3) create forums to discuss and to disseminate information relevant to EER. In this regard, a couple of respondents proposed the creation of a special ANFEI division or vocalía to help with the coordination of the EER enterprise. Concerns about EER recognition Participants were also asked how their home “At my institution EER institutions could best support their interests in does not have any conducting EER. Within the WEFs, they identified value… This is very financial support and time allocation, as well as local frustrating and training and the establishment of partnerships with discouraging because any efforts in this other institutions, as the institutional support needed. regard do not have In relation to this point, the group expressed a big repercussions.” concern about the recognition of EER as a legitimate engineering research field in Mexico. The following comment, written in a notebook, captures the essence of the concern: “at my institution EER does not have any value… This is very frustrating and discouraging because any efforts in this regard do not have repercussions.” Another participant stressed the importance of establishing collaborations with academics in “other countries where EER is recognized” to get access to bilateral funds. Some participants suggested a top-down approach as the best way to address the perceived lack of EER recognition, involving entities like CONACYT (National Council for Science and Technology), ANUIES (National Association of Higher Education Institutions), SEP (Secretariat of Public Education), and even the federal government. A N F E I - E E R W O R K S H O P • 7 The language barrier As mentioned earlier, the workshop was conducted both “Something important is the language; many in English and Spanish, as they are the native languages of us do not understand of the facilitators. The workshop presentation (adapted English… and we are in from previous events) and handouts were in English, a Spanish speaking and the group discussions were held in Spanish. country.” Participants‟ responses to the WEFs and notes were written mostly in Spanish.2 Some participants indicated that the language was a barrier to fully understand the information discussed during the session: “something important is the language; many of us do not understand English.” The use of a simultaneous translation system to make communication more effective was suggested. Goals attainment The four workshop objectives were stated as affirmative sentences in the WEFs, and participants rated the extent to which they agreed with the affirmation on a 5- point Likert scale of totally disagree (1), disagree (2), neither disagree, nor agree (3), agree (4), and totally agree (5). Objectives 1, 3 and 4 obtained high ratings, with over 70 percent of the participants either agreeing or strongly agreeing with the assertions (see Table 2). On the other hand, only 52 percent of participants agreed with the attainment of objective 2. Table 2. Objectives attainment (5-point Likert, 5=strongly agree) Agreement Workshop objectives Average percentage* 1. I gained familiarity with the principal features of EER and how they 3.9 72 compare with technical engineering research. 2. I learned about the importance of framing research questions and 3.5 52 situate them within appropriate theoretical frameworks. 3. I gained familiarity with EER web-based and print resources. 4.1 80 4. I became more aware of emerging global communities and their 4.1 78 networks, and how to connect with them. *Percentage of participants that agreed or strongly agreed. The comparative low rating of goal 2 might be explained by 1) the nature of activities participants engaged during the session, and 2) participants‟ expectation of learning how to conduct EER (including formulating research questions). While small groups developed posters to describe and to compare the characteristics of EER and technical engineering research (objective 1) and created a list of journals (published in Spanish) where EER could be disseminated 2 All participants’ quotes presented in this report were translated by the Spanish-speaking facilitator. A N F E I - E E R W O R K S H O P • 8 (objective 3), they were not prompted to generate and/or to frame research questions. Facilitators hypothesize that objective 2 was associated strongly with participants‟ legitimate interests in having deeper, more specific discussions about the research process per se, as their comments about the need of talking about research methodologies, and presenting case studies indicate. In general, the overall aspects of the workshop were assessed positively (see Table 3). Participants‟ specific recommendations for improving the workshop included having a better arrangement of the room to work in teams (including having computers available), making the session longer, having a follow-up workshop, providing more details about conducting EER, using a simultaneous translation system, and sending information in advance to enrich the on-site discussions. Table 3. Participants’ feedback (5-point Likert, 5=strongly agree) Agreement Workshop overall aspects Average percentage* The workshop fulfilled my expectations. 3.9 75 The information was relevant and useful to me. 4.1 80 The material and resources discussed will help me in the future 4.5 92 I am interested in learning more about EER 4.8 98 Workshop facilitators explanations were clear and kept my attention 4.4 90 I had the opportunity to interact with others and learn from them 4.4 90 *Percentage of participants that agreed or strongly agreed. Summary and Recommendations As stated by Jesiek, Newswander & Borrego(2009), “engineering education research has experienced a notable scale-up in recent years through the development of departments and degree programs, high-profile publications outlets, research agendas, and meetings.” The „EER movement’ has also embraced initiatives to develop networks and build research capability among engineering faculty, for example the Advancing the Global Capacity for Engineering Education Research program, AGCEER (a joint enterprise between the European Journal of Engineering Education and the Journal of Engineering Education; see J. Lohmann & De Graaff, 2008), the Conducting Rigorous Research in Engineering Education project, RREE (funded by the USA National Science Foundation; see Borrego, Streveler, Miller, & Smith, 2008), and the series of Building Capability and Communities in Engineering Education Research workshops. A N F E I - E E R W O R K S H O P • 9 The findings discussed in this report are consistent with those reported from similar events conducted in countries such as Hungary, Turkey, and Hong Kong (see Borrego, Jesiek, & Beddoes, 2008). In particular, workshop attendees emphasized themes such as the [lack of] recognition of EER as a legitimate field, funding, training opportunities, and dissemination forums (e.g. journals, and conferences). The “lack of outlets for publishing their research,” and the “lack of recognition for engineering education researchers” were concerns also expressed by participants of the 2007 International Conference on Research in Engineering Education (Jesiek, Newswander, et al., 2009). The pace at which these perceived trends turn around depends in great measure on the momentum that professional associations on higher education and engineering education societies might imprint to EER. In this sense, ANFEI could play a critical role in the advancement of EER in Mexico, championing the movement and involving entities like CONACYT, ANUIES, and SEP. Establishing a partnership with the Mexican Council for Educational Research (COMIE) would be of particular benefit. Among the activities that ANFEI could pursue to promote EER we suggest: The organization of longer workshops, seminars, or certificates. With the support of international partners (i.e. ASEE) and local expertise, ANFEI could adapt successful educational research capacity-building models from the USA (e.g. Borrego, Streveler, Chism, Smith, & Miller, 2006; Fincher & Tenenberg, 2006). Recently established by University of Michigan (2009), the Certificate in Engineering Education Research could also serve as a model. Multi-day workshops could be organized at engineering education conferences so that faculty could engage in conversations with engineering education experts about their concerns and interests. In this way, faculty from Mexico and other Latin American countries may be mentored by leading researchers and can receive guidance about projects that might be developed in their home countries. The publication of an indexed EER journal. Currently, there are not mainstream EER journals in Mexico, and according to the Organization for Economic and Co-operation and Development, OECD, in 2000 there were only six Mexican journals devoted to educational research, half of them meeting the CONACYT standards (Fuhman, Murillo, & Valencia, 2004). The Revista ANFEI, published quarterly between 2004 and 2008 (ANFEI, 2009), could undergo a transformation similar to the one of the Journal of Engineering Education, in which the periodical has a dual mission (“magazine for the dissemination of society communications as well as a journal to publish ideas and innovations on engineering education;” J. R. Lohmann, 2005) to be repositioned as the premier EER journal in Mexico, serving also other Spanish-speaking countries. The creation of forums to discuss and to disseminate information relevant to EER. While there are a number of academics already conducting EER in Mexico, and several more interested in starting their paths in the field, there are no “formal mechanisms” in place to establish/sustain a Mexican EER community; A N F E I - E E R W O R K S H O P • 1 0 such mechanisms might include electronic communication resources (e.g. websites, blogs, newsgroups) and networking venues (e.g. local and national conferences) (Borrego, et al., 2006). Spanish EER forums have a potential broader impact within the Latin American community, since “extensive networks are not currently in place to connect [engineering education] researchers from different countries who share an interest in similar topics,” and “shared native language often seem to encourage collaboration” (Jesiek, Borrego, Beddoes, & Hurtado, 2009). We also believe that international funding and educational partnerships can be developed to from engineering education collaborations between Latin American and U.S. researchers. Engineering education researchers in the U.S. might include Mexican faculty interested in engineering education in federally-funded research projects that focus upon international collaborations. One program of interest within the National Science Foundation (NSF) is the Office of International Science and Engineering. In this way, U.S. researchers may engage students, postdoctoral researchers, and faculty in projects with Mexican researchers via NSF‟s Developing Global Scientists and Engineers (International Research Experiences for Students (IRES), Doctoral Dissertation Enhancement Projects (DDEP), International Research and Education: Planning Visits and Workshops, International Research Fellowship Program (IRFP), and Partnerships for International Research and Education (PIRE) (NSF, 2009). In addition, universities in the U.S. and in Latin America may engage faculty and students in partnerships that allow all parties to learn about engineering education in global contexts. One model that serves as an example is provided by the doctoral program in Science, Engineering and Technology Education at Universidad de las Americas, Puebla (UDLAP, 2009) in Mexico, where faculty members from the School of Engineering Education at Purdue University serve as Visiting Professors at UDLA, and faculty from UDLA serve on student committees at Purdue.. These partnerships may be replicated in other Departments of Engineering Education (i.e., Virginia Tech University, Clemson University, and Utah State University) and within other departments in Mexican or Latin American universities. The findings from this workshop put forward some recommendations in which ANFEI can promote the advancement of EER in Mexico and, more broadly, in Latin America, joining the efforts of the Journal of Engineering Education of forming a “global community of scholars and practitioners dedicated to advancing engineering education through education research” (2005). Acknowledgments We wish to thank Ruth Streveler, Karl Smith and Jack Lohmann, co-designers of the workshop, for their invitation to facilitate the session at ANFEI‟s XXXVI Conferencia Nacional de Ingeniería. We gratefully acknowledge the efforts of Barbara Olds, senior associate editor for JEE, and Mario Gómez Mejía, ANFEI executive secretary, in the organization and coordination the workshop. We specially want to thank the participants of the workshop for their eagerness and significant contributions. A N F E I - E E R W O R K S H O P • 1 1 References ANFEI (2009). Revistas de la ANFEI Retrieved Sep 30th, 2009, from http://www.anfei.org.mx/revistas.html Borrego, M., Jesiek, B. K., & Beddoes, K. (2008). Advancing global capacity for engineering education research: Preliminary findings. Paper presented at the 38th ASEE/IEEE Frontiers in Education Conference, Saratoga Springs, NY. Borrego, M., Streveler, R. A., Chism, N., Smith, K. A., & Miller, R. (2006). Developing an engineering education research community of practice through a structured workshop curriculum. Paper presented at the 2006 ASEE Annual Conference. Borrego, M., Streveler, R. A., Miller, R. L., & Smith, K. A. (2008). A new paradigm for a new field: Communicating representations of engineering education research. Journal of Engineering Education, 97(2), 147-162. Fincher, S., & Tenenberg, J. (2006). Using theory to inform capacity-building: Bootstrapping communities of practice in computer science education research. Journal of Engineering Education, 95(4). Fuhman, S., Murillo, F. J., & Valencia, S. M. (2004). Revisión nacional de investigación y desarrollo educativo: México: Organisation for Economic Co-operation and Development. Jesiek, B. K., Borrego, M., Beddoes, K., & Hurtado, M. (2009). Internationalizing engineering education research: Mapping countries and keywords to identify new collaborative horizons. Paper presented at the 2009 ASEE Annual Conference. Jesiek, B. K., Newswander, L. K., & Borrego, M. (2009). Engineering education research: Discipline, community, or field? Journal of Engineering Education, 98(1), 39-52. Journal of Engineering Education (2005). JEE strategic plan, 2005-2010: Advancing engineering education research worldwide. Journal of Engineering Education, 94(3), 283-284. Lohmann, J., & De Graaff, E. (2008). Advancing the global capacity for engineering education research (AGCEER): A year of international dialogue. Paper presented at the 2008 ASEE Conference & Exposition Pittsburgh, PA. Lohmann, J. R. (2005). Building a community of scholars: The role of the journal of engineering education as a research journal. Journal of Engineering Education, 94(1), 1-6. NSF (2009). National Science Foundation office of international science and engineering (OISE) Retrieved Oct, 2nd, 2009, from http://www.nsf.gov/div/index.jsp?div=OISE Streveler, R., Borrego, M., & Smith, K. (2007). Moving from the „Scholarship of Teaching and Learning‟ to „Educational Research‟: An Example from Engineering. To Improve the Academy, 25. A N F E I - E E R W O R K S H O P • 1 2 Universidad de las Américas Puebla (2009). Doctorado en educación de las ciencias, ingenierías y tecnología Retrieved September 10th, 2009, from http://www.udlap.mx/ofertaacademica/doctorados/educacioncienciasinge nieria/Default.aspx University of Michigan (2009). Certificate in engineering education research Retrieved September 10th, 2009, from http://www.engin.umich.edu/teaching/crltnorth/rackhamcert.html A N F E I - E E R W O R K S H O P • 1 3 Appendix A: Workshop Facilitators’ Biographies Monica F. Cox is an Assistant Professor in the School of Engineering Education at Purdue University. Her research interests include development and validation of engineering education assessment tools; assessment of engineering doctoral education; undergraduate and graduate students‟ acquisition of engineering norms, skills, and attributes within academia and industry; and assessment of pedagogy within engineering classrooms and laboratories. She obtained a B.S. in Mathematics from Spelman College, a M.S. in Industrial Engineering from the University of Alabama, and a Ph.D. in Leadership and Policy Studies from Peabody College at Vanderbilt University. She is a 2008 recipient of a NSF Faculty Early Career award and most recently named one of the 2009 recipients of the Presidential Early Career Award for Scientists and Engineers. Address: School of Engineering Education, Purdue University, Armstrong Hall, 701 West Stadium Avenue, West Lafayette, IN, USA 47907-2045; telephone (+1) 765-496- 3461; e-mail:

Rocío C. Chavela Guerra is a doctoral candidate in the School of Engineering Education at Purdue University. Her research interests involve faculty development, course and curriculum development, and engineering education research communities. She is an assistant professor (on professional leave) in the Chemical and Food Engineering Department at Universidad de las Américas Puebla (Mexico), where she received her bachelor and master degrees in Chemical Engineering. Address: School of Engineering Education, Purdue University, Armstrong Hall, 701 West Stadium Avenue, West Lafayette, IN, USA 47907-2045; telephone (+1) 765-496- 9666; e-mail:

A N F E I - E E R W O R K S H O P • 1 4 Appendix B: Workshop Evaluation Form (in Spanish) A N F E I - E E R W O R K S H O P • 1 5