(PDF) Evaluating COs of computer programming course for OBE-based BSc in EEE program

International Journal of Learning and Teaching Volume 12, Issue 2, (2020) 86-99 www.ij-lt.eu Evaluating COs of computer programming course for OBE-based BSc in EEE program Muhibul Haque Bhuyan *, Department of Electrical and Electronic Engineering, Southeast University, 251/A and 252 Tejgaon Industrial Area, Dhaka 1208, Bangladesh https://orcid.org/0000-0001-9255-6708 Azwad Tamir, Department of Electrical and Computer Engineering, University of Central Florida, 4328 Scorpius Street, Building 116, Orlando, FL 32816-2362, USA Suggested Citation: Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Received date August 21, 2019; revised date December 6, 2019; accepted date March 25, 2020 Selection and peer review under responsibility of Prof. Dr. Hafize Keser, Ankara University, Ankara, Turkey. © 2020 United World Centre of Research Innovation and Publication. All rights reserved. Abstract It is an important and challenging task to develop concepts and skills of undergraduate engineering students in computer programming course and hence their evaluation on higher order skills. Already several methods are developed to evaluate the students of this course for various engineering programs, but a method for undergraduate electrical and electronic engineering (EEE) program was not found in the literature. In this paper, a simple evaluation method for the students of computer programming course of undergraduate EEE (BSc in EEE) program has been reported using result-oriented learning. Detail methodology, course syllabus design, course outcomes (COs) and mapping it with program outcomes (POs) of BSc in EEE, question setting following Bloom’s taxonomy, laboratory experiment, assessment plan, course and PO evaluation data and graphs have been presented along with relevant statistics. All data are presented for a cohort of students who took this course in summer 2019 Semester at EEE Department of Southeast University. It has been observed that the target set by the course teacher has been achieved by the students. Recommendations of the course teacher for further improvement of the COs’ achievement have also been presented. Keywords: CO evaluation, programming course, OBE * ADDRESS FOR CORRESPONDENCE: Prof. Dr. Engr. Muhibul Haque Bhuyan, Department of Electrical and Electronic Engineering, Southeast University, 251/A and 252 Tejgaon Industrial Area, Dhaka 1208, Bangladesh E-mail address:

/ Tel.: +0880-1815-657346 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 1. Introduction The University Grants Commission (UGC), Bangladesh, has permitted Southeast University (SEU) to run the Bachelor of Science in Electrical and Electronic Engineering (EEE) (BSc in EEE) program since 15 November, 2009. Since then it is accepting fresh students in every trimester (spring, summer and fall) of every academic year. So far, it has produced over 500 graduates from the first 18 graduating batches of students. However, its program had not got the accreditation from the Board of Accreditation for Engineering and Technical Education (BAETE), which works under the Institution of Engineers, Bangladesh (IEB) (BAETE, 2020). Therefore, graduates were facing several problems in the job market, such as getting jobs, becoming a member of IEB, getting promotion or salary hike, getting recognition, signing the engineering documents in the government and non-government offices and so on. Because, in many industries a graduate cannot get a job as an engineer until and unless he/she becomes a member of IEB and IEB does not allow any engineering graduate to become a member if the program is not accredited by BAETE. After getting this feedback from our graduates, the Department of EEE decided to apply to BAETE for accreditation of its BSc in EEE program in June 2017 to evaluate the effectiveness of this program. However, the program obtained accreditation only for 1 year (BAETE, 2019) and was asked to apply within January 2019 for outcome based accreditation (OBA). Then, EEE department took a hurried decision to design an OBE curriculum to be implemented from spring 2019 Trimester that spans from January to April. Based on new OBE manual that become effective since 1 July, 2017, the department of EEE re-applied for the accreditation visit from BAETE in March 2019. The departmental academic committee decided to use all the core courses under the new curriculum of the program to be assessed and evaluated based on OBE from the same trimester. Quality education is defined as the one that ‘provides the outcomes needed for individuals, communities and societies to prosper’ (Slade, 2017). At present, there are over 150 public, private and international universities operating in Bangladesh in extremely competitive way (UGC, 2020). Therefore, to attract students they must need to deliver world standard quality education in its different areas. Outcome based education is a process that ensures providing quality education to its graduates. Therefore, BAETE adopted OBE model to accredit an engineering program through a rigorous evaluation process. Although the OBA of engineering program through assessment activities is a new concept in Bangladesh, the willingness to accredit the program by a university is very encouraging. When EEE department started to implement the OBE curriculum all its full-time faculty members provided full support to implement it. In its curriculum, there are 21 theory and 16 laboratory courses in its core course group. Of them, computer programming course is divided into two parts – one is a 3-credit theory and the other is a 1-credit laboratory course. Theory course is of three credits and laboratory course is of 1 credit. This paper describes an assessment and evaluation process of computer programming course based on an assessment plan which has several performance indicators. These indicators are defined to quantity the fulfilment status of each course outcome (CO) and conforming program outcome (PO) from direct assessment (Sinha, 2017; Felder and Brent, 2003). These data are used as part of its re- accreditation process in 2019. The BSc in EEE curriculum adopted the 12 POs provided by BAETE, which are given in their new manual as the assessment criterion of POs (BAETE Manual, 2017). The result of this study provides EEE department of Southeast University (SEU), its concerned faculty members and the university management regarding the effectiveness of their academic programs. It also assists the department in developing strategies that extend the quality assurance framework to support sustainable quality education, which will contribute to produce creative and dynamic graduates who will be able to find adequate job opportunities and ensure satisfactory well- being. 87 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Nomenclature CO Course Outcome PO Program Outcome OBE Outcome Based Education OBA Outcome Based Accreditation EEE Electrical and Electronic Engineering Program CP Computer Programming SEU Southeast University BAETE Board of Accreditation for Engineering and Technical Education 2. Literature Review The assessment of POs has become an important part in engineering program evaluation to make sure that the program is effective, sustainable and providing quality education and also improving its quality and standard continuously (Sikander et al., 2017). As such, in engineering education, OBE is gradually being acknowledged and made mandatory by the accrediting agencies (Buzzetto-More and Alade, 2006). Outcome assessment is a kind of several tasks that are required for a program to evaluate through the various activities focusing on examination of students’ course or POs from one course to several courses. This is stupendous volume of tasks to assess and evaluate the each outcome of every student of a particular program to divulge the strengths, weaknesses, limitations, opportunities and obtain the recommendations for future improvement of the program (BAETE, 2017; Rashid, 2013; Mutaliba et al., 2012). By the term assessment, we mean the enduring methodical practices that are aimed at identifying, collecting, analysing and interpreting the achievements data of the learners of a class to measure the accomplishment of each CO and to compare the performance parameters between the actual and standard values. Finally, this process will also help to take appropriate actions for improving the student learning as well as for ensuring and refining the quality of student learning and hence the tertiary level education (Angelo, 1995). Effective assessment uses quantitative, qualitative, direct and/or indirect measures as appropriate to the outcome being measured (ABET, 2010). Moreover, the assessment is an essential part of certifying that a particular program of an educational organization full-fill the minimum required standards and has the necessary resources to provide quality education (Love and Cooper, 2004). The most important requirement of all accreditation agencies for any engineering program is to stipulate a bunch of skills and knowledge that they expect the students to attain upon completion of their graduation and to prepare an calculation process to ascertain the magnitude to which the POs have successfully been achieved by providing necessary supporting environments to the students achieving these course and program level learning outcomes. Besides, there should be a Continuous Quality Improvement (COI) process to close the loop for further development of the courses and hence the program (Sikander et al., 2017). These processes have been utilized to develop the teaching and learning practices at the course and program levels (Alzubaidi, 2017). COs mainly focus on the rational, interactive and collaborative development of students as it helps to become successful in the learning activities (Rajak et al., 2019). Course learning outcomes or simply the COs must identify the cognitive knowledge, psychomotor skills and behavioural attitudes that the 88 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 students must be able to achieve on successful completion of that particular course in their curriculum through the teaching-learning experiences (Asheim et al., 2017; Mason and Dragovich, 2010; Saulnier, 2014; Abdeljaber and Ahmad, 2017). COs have direct impact on the curriculum design of the engineering program as well as on quality assurance method. In OBE system, it represents a transition from the traditional teacher-centric teaching and learning methodology to the adoption of the student-centric teaching and learning approach, which gives an emphasis to the teaching and learning valuation relationship and the vital associations between the course syllabus design, knowledge transfer method and CO measurement (Adam, 2004). However, adopting CO based on approach has proven beneficial at the program level in the computer science program (Clarke and Reichgelt, 2003), individual courses level as well as at library level that allows the students to acquire the definite skills and abilities (Gowan et al., 2006). Different types of assessment schemes are being utilized to quantify the achievement of POs. In 2003, Sanders and McCartney described a set of 12 assessment tools that were used in the program accreditation process. These tools include would be graduate passing out survey, alumni survey, written and oral examinations of the graduates while leaving the program, portfolio of the external and industry advisory board, which have a set of limitations, as they are all considered as indirect assessment tools insofar not course-based (Sanders and McCartney, 2003). Another type of assessment focuses on course assessment tools (Blanford and Hwang, 2003), whereby instructors use various direct assessment tools to evaluate each individual student of various types of courses selected for the PO evaluation. Thus, CO assessment processes enable a program to demonstrate how a particular PO has been addressed in the curriculum. Hence, CO assessment has become a very time-consuming task. In 2017, Alzubaidi suggested an assessment approach for direct measurement of how well the individual students achieve the COs and as such the POs that define a set of measurable performance indicators in strong corelationship with the courses being taught. These performance indicators should be assessable characteristics recognizing the achievement needed to full-fill the demand of the POs (Rogers, 2003). 3. Objectives of the Work At present, it is an important and challenging task to develop concepts and skills of undergraduate students in computer programming course. However, this course is a very important course for any engineering students. Now in every engineering industrial sector, works and processes have been automated. So, for automation purpose, students need to learn any computer programming tools. It is a common belief that training on computer programming could build up students’ higher order learning and thinking skills. However, many faculty members have reported the difficulty of assessing and evaluating the teaching and learning level of this course. Several studies have been attempted to develop assessment and evaluation method of this course in several engineering and non-engineering programs (Sikander et al., 2017; Mutaliba et al., 2012; Rajak et al., 2019), but a method for undergraduate EEE program is not still available in the literature. Therefore, the main objective of this research task is to discuss and explain a simple technique to measure and evaluate the COs of the computer programming course to be used to calculate few POs out of the total 12 POs for the undergraduate level students of EEE program. The other purposes of this work are to: i. Study various literatures on outcome assessment evaluation process and formulate an evaluation plan for measuring the attainment of course and POs for computer programming course taught in the undergraduate EEE program. ii. Identify various measurable performance indicators for each course and PO. iii. Evaluate the level of achievement of each performance indicator. iv. Evaluate the achievement of each relevant PO mapped through CO. 89 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 v. Determine the strengths and weaknesses of the course and hence recommend for appropriate remedial measures to be taken by the departmental academic committee. 4. Methodology Today, academic program assessment is considered as a vital step to ensure quality education. At SEU, recognition of the importance of program assessment started back in 2017, when SEU management decided to apply to BAETE for three of its engineering program, namely Computer Science and Engineering (CSE), EEE and Textile Engineering (TE) for accreditation (BAETE, 2019). The main objective of this step was to provide these academic program offering departments with the necessary support to develop them as quality engineering education providers. As such, these departments tried to find necessary tools and techniques to set-up and measure their POs, take initiatives to ensure quality education through establishing necessary laboratory equipment required by the course curriculum, recruit qualified faculty members as appropriate to the program, prepare instructions and guidelines on various aspects of the program and COs, train the faculty members on OBE activities, orient the students about OBE system and policies, etc. The assessment and evaluation of the BSc in EEE program started in January 2019, where a model was defined and used to measure the achievement of its POs through its curriculum committee meeting based on various direct and indirect measurements. Courses included in the program were mapped to POs by examining individual COs of each course. The first step was to examine the achievement of learning outcomes of each course. Then, each PO was analysed individually based on data collected from class test, assignment, midterm and final examinations’ question-wise marks; feedback of the faculty members, existing students, alumni of EEE department, intern students, parents and employers to quantity its level of accomplishment (Mehdi et al., 2013). The results produced by this model were not accurate as the same weight was given to all direct and indirect assessment tools contributing in the assessment of the same PO. 4.1. Sample The sample of 12 students used in the study was chosen from the pool of undergraduate students enrolled in the computer programming course offered in summer 2019 Semester only of the academic year 2019 at EEE department of SEU. Data were collected from direct assessment tools of the computer programming course offered at the second semester of EEE department for one cohort of students. It is to be mentioned that EEE department of SEU started OBE curriculum implementation from spring 2019 Semester with the fresher’s admitted in that particular semester. To evaluate the students’ course and POs attainment based on direct assessment tools, we have used data only from the courses offered in spring 2019 and summer 2019 Semesters as we started to implement OBE at EEE department of SEU from spring 2019 Semester and OBE curriculum is effective from this semester. There was no indirect assessment tool for the students in these two semesters. 4.2. Course outcome A CO is basically a list of skills, competencies and/or attitudes a successful student will develop at the end point of a particular course. There may be further higher-level COs during the entire program, but the computer programming course is a very basic and fundamental course in the undergraduate EEE program (Kasilingam et al., 2017). The understanding of many courses where programming language is required depends on the clear understanding of this particular course as well. Therefore, the COs of the computer programming course should be planned in such a manner to develop the deep level understanding on a particular programming language can be attained by the learners of the computer programming course. In the computer programming course, the C language is taught to the students in both theory and laboratory class. To prepare the COs, we followed the SMART principles and the following standard formula for each COs: 90 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Action Verb + Condition + Standard Then, we wrote the following three COs for the computer programming course: [CO1] Define and explain the basic digital computer and programming language terminologies, algorithms, flow charts and C programming keywords, syntax and simple functions. [CO2] Apply C programming languages to write various simple algorithms and programs required for EEE problem solving. [CO3] Analyse different types of C and C++ programs and find corresponding valid outputs or results. At EEE department of SEU, course delivery is conducted face to face inside the classes, with Moodle e-Learning system and Google class room technique being used as a complementary learning management system, though the latter two techniques are yet to implement fully. Course syllabi are distributed to the students at the very 1st day of the class of each semester where all lecture and examination schedules, required text and reference books are mentioned, assessment plans are specified clearly (i.e., which questions of class tests, midterm and final examinations, assignment topics and course projects will be counted for CO assessment) and CO-PO mappings are also shown. 4.3. Program outcomes The Bachelor of Science in EEE program offered at SEU has four major areas of concentrations, namely power and energy, electronics, communication and computer. The B.Sc. degree in EEE requires that students earn the degree with any one area as his/her major and another one area as his/her minor with minimum degree requirements of 153 credits. Its curricula follow the guidelines set by the UGC, Bangladesh (UGC, 2018; Bhuyan and Khan, 2018) and BAETE, Bangladesh. The curricula have been designed by the academic and curriculum committee comprising five external academic and industry members and also taking the opinions of local and regional requirements from the Industry Advisory Panel (IAP) comprising ten external industry expert members and one alumni member. There are 12 POs of this program as suggested by the BAETE (BAETE Manual, 2017). Graduates are expected to be able to achieve the following qualities at the instant of their degree completion: [PO1] Engineering Knowledge: Apply the knowledge of mathematics, science, engineering fundamentals and an engineering specialization to the solution of complex EEE problems; [PO2] Problem Analysis: Identify, formulate, research the literature and analyse complex EEE problems and reach substantiated conclusions using first principles of mathematics, the natural sciences and the engineering sciences; [PO3] Design/Development of Solutions: Design solutions for complex EEE problems and design systems, components or processes that meet the specified needs with appropriate consideration for manufacturability and sustainability, public health and safety as well as cultural, societal, economic, political, ethical and environmental concerns; [PO4] Investigation: Conduct investigations of complex EEE problems, considering design of experiments, analysis and interpretation of data and synthesis of information to provide valid conclusions; [PO5] Modern Tool Usage: Create, select and apply appropriate techniques, resources and modern engineering and IT tools including prediction and modelling to complex EEE activities with an understanding of the limitations; [PO6] The Engineer and Society: Apply reasoning informed by contextual knowledge to assess societal, health, safety, legal and cultural issues and the consequent responsibilities relevant to professional EEE practice; 91 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 [PO7] Environment and Sustainability: Understand the impact of professional EEE solutions in societal and environmental contexts and demonstrate the knowledge of, and need for sustainable development; [PO8] Ethics: Apply ethical principles and commit to professional ethics, responsibilities and the norms of the EEE practice; [PO9] Individual Work and Teamwork: Function effectively as an individual and as a member or leader of diverse teams as well as in multidisciplinary settings; [PO10] Communication: Communicate effectively about complex EEE activities with the engineering community and with society at large. Be able to comprehend and write effective reports, design documentation, make effective presentations and give and receive clear instructions; [PO11] Project Management and Finance: Demonstrate knowledge and understanding of the EEE and management principles and apply these to one’s own work as a member or a leader of a team to manage projects in multidisciplinary environments; [PO12] Life-long learning: Recognize the need for and has the preparation and ability to engage in independent, life-long learning in the broadest context of technological change. At EEE department of SEU, course delivery is conducted face to face inside the classes, with Google Meet e-Learning system of SEU and Google classroom technique being used as a complementary learning management system, though the latter two techniques are yet to implement fully. Course syllabi are distributed to the students at the very 1st day of the class of each semester where all lecture and examination schedules, required text and reference books are mentioned, assessment plans are specified clearly (i.e., which questions of class tests, midterm and final examinations, assignment topics and course projects will be counted for CO assessment) and CO-PO mappings are also shown. 4.4. CO-PO mapping and performance assessment Performance Indicators (PI) are measurable quantities that every student must meet to justify attainment of the outcome in several courses of the program where he/she is studying (ABET, 2010). These parameters indicate what performance the learners is expected to establish in themselves right after their completion of their program. For each PO listed above related to the BSc in EEE program, the expected knowledge, skills and attitude required to achieve that outcome was listed to define various performance indicating parameters for the BSc in EEE program. Related specific teaching domain with levels, teaching-learning strategies and assessment tools of a particular course are identified and listed in Table 1 with the PO mapped to each CO of this particular course. To provide students with necessary knowledge of computer programming, various levels of cognitive domain in the teaching-learning strategies (from remember to evaluate) have been followed in this course. It has been observed that this technique is more realistic than that in the traditional technique of teaching- learning strategies (Bhuyan, 2014; Bhuyan and Khan, 2014; Bhuyan et al., 2014). The measurement of the attainment level of each CO and PO is carried out by using one or more components of the direct assessment tools (as shown in Table 1). The assessment plan is shown in Table 2 with the breakdown of each component of the direct assessment tools. These are mainly few questions of class tests (one taken before midterm and one before final examinations), few questions of midterm and final examinations. It is to be noted that the allotted full marks of each question are not same for all questions that are used for CO assessment. Table 1. CO-PO mapping, teaching domain, teaching-learning strategy and assessment tools of the computer programming course Program Teaching Domain Teaching-Learning Assessment Course Outcome Outcome and Level Strategy Tools 92 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 [CO1] Define and explain the basic digital Lecture computer and programming language PO1 Cognitive Domain/ Discussion terminologies, algorithms, flow charts (Engineering Remember/ Demonstration and C programming keywords, syntax Knowledge) Understand Question and Answer Direct and simple functions. Assessment Lecture Tools like [CO2] Apply C programming languages to PO3 (Design/ Discussion Class Test, write various simple algorithms and Cognitive Domain/ Development Demonstration Midterm and programs required for EEE problem Apply/ Synthesis of Solutions) Problem Solving Final solving. Question and Answer Examinations Lecture [CO3] Analyse different types of C and PO2 (Problem Cognitive Domain/ Discussion C++ programs and find corresponding Analysis) Evaluate Problem Solving valid outputs or results. Question and Answer Table 2. Assessment plan of computer programming course Course outcomes Assessment Tools Question # Full Marks CO1 CO2 CO3 Class Test 1 Q1 4 X Class Test 1 Q2 5 X Class Test 2 Q2 5 X Midterm Exam Q1a 2 X Q1c 3 X Q1d 3 X Q3c 5 X Final Exam Q1a 3 X Q1b 3 X Q3c 5 X Q4b 2 X Q4d 3 X A performance scale is also developed on the basis of the percentage of scores obtained in each CO contributed from different direct assessment tools discussed in Table 2. This is shown in Table 3 as per policy of the SEU across all the departments (Bhuyan and Khan, 2018). Initially, CO achievement target has been set to 50%; that means, 50% students of the cohort of this course should be in the satisfactory or above level. However, in satisfactory level, numerical scale is also 50%. Therefore, setting the 50% as the threshold level is justified. Table 3. Performance scale based on the percentage of marks obtained 93 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Performance level Numerical scale Excellent 80% and Above Very good 70–79% Good 60–69% Satisfactory 50–59% Developing 40–49% Unsatisfactory Below 40% 4.5. Program outcome assessment To measure the accomplishment levels of various POs, each CO is assigned to at least one PO each out of 12 POs set for the entire program of BSc in EEE of SEU. The level of attainment for each PO is computed as follows: i. Each CO contributes equally to each PO for this particular course. ii. From Table 1, we see that CO1, CO2 and CO3 help to achieve PO1, PO4 and PO2, respectively. iii. The percentage of scores is calculated and is assigned to PO contribution by each student. iv. The percentage of students in each CO is computed and this is same for a particular PO as well. v. A PO is considered achieved if the combined percentages of students in the ‘Excellent’, ‘Very Good’, ‘Good’ and ‘Satisfactory’ categories are within 50% score. This is roughly equivalent to 50% of the students scoring grade C+ (50%) and above. A PO is considered achieved if it scores a total value greater than or equal to 50%; an unachieved PO is defined with a score less than 50%. PO with a level of achievement between 50% and 59% are considered as marginally attained, and those with a total score of achievement between 60% and 69% need further improvements in knowledge and skills. Besides, PO with a level of achievement between 40% and 49% are considered as developing stage and may require additional care for the attainment of COs and POs. 5. Results and Discussion 5.1. Course and program outcome evaluation Table 4 gives a summary of the level of attainment of the course and hence the PO. Table 4 shows that the required engineering knowledge through programming language terminologies, algorithms, flow charts and C programming keywords, syntax and simple functions has been achieved above satisfaction level by more than 50% of the students in the class and hence the course and POs through this course have been achieved by this cohort of students as per first CO of the computer programming course and thus the first PO for these students have also been achieved partially. It is also observed that three students of the course did not take part in the process at all. Table 4. Course and program level assessment and attainment table PO >> PO1 (Knowledge) PO3 (Design/Development of Solutions) PO2 (Problem Analysis) CO >> CO1 CO2 CO3 MidQ1d MidQ1a MidQ3c MidQ1c FinQ1b FinQ4b FinQ4d FinQ1a FinQ3c CT2Q2 CT1Q1 CT1Q2 Total Total Total In % In % In % Tools* >> Std Std SL # 5 2 3 3 13 100.0% 4 3 5 2 14 100.0% 4 5 3 3 15 100.0% ID # Name 1 ID 1 Name 1 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 94 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 2 ID 2 Name 2 4 2 3 3 12 92.3% 4 3 4 2 13 92.9% 4 5 2 3 14 93.3% 3 ID 3 Name 3 0 1.5 3 3 7.5 57.7% 0 0 0 0 0 0.0% 4 5 3 3 15 100.0% 4 ID 4 Name 4 0 0 2 3 5 38.5% 0 0 0 0 0 0.0% 2 0 1 3 6 40.0% 5 ID 5 Name 5 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 6 ID 6 Name 6 0 1.5 3 3 7.5 57.7% 0 0 0 0 0 0.0% 0 2 2 3 7 46.7% 7 ID 7 Name 7 4 2 3 3 12 92.3% 0 0 0 0 0 0.0% 4 5 2 3 14 93.3% 8 ID 8 Name 8 5 2 3 3 13 100.0% 0 2 0 0 2 14.3% 4 0 2 3 9 60.0% 9 ID 9 Name 9 2 1 0 0 3 23.1% 0 0 0 0 0 0.0% 4 5 3 3 15 100.0% 10 ID 10 Name 10 0 2 3 3 8 61.5% 4 3 0 2 9 64.3% 0 5 3 3 11 73.3% 11 ID 11 Name 11 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 0 0 0 0 0 0.0% 12 ID 12 Name 12 2 1 2 2 7 53.8% 0 2 0 2 4 28.6% 0 5 2 3 10 66.7% * Legends for Tools: CT2Q2 = Question No 2 of Class Test No 2, MidQ1a = Question No 1 (a) of Midterm Examination, FinQ1a = Question No 1 (a) of Final Examination, ID 1 = Identification Number of Student No 1 in the class, Name 1 = Name of Student No 1 in the class (Real IDs and Names are not disclosed here). However, it shows that writing skills of various simple algorithms and programs required for EEE problem solving have not been achieved by them. Thus, the contribution to PO3 has not been achieved by most of the students except only two (16.7%) and thus entail substantial improvements for the CO2 and hence PO3. On the other hand, it is observed that the skills required to analyse different types of C and C++ programs and to find corresponding valid outputs or results are above expectation (58.3% of the students could achieve CO3 and hence PO2). It is interesting to note that one-third students could achieve score above 90%. Using the same way, we are assessing the COs and POs of the students and ultimately POs will be evaluated from all the COs of the core courses of BSc in EEE program. Table 5 gives a summary of the level of attainment of each of the COs by the number of students of the computer programming course and same data are produced in percentage in Table 6. Table 5. Number of students achieving the performance levels for COs of the computer programming course of EEE Department at SEU Excellent Very Good Good Satisfactory Developing Unsatisfactory CO1 3 0 0 2 0 4 CO2 1 0 0 0 0 8 CO3 4 0 1 0 2 2 Table 6. Percentage of students achieving the performance levels for the COs of the computer programming course of EEE Department at SEU Excellent Very Good Good Satisfactory Developing Unsatisfactory CO1 33.3% 0.0% 0.0% 22.2% 0.0% 44.4% CO2 11.1% 0.0% 0.0% 0.0% 0.0% 88.9% CO3 44.4% 0.0% 11.1% 0.0% 22.2% 22.2% 95 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Based on the results obtained from the tables, a graphical representation of Table 6 is produced as shown in Figure 1. Since the benchmark of achievement has been set as 50% for this course, it is seen that most of the students (around 90%) could not achieve CO2 and hence could not contribute to their PO3 from this course. However, from CO1 to CO3 above 50% students could contribute toward their PO1 and PO2, respectively, from this course. Therefore, it needs much care and refinement in the CO design, teaching-learning strategy and question setting following Bloom’s Taxonomy. Figure 1. CO evaluation matrix 96 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 5.2. Suggestions for improvement through outcome evaluation To improve the level of attainment of COs and hence POs, the EEE department has developed a set of possible remedial actions. One or more remedial action(s) may be applied to the computer programming course. Actions can be one or more of the following: a. Engaging students with more home works and assignments. b. Assigning students with more laboratory exercises at home. c. Engaging students with more group work that is relevant to achieving CO2. d. Devoting more time for tutorial classes to the areas where improvements are needed. e. Suggesting a different text or reference book, may be more than one. f. Using a different teaching-learning method/strategy, especially to address non-attainment of CO2, psychomotor domain is recommended to be applied. g. Replacing the course instructor with a one who has industry expertise. h. Giving more emphasis to independent works done by the students. i. Reducing correlational level of a particular CO to the corresponding PO. The EEE department has required number of concerned faculty members to document all the activities to be performed for the purpose of progressing the attainment of COs/POs and to submit at the end of the following offering semester a report indicating whether there have been any significant improvements on the achievement levels of COs/POs as a result of their engagements. 6. Conclusion This paper describes a simple method that the EEE department of SEU has exercised to collect, compile and then evaluate the course and POs achieved by the learners of the computer programming course offered by this department. This data of the outcome-based evaluation process are used for applying the re-accreditation of its BSc in EEE program through OBE-based curriculum and teaching- learning process to the BAETE. This method used several direct assessment tools to assess three COs of the computer programming course and hence its contribution to POs. After the end of the semester, the course teacher has submitted a detail report to the department chairman for further incorporation of it to the POs at program level. For this purpose, a database is also developed. Faculty members has also submitted the recommendations and analysis to the department suggesting few improvement plans to help the department chairman to decide which action/s need to be taken through its OBE activity committee. From the assessment of computer programming course, we observed that out of 3 COs, 2 COs are achieved and contributed to POs of the program. However, the recommendations for corrective measures suggested by the course teachers may be implemented before offering this course in the next semester so that the next cohort of students can achieve all the COs of this course. Implementing a new method to measure the achievement of POs for any academic program helps the institution to identify challenging parts and take appropriate corrective measures. The method described in this paper is very much generic; it may be applied to any course of any academic program with measurable COs. In this study, data collection and analysis were carried out manually. These two tasks require a considerable amount of time from the faculty members. As a future work, we will develop a software tool, which could facilitate the whole process of assessing and evaluating COs and hence POs of any academic program. In some case, several performance indicators and rubrics may be required to develop to assess and evaluate the students’ knowledge in cognitive domain, skills in psychomotor domain and behaviour in affective domain that are required to attain by each student at the point of their graduation. Therefore, we can say that the outcome assessed by the computer programming course will help the department to implement the OBE process successfully in the EEE department of SEU. 97 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 References Abdeljaber, H.A.M., & Ahmad, S. (2017). Program outcomes assessment method for multi-academic accreditation bodies: Computer science program as a case study. International Journal of Emerging Technologies in Learning, 12(5), 23-35. ABET. (2010). Criteria for accrediting computing programs. Accreditation board for engineering and technology. Computing Accreditation Commission. Retrieved 2019 August 07 from http://www.abet.org Adam, S. (2004). A Consideration of the nature, role, application and implications for european education of employing learning outcomes at the local, national and international levels. Paper presented at the Bologna Seminar. Edinburgh, United Kingdom: Heriot Watt University. Alzubaidi, L. (2017). Program outcomes assessment using key performance indicators. Boston, USA: Proceeding of 62nd ISERD International Conference. Angelo, T. (1995), Reassessing (and defining) assessment. AAHE Bulletin, 48, 7-9. Asheim, C., Gowan, A., & Reichgelt, H. (2017). Establishing an assessment process for a computing program. Information Systems Education Journal, 5(1), 3-12. BAETE. (2017). Board of accreditation for engineering and technical education. BAETE Accreditation Manual 2017 an Introduction. Retrieved 2019 August 12 from http://www.baetebangladesh.org/download.php. BAETE. (2019). List of programs under process of accreditation. Retrieved 2020 April 05 from http://www.baetebangladesh.org/programs.php BAETE. (2020). About BAETE Bangladesh. Retrieved 2020 April 08 from http://www.baetebangladesh.org/baete.php Bhuyan, M.H. (2014). Teaching electrical circuits course for electrical engineering students in cognitive domain. Journal of Bangladesh Electronics Society, 14(1-2), 83-91. Bhuyan, M.H., & Khan, S.S.A. (2014). Teaching a numerical analysis course for electrical engineering students in the cognitive domain. International Journal of Electrical Engineering Education, 51(1), 82-92. Bhuyan, M.H., & Khan, S.S.A. (2018). Motivating students in electrical circuit course. International Journal of Learning and Teaching, 10(2), 137-147. Bhuyan, M.H., Khan, S.S.A., & Rahman, M.Z. (2014). Teaching analog electronics course for electrical engineering students in cognitive domain. Journal of Electrical Engineering, the Institute of Engineers Bangladesh, 40(1-2), 52-58 Blanford, D., & Hwang, D. (2003). Five easy but effective assessment methods. Association Bulletin 35, No. 1 for Computing Machinery, ACM SIGCSE. Buzzetto-More, N., & Alade, A. (2006). Best Practices in e-Assessment. Journal of Information Technology Education, 5, 251-269. Clarke, F., & Reichgelt, H. (2003). The importance of explicitly stating educational objectives in computer science curricula. ACM SIGCSE Bulletin, 35(4), 47-50. https://doi.org/10.1145/960492.960524 Felder, R.M., & Brent, R. (2003). Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education, 92(1), 7-25. Gowan, A., MacDonald, B., & Reichgelt, H. (2006). A configurable assessment information system. 7th conference on information technology education. USA: SIGITE. https://doi.org/10.1145/1168812.1168833 98 Bhuyan, M.H. & Tamir, A. (2020). Evaluating COs of computer programming course for OBE-based BSc in EEE program. International Journal of Learning and Teaching. 12(2), 86-99. https://doi.org/10.18844/ijlt.v12i2.4576 Kasilingam, G., Nithiyananthan, K., & Mani, P.R. (2017). Implementation and assessment of outcome based education in engineering education. International Journal of Pure and Applied Mathematics, 117(17), 217-228. Love, T., & Cooper, T. (2004). Designing online information systems for portfolio-based assessment: Design criteria and heuristics. Journal of Information Technology Education, 3, 65-81. Mason, G., & Dragovich, J. (2010). Program assessment and evaluation using student grades obtained on outcome-related course learning objectives. Journal of Professional Issues in Engineering Education and Practice, 136(4), 206-214. Mehdi, R., & Naaj, M.A. (2013). Academic program assessment: A case study of a pragmatic approach. Creative Education Journal, 4(1), 71-81. https://doi.org/10.4236/ce.2013.41010 Mutaliba, A.A., Rahmata, R.A.A., Rashida, A.K.A., Sujaa, F., & Sahrila, S. (2012). Measurement and evaluation of program outcomes in the civil engineering courses. UKM Teaching and Learning Congress, 60, 333-342. Rajak, A., Shrivastava, A.K., & Shrivastava, D.P. (2019). Course outcome attainments in OBE for weak students. International Journal of Innovative Technology and Exploring Engineering, 8(11), 506-509. Rashid, M.H. (2013). The process of outcome-based education-implementation, assessment and evaluations. American Society for Engineering Education, ASEE International Forum, Paper ID #8242, Retrieved 2020 April 07 from http://www.asee.org/public/conferences/27/papers Rogers, G. (2003). Do grades make the grade for program assessment? Retrieved 2019 August 27 from http://www.abet.org/wp-content/uploads/2015/04/do-grades-make-the-grade.pdf Sanders, K., & McCartney, R. (2003). Program assessment tools in computer science: A report from the trenches. ACM SIGCSE Bulletin, 35(1), 31-35. Retrieved 2019 August 18 from https://doi.org/10.1145/792548.611926 Saulnier, B. (2014). A Paradigm for student learning outcome assessment in information systems education: Continuous improvement or chasing rainbows? Information Systems Education Journal, 12(1), 4-14. Sikander, T., Aziz, H., Wasim, A., Hussain, S., & Jahanzaib, M. (2017). Continuous quality improvement (CQI) framework: A case of industrial engineering department. International Journal of Cognitive Research in Science, Engineering and Education, 5(1), 107-119. Sinha, G.R. (2017). Lecture based mapping towards achieving excellence in outcome based education (OBE) framework. International Education and Research Journal, 3(6), 227-229. Slade, S. (2017), What do we mean by a quality education? Retrieved 2019 August 20 from http://www.huffingtonpost.com/sean-slade/what-do-we-mean-by-a-qual_b_9284130.html UGC. (2018). Guidelines for preparing standard curriculum of four year degree in engineering program. University Bangladesh: Grants Commission Bangladesh. Retrieved 2020 April 09 from http://www.ugc.gov.bd/site/view/policies UGC. (2020). List of public, private and international universities in Bangladesh. Agargaon, Dhaka: University Grants Commission. Retrieved 2020 April 09 from http://www.ugc-universities.gov.bd/public- universities 99