(PDF) Development, Test, and Evaluation of New Continuing Education for Museum Staff in Scandinavia

Johan Lövgren, Lasse Sonne, Michael Noah Weiss (Eds.) New Challenges – New Learning – New Possibilities Folk High School Research Edited by Johan Lövgren (University of South-Eastern Norway) Volume 2 LIT New Challenges – New Learning – New Possibilities Proceedings from the 9th Nordic Conference on Adult Education and Learning Edited by Johan Lövgren Lasse Sonne Michael Noah Weiss LIT Cover image: Lovgren publishing Printed with support of the University of South-Eastern Norway Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at https://dnb.dnb.de. ISBN 978-3-643-91658-7 (pb) ISBN 978-3-643-96658-2 (PDF) DOI: https://doi.org/10.52038/9783643916587 This work is licensed under a CC BY 4.0 license. https://creativecommons.org/licenses/by/4.0/deed.de A catalogue record for this book is available from the British Library. © L IT VERLAG GmbH & Co. KG Wien, Zweigniederlassung Zürich 2023 Flössergasse 10 CH-8001 Zürich Tel. +41 (0) 78-307 91 24 E-Mail:

https://www.lit-verlag.ch Distribution: In the UK: Global Book Marketing, e-mail:

In Germany: LIT Verlag Fresnostr. 2, D-48159 Münster Tel. +49 (0) 2 51-620 32 22, Fax +49 (0) 2 51-922 60 99, e-mail:

Content Chapter 1 Introduction to the Nordic Conference on Adult Education and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 By Johan Lövgren, Lasse Sonne and Anne Cathrine Hogsnes Asplin Chapter 2 Research Lifelong Learning: then and now . . . . . . . . . . . 19 By Professor Michael Osborne Chapter 3 How to Begin? A philosophical investigation on the educational time and its political implications . . . . . . . . . 37 By Walther Omar Kohan, Johan Lövgren and Michael Noah Weiss Chapter 4 Citizenship, ‘Corona Dugnad’ and Free Spaces . . . . . . . . 59 By Jorun M. Stenøien and Christin Tønseth Chapter 5 Why Teach History for Adults? Trends in history teacher education from a national discipline to diversity, cross-cutting themes and learning outcomes with an emphasis on Europe and Norway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 By Lasse Sonne Chapter 6 To Know and be Known – Cross-sectoral learning initiatives aimed at older adults . . . . . . . . . . . . . 87 By Magdalena Popławska Chapter 7 The Folk High School and the Digital: Forever foes or saving grace? . . . . . . . . . . . . . . . . . . . . 103 By Julie Shackleford Chapter 8 Why Learn History, Religion and Ethics in order to Prepare Young People for Further Studies and Lifelong Learning? Opportunities and challenges in new curricula for upper secondary school . . . . . . . . . 121 By Anita Wiklund Norli and Lene Kristin Liabø Chapter 9 Development, Test, and Evaluation of New Continuing Education for Museum Staff in Scandinavia . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz, Martin Brandt Djupdræt iv Content Chapter 10 Addressing Ableist Normativity in Adult Education – Conceptualising Educational Ableism . . . . . . . . . . . . . . 163 By Tine Fristrup Chapter 11 Adult Learning in a Master’s Program in Accounting and Auditing – key challenges and employer-university cooperation to enhance learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 By Ellen M. Kulset, Eivind Ludvigsen and Kjell Magne Baksaaas Chapter 12 The Transition from Sectors of Adult Education to Systems of Lifelong Learning . Role of the OECD and the World Bank in facilitating policy change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 By Shalini Singh Chapter 13 Escape Games in Adult Education – A literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 By Peter Grepperud Chapter 14 Increased Work Demands for Early Childhood Education Leaders: A self-study on the use of case method in adult leadership education . . . . . . . . . . . 237 By Marit Bøe and Elsa Kristiansen Chapter 15 Caught between Study and Family: Mature female students’ thesis writing during Covid . . . . . . . . . . . . . . . 249 By Anne Larson, Pia Cort, and Helle Merete Nordentoft Chapter 16 Adult Education in the Tension between Market and Bildung. Concluding reflection from the editorial committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 By Johan Lövgren, Lasse Sonne and Michael Noah Weiss Author presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Chapter 9 Development, Test, and Evaluation of New Continuing Education for Museum Staff in Scandinavia By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz, Martin Brandt Djupdræt Abstract In the Nordplus Adult development project Increased Learning Through Social Spaces (2018-2022) we collected examples of social interactions at cultural heritage organisations in Scandinavia which contributed to in- teractivity and learning. In addition, we analysed how cultural heritage organisations actively create social spaces with the intention of promot- ing learning opportunities and how they evaluate these learning opportu- nities. Based on the examples, we developed a hypothesis on how various elements can create social interactivity and learning. By departing from research literature such as museum studies (Falk & Dierking, 2013) and lifelong learning (Jarvis, 2017), combined with the British planning and evaluation tool Generic Learning Outcomes (Graham, 2013) and a meth- odological approach departing from action research (Torbert, 2004; Ulvik et al., 2022), we developed a foundation for a new continuing education programme for museum staff on learning through social spaces. This pro- gramme was developed during the project period and tested and evaluated in September 2021 in Östersund, Sweden, on Scandinavian museum staff. In this chapter, we analyse the development of the continuing educa- tion for museum staff in Scandinavia and present the test and evaluation results. The evaluation programme consisted of an electronic survey dis- tributed to the course participants before and after the course, as well as observations during the course, discussions with the participants during the course and a concluding evaluation based on the Generic Learning Outcomes evaluation tool. By departing from educational practice, the Scandinavian research group will implicitly seek to answer the symposia question related to what the societal challenges are and how these looks like from the perspective of heritage institutions. The research group will seek to answer the question related to what possibilities that are generated for heritage institutions. Finally, the group answers the question of how adult education and learning related to heritage institutions might develop 140 9 Development, Test, and Evaluation because of these challenges and possibilities by, for example, engaging in new professional development or through the development of new profes- sions related to cultural heritage organisations. Keywords Adult education, museum institutions, museum staff, Scandinavia, Generic Learning Outcomes, social spaces, action research Introduction The museum institution is changing. The idea that museums are something naturally good has been replaced by result-oriented demands. As a gradual shift during the last four decades, the demands on museums are no longer only to manage, research, preserve and disseminate. Now they also need to know their audience by being a new type of resource in society. Muse- ums need to be relevant, accessible, and contribute to social inclusion and learning (Hooper-Greenhill, 2004, p. 152), among other things, by contrib- uting as a counterweight to the fragmentation of public discourse. Thus, it is expected that museums facilitate exchanges of opinion, allow for a diversity of meanings, and work for demographic diversity among visitors and staff (Brenna, 2016, p. 36). Museums are also seen as flexible learn- ing environments, where visitors must play an active role themselves. An expressed goal is to reach out to groups who make only limited use of the museum offerings. Following this, cultural life is now defined as important for bringing people together as a common public, whether it is physical or digital. Thus, a change has taken place in terms of how museums and museum activities should be understood. Museums are, in principle, related to all areas of lifelong learning. In the Scandinavian countries, the most developed area in relation to learning activities is the area of collaboration with schools (Sonne, 2020, p. 22- 23; Risan, 2020, p. 31; Olesen, 2020, p. 45). The museums’ opportunities in terms of adult education and lifelong learning are far less developed. Additionally, there is a great potential for development at the intersection of formal, informal, and non-formal learning, such as courses for new cit- izens with a focus on language learning or as an introduction to unfamiliar cultures and ways of being. In the latest European agenda for culture from 2018, culture has been given a completely different purpose in European cultural policy. Now, it is explicitly mentioned that culture must be an en- gine in economic development. This will be realized, among other things, by promoting culture and creativity in formal, informal, and non-formal education and learning at all levels of society. There is also a focus on By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 141 promoting skills that the cultural and creative sector needs, including a particular focus on digital and entrepreneurial skills. One goal is to develop new so-called ecosystems for cultural and creative industries with a focus on innovation capacity and cross-sectoral cooperation – i.e., cooperation between the cultural area and other areas in society (European Commis- sion, 2018, p. 4). Today, many museum employees lack the skills to adapt museum institutions to these new demands (Sonne, 2022, p. 105-106). With this in consideration, there is a great need for the development of new adult and continuing education to handle the new paradigm that muse- ums are now included in. Thus, our project goal was to develop a new continuing education programme for museum staff in Scandinavia. We wanted to train the staff in understanding and being capable of deal- ing with the museum as a space for social interactions and a place for learning. We decided to depart from a methodological approach based on action research and the literature about action research such as Reason & Brad- bury (2001), Torbert (2004), Ulvik et al. (2022), Tiller (2016) and Duus et al. (2012). Action research is systematic research on our own practise (Ulvik et al. 2022, p. 40), and was originally developed by Kurt Levin as a research method that could be used to resolve social conflicts (Levin, 1946). Today it is a method often used in educational science, but also in leadership research, where a goal might be to transform an organisation to, for example, achieve long-term sustainability (Tobert, 2004). In our case, the organisations are museums. In their handbook on action research, Peter Reason and Hilary Brad- bury define that action research “bring together action and reflection, the- ory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people” (Reason & Bradbury, 2001: p. 1).In a more practice-oriented description by Benedicte Madsen, action research is defined by these four elements: (1) When the researchers participate, intervene and/or take the initiative in incidents in the practice of the researched party, (2) when the researchers contribute to the prac- titioners’ extended insight, understanding and learning, (3) when the re- searchers contribute constructively to the development of the investigated practice and/or the investigated organization, and (4) when the researchers produce general knowledge or knowledge on the basis of the experiences from the participation (Madsen, 2012: 6). As will be demonstrated below, we have taken an initiative for in- tervening incidents in collaboration with the participants to expand their insight and produce knowledge based on their experiences. 142 9 Development, Test, and Evaluation As a part of the Nordplus Adult development project Increased Learn- ing through Social Spaces (Wollentz et al., 2021, 2022), a new course for museum staff was developed, tested, and evaluated. One goal was to devel- op new learning in museums that are seen as a social space. A second goal was to teach staff how different elements in museums can stimulate social interactions. Thirdly, we wanted to teach staff how social learning spaces can be evaluated and why that is important. The final goal was to teach the evaluation methods of social learning spaces in practice. In this chapter, we present the results of our work with developing continuing education for museum staff. The course was tested on 28–29 September 2021 at the museum Jamtli in Östersund, Sweden. The course curriculum, with its intended learning outcomes, was developed both vertically by the project participants and horizontally by involving the test course participants as co-creators of the course, not least when it came to formulating the learning outcomes in the course curriculum. The learning outcomes were developed and evaluat- ed through the planning and evaluation tool Generic Learning Outcomes (GLO) which will be introduced later in this article. In addition, a pre- course electronic questionnaire was sent to the course participants. 16 of the participants answered (Kurs om sociala rum, 2021). The answers were used to further formulate the anticipated learning outcomes. Furthermore, a questionnaire was sent to the participants after the course, to follow up the actual learning outcomes. This meant that two different tools were used to plan and evaluate the test course. 24 participants signed up for the course. The largest group was from Sweden, but participants also signed up from Denmark and Norway. Most had a professional background from museums, mainly as responsible for exhibitions. Two participants were from universities. The teachers and or- ganisers of the test course were the University of South-Eastern Norway, the Nordic Centre of Heritage Learning and Creativity in Sweden, Den Gamle By in Denmark and the Scania Regional Museum in Sweden. Development: pedagogical approach and learning outcomes There is a comprehensive amount of literature about methodological ap- proaches to learning and evaluation of learning that the project group used as background material to develop the test course (e.g., Ehlers, 2019; Hattie, 2015; Biggs, 2011; Prøitz, 2016; Andersen, Wahlgren & Wandall, 2017; Hylland, 2017). Participant-centred and self-directed learning were seen as important approaches to learning, partly because the target group By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 143 was adults, and partly, because according to the literature above, it is seen as the most effective pedagogical approach to learning. To develop the test course, the project development group discussed different possibilities in formulating the learning objectives. Several tools have been developed to measure the effects of learning, teaching, and ed- ucation, but they have primarily been developed for formal learning insti- tutions such as schools, colleges, and universities (Andersen, Wahlgren and Wandall, 2017). Thus, they are less useful in more informal learning environments, such as museums, where an immediate measuring of learn- ing and social effects can be challenging. Furthermore, the effects in a mu- seum context can be significantly different from the effects of teaching in a school. While in a formal education course one can measure learning over time via local and national tests set up against predefined criteria expressed in curricula, cultural institutions do not have the same possibilities. At the same time, there is still a need to be able to say something systematic about the audience’s learning after visits. One of the most ambitious planning and evaluation tools specialized for the cultural heritage area were developed in England by the Museums, Libraries and Archives Council project Inspiring Learning for All (Sonne & Banik 2021, p. 35–39; Thorhauge, 2014, p. 75ff). As part of this, a tool was developed for planning and evaluating learning at archives, libraries, and museums – The Generic Learning Outcomes (GLO). GLO is a holistic evaluation and planning tool. It is based on a con- structivist approach to learning, meaning that learning at cultural heritage institutions according to GLO should depart from participation and dia- logue. It emphasises the fact that visitors can also learn from each other in the process. In conversations they can, as the model also allows for, apply previous knowledge, and combine it with new observations and through conversation come to a new common understanding. GLO thus represents a break with traditional cultural heritage dissemination. Furthermore, GLO is a tool that can be used to structure learning activ- ities in museums by following questionnaires or qualitative interviews with its five main points, and it can also be used to create structure in already existing material. One of its strengths, according to its spokespersons, is that it helps museum staff to ask specific questions for both planning and evaluation, and thus contributes to a more fruitful dialogue with visitors (Jönsson and Peterson, 2011, p. 52; Thorhauge, 2014 p. 76–79; Knudsen, 2017, p. 40–41). GLO was developed within five areas (see Figure 1 below). The first, knowledge and understanding, focuses on the ability to create meaning or achieve deeper understanding, to create connections and relationships 144 9 Development, Test, and Evaluation between different things and apply existing knowledge in new ways. Skills, the second point, is understood as knowing how to do something concrete. It can be intellectual skills, such as being able to read, think critically and analytically and assess connections, the ability to understand numbers, to be able to write, or to handle large amounts of information, for example via information technology. Furthermore, it includes social, emotional, com- municative, and physical skills. The next point, attitudes, and values deals with feelings and perceptions in relation to us and other people. It also includes the ability to justify one’s actions and attitudes, and how they are connected. The fourth, fun, inspiration, and creativity is about the ability to make others and oneself happy. Inspiration is considered a prerequisite for learning and for being able to think and act innovatively and experi- mentally. It also contributes positively to motivation. The fifth is activity, behaviour, and progression. Living in a post-industrial society requires that people be able to change the way they have arranged their lives. This applies not only to work, but also in relation to family, studies, and being part of a larger social context in general. It is thus concerned with the need to exhibit proactive behaviour. To be able to master this, it is important to also be proactive when it comes to learning new things and developing new skills. In other words, it is about being able to implement change on a subjective and general level (Sonne, 2009, p. 14–19). GLO has been subject to some criticism. Among other things, it has been said that GLO only measures what the visitors themselves say about their own learning. (Brown, 2007, p. 22–30; Thorhauge, 2014, p. 83). An- other criticism is that GLO is not suitable for measuring learning effects over time. A substantial amount of learning happens after a visit to a muse- um, when there has been time for reflection. GLO has also been criticized as a tool being so broad that it can be difÏcult to define what learning is and is not. Despite these criticisms, GLO is seen as an important learning tool for the cultural area, as it is well suited to measuring other areas than purely economic values. Furthermore, GLO’s social constructivist starting point for planning and evaluating learning is in line with overall prevailing per- ceptions of how learning best takes place. To formulate our learning outcomes, we therefore used GLO as a start- ing point in the way that our course was supposed to stimulate the course participants’ developments in the following areas: 1. Knowledge and understanding 2. Skills 3. Attitudes and values By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 145 4. Enjoyment, inspiration, and creativity 5. Activity, behaviour, and progression. We asked the following questions in the digital questionnaire that was sent to the test course participants before the course: 1. How much previous experience do you have with producing ex- hibitions? 2. Have you previously worked on the development of socially stim- ulating museum environments? 3. What would you like to learn and improve by participating in the course? 4. What type of content do you assume is provided during the course? Please select alternatives and comment on them in the comment field 5. Which of these alternatives do you think are most important? Please explain why. The formulation of the learning outcomes as a co-creative process based on pre-evaluation for the test course was a comprehensive process. Here we only give some examples of the learning outcomes that we devel- oped. The answer of the first question gave some indication of the level of experience among the test course participants. 81 percent answered they had only little experience. 13 percent had great experience. 6 percent had no experience. 1. A representative answer on question 2: Have you previously worked on the development of socially stimulating museum envi- ronments? Was: “I have worked with social stimulation between people, but not between exhibition and visitor”. The answer is an example of how to construct a learning outcome. By having this kind of information beforehand, it was possible to develop a learning outcome in the following way: 1. The participant should have a greater knowledge of how to work with exhibitions to stimulate social stimulation between the visi- tors after the course. Some representative answers on question 3 What would you like to learn and improve by participating in the course? were, for example: 146 9 Development, Test, and Evaluation Tools to develop socially stimulating environments and situations The social interaction for learning and how to collect the experiences of the visitors Methods and practical examples on social interaction I would like to hear examples about social interaction at exhibitions As in question 2, it was now possible to formulate learning outcomes based on the participants answers. In this case, we formulated the learning out- come in the following way: 2. After the course, the participant should have knowledge and practi- cal experiences in socially stimulating the museum visitor through the development of exhibitions. Question 4, What type of content do you assume is provided during the course? Please select alternatives and comment on them in the comment field was answered with three different answers from the participants: About how social interactive environments are developed (32 percent) About the links between social interaction and learning at museums (30 percent) About the different elements that can stimulate social interaction at mu- seums (24 percent) Based on the participants’ answer on question 4, the following learning outcomes were formulated: 3. After the course, the participant should have knowledge, under- standing and practical insight into how socially interactive envi- ronments are created. 4. After the course, the participant should be able to understand links between social interaction and learning. 5. After the course, the participant should have an insight into the different elements that stimulate learning at museums. 6. In question 5, Which of these alternatives do you think are most important? Please explain why the participants were given the opportunity to point out the most important alternative for them. Most participants agreed in their answers, and provided answers like these ones: By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 147 The links between social interaction and learning Learning is something I am especially interested in The answers to question 5 meant that the course organisers in a learning outcome formulation emphasised the connections between social interac- tion and learning as a core area of the course. The learning outcome was formulated in the following way: 7. After the course, the participant should have a well-developed knowledge and practical experiences of the links between social interaction and learning at museums. In our project, a starting point was the general theory of social learning that we put into practise. For example, the British education scientist Pe- ter Jarvis argues that learning does not take place in isolation from a social context and that the combination of the concepts of social and learning is important (Jarvis, 2007, p. 1–2). Further, he argues that learning is a trans- forming process of all our experiences through thoughts, actions, emotions that transform ourselves as we build perceptions of an external world in our own biography. It is therefore not possible to detach oneself completely from a social context when learning something new. According to Jarvis, learning is a social thing and is also socially constructed (Jarvis, 2007, p. 5–7). The Importance of the social aspect of learning is also emphasized by, for example, the European Union (EU) through the key competenc- es for lifelong learning. A social competence is defined as the ability to work with others in a constructive way and handle conflicts in an inclusive and supportive context. Social competence is also about understanding the widely accepted patterns of behaviour and rules for communication in different societies and environments. These skills include the ability to learn and work with others and seek support, when relevant, and manage working life and social interaction effectively. The individual should be able to communicate constructively in different environments, collaborate in teams and be able to negotiate. This means that it is important to show tolerance, express and understand different points of view, create trust, and feel empathy (Henstillinger, 2018, p. 10). This social competence is based on a positive attitude towards one’s personal, social, and physical well-being and towards lifelong learning. It is attitudinally based on cooperation, impact, and integrity. This implies a respect for the diversity of others and their needs and a preparedness to overcome prejudice and be able to compromise (ibid.). Placing the muse- um as a social meeting place for lifelong learning is thus very relevant. Not 148 9 Development, Test, and Evaluation only can it be related to scientific learning theory, but it is also linked to political objectives for lifelong learning with a focus on social competence development, which is part of a broader societal development strategy to create economic, social, and sustainable development. In addition to social learning, we also included the learning spaces, in this case the museums as an arena for social learning. By doing so, we de- veloped the model illustrated in figure 1 where we applied the GLO plan- ning tool to the aims for the museums’ social interactions and the learning spaces, which resulted in our main goal: increased learning through the social spaces that museums constitute. Thus, an important learning outcome developed for the course was: 8. After the course, the participant should have developed skills and attitudes in interacting socially with other people (visitors) at a museum as a space for learning. Figure 1: Increased learning through social spaces. Sources: A further development of GLO developed by the Arts Council in En- gland based on the Nordplus development project Increased Learning through So- cial Spaces. The GLO-model is developed by Arts Council England. Other parts of the figure are developed by the Nordplus project group in Increased Learning through Social Spaces. By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 149 Testing the course In the beginning of the course the participants were presented with general theory of social learning and the museum’s role as a social space (Djup- dræt, Sonne & Banik, 2021). Thereafter, the participants were introduced to elements that inspire interaction (Hansen & Djupdræt, 2021). The seven elements found to prompt social interaction are divided into three main categories: reflection, physical impact, and activities. Often more than one of the elements is present in the social space at the museum. Table 1. The categories and elements that inspire social interaction in museums. Reflection Physical impact Activities Surprise Reminis- Ob- Staff in- Solo Collabora- Games & cence jects teraction activi- tive activ- competi- ties ities tion We will first describe the seven elements and then reflect on aspects men- tioned in the discussion with the participants of the course. Surprise. This element applies to the visitor experiencing something unexpected and varies from person to person depending on previous knowledge and experience. However, it is possible to construct elements in the exhibitions that intend to surprise. When visitors are surprised, they often turn to others and wonder what they are experiencing. Thus, surprises turn into positive social elements. Responses from visitors also suggests that doing so is a factor that makes the visit more entertain- ing and strengthens the learning process (Djupdræt & Hansen, 2021, p. 51–53). Reminiscence. This element focuses on people connecting some- thing in the exhibition to events or experiences in their own lives present or past. Through recognizable objects and situations, people might feel like sharing their own personal memories and experiences with others, which make these elements spark social interaction. In this way, learning is advanced not only through the objects on display in the museum, but in connection to events, objects, memories, and stories important to the members of the group (Djupdræt & Hansen, 2021, p. 48–51). Objects. Objects are to be understood as physical elements such as museum items, reconstructions, scenography, or graphic elements and cov- ers the museum’s classic features in an exhibition. The objects can create social interaction when they capture the visitors’ attention or curiosity, and 150 9 Development, Test, and Evaluation following that, initiate conversations, interaction and learning between the visitors (Djupdræt & Hansen, 2021, p. 57–61). Staff interaction. This element is about the interaction between the museum staff and the visitors. It can be a member of the staff who acts as a guide, who is part of a learning programme, who is available to answer questions, or who plays a role as a historical person. No matter what role the staff takes, the most important aspect is that the person is present in the museum and can make the museum experience more interactive (Djupdræt & Hansen, 2021, p. 54–57). Solo activities. This element is about activities designed for one per- son to try out alone, but the activity is also designed to be interesting, fun, or exciting for those who do not take part in it themselves but watch some- one else who do. This creates a social relation between the people who perform the activity and those who watch (Djupdræt & Hansen, 2021, p. 67–68). Collaborative activities. This element is an activity where you must collaborate in order for the activity/assignment to be solved (Djupdræt & Hansen, 2021, p. 61–63). Games and competitions. Games and competitions are engaging and can contribute to social interaction, either when individuals compete against each other or when trying to help each other (Djupdræt & Hansen, 2021, p. 63–66). Reflection from the participants. In the discussion of the elements during the course, the participants were free to choose what they want- ed to focus on and which elements they liked to share their experienc- es about. The most common chosen elements were the two reflective elements surprise and reminiscence. The participants were primarily museum staff, and one explanation could be that the elements connect- ed to Physical impact and Activities were already well-known tools used by exhibition planners, but the Reflection elements gave them a new insight, by explaining why the elements they already knew could have a successful social impact. The participants reflected on what the elements meant to the visitors and their use of the museum. An ele- ment frequently mentioned in the discussion was how visitors strived to find meaning that connected their museum experience to their own life, and here the social connection with other people had an important function and significance. Our own sense-making and sense of identity are both based on our own experiences (such as through reminiscence) but also closely linked to the dialogue and the reflection of the ongoing attitudes and actions of others. Meaning making is thus both individ- By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 151 ual and collective (Weick, Sutcliffe & Obstfeld, 2005), just as identity (Mead, 2005, p. 191; Castells, 2009, p. 6ff; Jenkins, 2016, p. 44ff, 98ff; Frello, 2012). During the course we presented the elements and had a session where participants discussed them and if they have come across or used some of the elements in their own work. Creating social spaces in practice The first day of the course ended with a workshop where participants had the opportunity to create and shape concepts for designing social spaces in dialogue with others. Prior to the course, each participant was asked to reflect upon and write down some words about an exhibition/museum production where social interaction is significant – either a current pro- duction or one that they wished to realize in a potential future. We asked the following questions: What does the exhibition consist of? What is the focus and purpose? Is there any social interaction among the visitors and what does it consist of? 14 participants answered the survey with different examples, ranging from an exhibition detailing the 1900s in the region of Scania, an exhi- bition on the experience of sleep and its health values, and an exhibition on traditions in Mexico. The connecting element was that each example included meaningful social aspects that the participant wanted to develop. Based on the examples provided beforehand, we divided the participants into four groups with four to five people in each group. The goal was to actively use the elements that stimulate social interaction presented earlier during the course (see above), to make the case more social. Furthermore, we wanted to participants to relate it to GLO in terms of what kind of learning could be stimulated through planned social interaction. We had the following time plan: • 20 minutes - individual reflection and development of their case in relation to the elements and GLO. • 20 minutes in a group - each person briefly presents their case to the others in the group. A case is selected to focus on. • 45 minutes - the group works together with their selected case based on GLO and the elements. • 20 minutes – everyone is gathered, and each group presents its case. Based on the feedback received and the evaluations many participants appreciated working on a specific case in dialogue with others and found the elements and GLO useful to work with. 152 9 Development, Test, and Evaluation Evaluating social learning spaces The second day started with a presentation on how to evaluate social spaces and why (Wollentz, 2021a&b). Our focus were our observa- tions that socially interactive environments were seldom evaluated in Scandinavian museums. Hence, we argued that the museum sector needs to improve its systematic evaluation and understanding of when, why, and how spaces in museums become social, and what kind of learning social interaction stimulates. Furthermore, we argued that it is an advantage to be clearer with how to evaluate different learning outcomes, since the learning paradigm has been criticized for being unclear in distinguishing between different forms of learning (Biesta, 2013). Additionally, we presented three different methods that were used in the project: 1) a survey with seven questions. Five related to the dif- ferent GLOs, and two related to the value of the social dimensions. 2) Observations noting social interaction in groups based on a scheme. 3) Semi-structured interviews following the GLO scheme but adding a social dimension. Emphasis was put on the value in combining several different methods to reach as nuanced a picture as possible, where also so- cial interaction without reflection is acknowledged, and which allows for many different forms of social interaction, including the ones not intended by those producing the exhibition or designing the activity. We showed how observations could recognize forms of social interaction that surveys would not, which often seemed to be based on preconceived ideas of what social interaction means, what a museum experience should consist of, or who is a learner. We also demonstrated the value of semi-structured interviews when the subject at hand is emotional or sensitive, since it is possible to note emotions through the way a person speaks or gesticu- lates. The format of semi-structured interviews is also beneficial in the context of difÏcult subjects, as it allows for follow-up questions in a way that surveys do not. Surveys, however, remain useful since these can be quantified more easily and are not as time-consuming as observations or semi-structured interviews. In sum, all three methods are useful depend- ing on the context and time at hand – especially when combined with each other. Evaluating social learning spaces in practice The presentation of the different forms of evaluations was followed by a practical exercise where participants could try the three different forms of evaluations (surveys, observations, and semi-structured interviews) in four By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 153 different exhibition spaces of the museum Jamtli. One was the permanent exhibition detailing the cultural history of Jämtland, another was a photo exhibition of different kind of Nordic food and landscapes, a third was an art exhibition focusing on Nordic myths, and lastly an exhibition on how food connects to values and identities in the past and in the present. In other words: we encouraged participants to apply in practice these evalu- ation methods in very different forms of museum spaces, where the social interaction expresses itself in different ways. The participants were divided into three groups to test the methods in combination with each other. Since there were few visitors at the museum at that time, the participants also evaluated each other. In such a way, it was both a practical exercise of test- ing different forms of evaluations and a role-playing activity of pretending to be a visitor to the museum and engaging in conversations. Interestingly, our own evaluation of the course noted that some participants found it use- ful to take the role of a museum visitor themselves, since it forced them to shift perspectives. However, others found it challenging and would have preferred to test evaluation methods on “real” visitors instead. The exercise was followed by a session where the participants en- gaged in a critical discussion of the different methods and how they relate to different kind of museum spaces. We also wanted participants to reflect upon how the methods can be combined to receive a layered and informative understanding. Many of the participants also provided input on how evaluations can be applied as well as how these three methods can be complemented with other aspects. It was stimulating to note how engaged the participants were, revealing that the subject of how to evaluate socially interactive spaces is important and rele- vant. Course evaluation result The evaluation was conducted in relation to the planned learning out- comes formulated through GLO and an electronic questionnaire de- veloped in the digital platform Crowdsignal after the course. The GLO-evaluation was conducted in the final session of the course (GLO-evaluation, 2021; Museet som ett social rum – efter kursen, 2021). GLO evaluation The GLO evaluation was conducted as a so-called café evaluation method (GLO-evaluation, 2013). Most of the participants in the course participated in the evaluation. The method organises the evaluation as 154 9 Development, Test, and Evaluation group discussions at “café tables” between four course participants. Each table has one leader who is also the referent for the table. The other participants circulate from table to table until all participants have taken part in the evaluation at all tables. In our evaluation of the course, we had five tables. Each table had one question related to an area in the GLO planning and evaluation tool. By doing so, we were able to evaluate the learning outcomes that were developed before the test course. In figure 3 and 4, we present the result of the GLO evaluation. A first impression is that the different areas of GLO are rather equally represent- ed. Enjoyment, Inspiration and Creativity gets the highest score with 3.7 or 24 percent followed by Activity, Behaviour and Progression with 3.5 or 23 percent. An interpretation of this could be that the participants had fun during the test course and that they were active learners during the learning process. The result shows that the intention of having a learner-centred approach and that the participants should be socially engaged most likely worked during the course. Figure 2. Result of GLO evaluation (average). Test course in Östersund, Sweden, 29 September 2021 (1=low, 5=very high). Source: GLO evaluation material developed by the project participants in the Nordplus Adult development project Increased Learning through So- cial Spaces. 29 September 2021. By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 155 Figure 3. Result of GLO evaluation (division). Test course in Öster- sund, Sweden, 29 September 2021. Source: GLO evaluation material developed by the project participants in the Nordplus Adult development project Increased Learning through So- cial Spaces. 29 September 2021. In addition, the areas of Skills, Knowledge and Understanding and At- titudes and Values scored almost equally with 3.0 or 19 percent, 2.7 or 17 percent, and 2.6 or 17 percent respectively. This meant that the participants developed new theoretical knowledge and practical skills during the course, and at the same time felt that their attitudes and values were challenged. One area the participants emphasised was that preconceived notions were challenged. In the case of Knowledge & Understanding, they emphasised an understanding of how to evaluate social spaces, for example, by using GLO. Regarding Skills, the participants emphasised that they had learned concrete practical methods to evaluate. Regarding Attitudes & Values, the participants, for example, emphasised new perspectives and insights as something that changed or challenged their attitudes and values together with the exchange of new ideas with other participants in the course. Enjoyment, Inspiration & Creativity had the highest score. In particu- lar, the participants thought it was inspirational and fun to meet colleagues from other museums to talk and socialise. The area of Activity, Behaviour & Progression was almost as high. The participants felt it was stimulating to work with evaluation methods in practice. They also felt it stimulating 156 9 Development, Test, and Evaluation for their behaviour that the learning activities have a high focus on the participants. For example, participants were given the opportunity to test themselves in the role of a museum visitor and acquire new insights into how it is like to be a visitor and not only an employee at a museum. Evaluation of co-created learning outcomes The result of the co-created learning outcomes was measured in a post course digital evaluation. 10 participants answered the evaluation ques- tions formulated in accordance with the pre-course evaluation. Despite the answers not being a full representation of the course in general, the answers nevertheless gave us some indications of trends. For example, the answers gave us insight into whether the learning outcomes had been reached as in- tended or if adjustments or improvements ought to be considered before a next course. In this article, eight learning outcomes were formulated based on the pre-evaluation. In response to learning outcome 1, the participants should after the course have a greater knowledge on how to work with exhibitions to stim- ulate social stimulation between the visitors The impression from the an- swers was that the participants had in fact developed a better understanding of which elements that stimulate social interactions in exhibitions. Concerning learning outcome 2, the participant should after the course have knowledge and practical experiences in socially stimulating the mu- seum visitor through the development of exhibitions. The response of the participants was that they had developed practical insights into how to cre- ate an exhibition. In learning outcome 3, the participant should after the course have knowledge, understanding and practical insight into how socially interac- tive environments are created. Our impressions from the feedback were that there had been a good mix between theory and practice during the course. Regarding social interaction at a museum with learning objectives (learning outcome 4), our goal was that the participants after the course should be able to understand links between social interaction and learning. With regards to learning outcome 4, the evaluation indicates that it is doubtful if this learning outcome was reached. In a weighted average of four questions, the participants’ answers indicate that the connection between social interaction and learning is the area they had understood the least, out of the four areas the questions concerned (2.10 of 4.00). Second lowest was learning outcome 5, “The participant should after the course have an insight into the different elements that stimulate By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 157 learning at museums” (2.50 of 4.00). Highest was learning outcome 2 (2.70 of 4.00). This indicates that regarding learning outcomes 4 and 5, the course organisers need to make improvements, to explore the links between social interaction and learning on one hand, and the connection between elements that stimulate learning and social interaction on the other. This might not be a surprise since we are exploring a new transdisciplinary approach to museum and exhibition studies combined with social studies and education science. Further information might be found in the answers on learning out- come 6, “After the course, the participant should have a well-developed knowledge and practical experiences of the links between social interac- tion and learning at museums”. In relation to this, one course participant commented that there should have been more links between learning and social interaction. Other comments from the course participants also addressed the lack of pre-knowledge before the course that might have made the links be- tween social interaction and learning clearer. One participant wanted “rel- evant theory and research literature connected to the practical exercises”. Another participant commented that there should have been “more clarity in the prior knowledge and the target group the course is aiming towards.” Thus, the evaluation of learning outcome 6 also address the important question about who the test course participants were. A clear majority was museum staff, working with exhibitions at their own museums. Most of the participants did not have a pedagogical education such as a schoolteacher education. Therefore, it might not be a surprise that the concept of learning was a new concept to many of the participants. Learning outcome 7 was in the evaluation question formulated as “The participant should after the course have developed skills and attitudes in interacting socially with other people (visitors) at a museum as a space for learning”. This outcome was about developing skills and attitudes in social interaction. The participants responded that it is important with social in- teraction. It is however unclear to what extent the participants developed this skill. Even if the answers were shallow in relation to learning outcome 7, they at least reflect that the participants had been made aware of the intention of the course to raise understanding of the importance of social interaction for learning, in this case at museums. Concluding remarks In the Nordplus development project Increased Learning through Social Spaces, we developed, tested, and evaluated a new course as a part of adult 158 9 Development, Test, and Evaluation and continuing education for museum staff in Scandinavia. Our ambition was to increase understanding and capability of dealing with the museum as a space for social interaction and a place for informal and adult learning. As a part of the course, the participants learned how elements in museums can stimulate social interaction. Another ambition was to teach staff the importance of why and how social learning spaces can be evaluated. Fi- nally, the goal with the course was to teach evaluation of social learning spaces in practice. We departed from a theory and methodological approach of action re- search as we wanted to study and stimulate change while carrying out our research project. The long-term aim of such a course would be to change museum professionals’ way of thinking about learning and social interac- tion and inspire them to work in a different way in future exhibitions. This change would be part of the learner’s identity and way of thinking, which in turn would affect the museum organisations’ way of approaching these issues. The results of the evaluation showed some interesting results. The GLO-evaluation showed that the participants enjoyed the test course, and that the test course affected their behaviour and progression. An area scor- ing lower, was the GLO-area knowledge and understanding, leading us to conclude that this part of the course could be developed further when the course is adjusted. The course developers also need to pay attention to the GLO-areas of attitudes and values; and the GLO-area skills when the course is adjusted. The post-course evaluation also gave useful results. The test course participants gave, in general, positive feedback. The most im- portant area for improvement, however, seems to be in the coupling of learning and social interaction/social spaces. This area is important to improve in the adjusted course because it was at the very centre of the development project and the test course. The result probably shows that a two-day course is too limited amount of time to learn, process, and internalize the concept of combining thoughts on social interac- tion and learning, which was new to many participants. Making the connection between the concepts and seeing them as intertwined also implies a change of the museum staff’s competencies and the museum institution into a learning organisation. These fundamental changes or reforms, however, take more than two days. On the other hand, the result of the evaluation showed that there is an interest among mu- seums to take part in the new paradigm of, for example, adult and lifelong learning. Nevertheless, it is important to be realistic about how fast this change might come. The test course showed that staff at By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 159 the museums want to be part of the change, but that it will take strong effort to change both the competencies of the staff, and with that, the orientation of the museum institution towards a more professional ap- proach to adult learning. References Andersen, M, Wahlgren, B. & Wandall, J. (2017). Evaluering af læring, undervisning og ud-dannelse. Hans Reitzels Forlag. Biesta G. (2013). Interrupting the Politics of Learning. Power and Education. 5(1):4– 15. doi:10.2304/power. 2013.5.1.4. Biggs, J. B. (2011). Teaching for quality learning at university: what the student does. RHE and Open University Press. Brenna, B. (2016). Kvalitetet og deltakelse i museer, i K. O. Eliassen og Ø. Prytz (red.), Kvalitetsforståelser. Kvalitetsbegrepet i samtidens kunst og kultur. Oslo. P. 36–52. Brown, S. (2007). “A Critique of Generic Learning Outcomes”, Journal of Learning Design, Vol. 2, No. 2, 2007. Castells, M. (2009). The Power of Identity. Wiley-Blackwell. Djupdræt, M. B., Sonne, L., Banik, V. K. (2021). Museet som sosialt rom og arena for livslang læring en introduksjon. PowerPoint-presentation. November 10, 2021. Djupdræt, M. B. & Hansen, A. (2021). “Måder at skabe sociale oplevelser og lærings- rum på museerne“, in Wollentz et al (eds). Museet som ett socialt rum och en plats för lärande. Östersund, Jamtli forlag, p. 43–70. Duus, G. et al. (2014). Aktionsforskning. En grundbog. Samfundslitteratur. European Commission. (2018). Proposal for a Regulation of the European Parliament and of the Council establishing the Creative Europe programme (2021 to 2027) and repealing Regulation, (EU) No 1295/2013, Brussels, 30.5.2018, COM(2018) 366 final, 2018/0190 (COD). Ehlers, S. (2019). The Rise of a Learning Regime in Europe. Transnational Policy Terms Mapped and Discussed. In: Fristrup, T. (ed.). Socially Engaged Practices in Museums and Archives. Fornvårdaren 38, Östersund, Jamtli Publishing, p. 17–68. Falk, J. H. & Dierking, L. D. (2013). The Museum Experience Revisited. Walnut Creek, CA: Left Coast Press. Frello, B. (2012). Kollektiv identitet - kritiske perspektiver. Frederiksberg: Samfund- slitteratur. GLO-evaluation. (2013). Café-table evaluation. Developed by Lasse Sonne. Graham, J. (2013). Evidencing the impact of the GLOs 2008-13. Learning Unlimited. January 2013. Hansen, A. & Djupdræt, M. B. (2021). Elementer som stimulerer social interaktion. PowerPoint-presentation. November 10, 2021. Hattie, J. (2015). The applicability of Visible Learning to higher education. Scholar- ship of Teaching and Learning in Psychology, 1(1), p. 79–91. Henstillinger. (2018). Rådet, Rådets henstilling af 22. maj 2018 om nøglekompetenc- er for livslang læring (EØS-relevant tekst), (2018/C 189/01), Den Europæiske Unions Tidende C 189/1, 4.6.2018. 160 9 Development, Test, and Evaluation Hooper‐Greenhill, E. (2004), “Measuring Learning Outcomes in Museums, Archives and Libraries: The Learning Impact Research Project (LIRP”). In International Journal of Heritage Studies, no. 2 (2004): 151–174. Visited 28.11.2021 on https:// doi.org/10.1080/13527250410001692877. Hylland, O. M. (2017). Museenes samfunnsrolle. Et kritisk perspektiv. Norsk museum- stidsskrift, nr 2, p. 77–91. https:// DOI: https://doi.org/10.18261/issn.2464-2525. Jarvis, P. (2007). Globalisation, Lifelong Learning and the Learning Society. London. Routledge. Jenkins, R. (2016). Social identitet. Aarhus: Academica. Jönsson, A. and Peterson, E. (2011). Det angelägna museet, GLO – ett språk för förän- dring. Exsamensarbete (30 högskolepoäng) i museologi för masterexamen inom ABM-masterprogrammet vid Lunds universitet.Supervisor: Pernilla Rasmussen. Knudsen, G., L. (2017). Motivations for and barriers against using the GLO-framework. A European Methodology for Museum-School Collaboration and Coursework. Editors: Thorhauge. S. & Falomo, L. Pavia. University Press. Horsens. Danish In- dustrial Museum. http://archivio.paviauniversitypress.it/oa/9788869520822.pdf. Kurs om sociala rum. (2021). Utvärdering före kursen. Crowdsignal-evaluation. De- veloped by Gustav Wollentz. Levin, K. (1946). Action research and minority problems. Journal of Social Issues, 2 (4): 34-46. Madsen, B (2012). Aktionsforskning. Illustreret med en case: aktionsforskning om ak- tionslæring. Psykologisk Institut, Aarhus Universitet. Mead, G. H. (2005). Sindet, selvet og samfundet : fra et socialbehavioristisk stand- punkt. Kbh: Akademisk Forlag. Museet som ett social rum – efter kursen. (2021). Utvärdering efter kursen. Crowdsig- nal-evaluation. Developed by Gustav Wollentz. Olesen, M. B. (2020). Museums and Education in Denmark. In Fristrup, T. (ed.). Mu- seums and Education and in the North. Fornvårdaren 39, Jamtli Förlag, Öster- sund, p. 37–49. Prøitz, T. S. (2016). Læringsutbytte. Universitetsforlaget. Risan, T. (2020). Museums and Education in Sweden. In Fristrup, T. (ed.). Museums and Education and in the North. Fornvårdaren 39. Jamtli Förlag. Östersund, p. 29–35. Sonne, L. (2009). Kulturarvens muligheder i almen kompetenceudvikling, Kortlægn- ingsrapport, Östersund: Nordisk Center for Kulturarvspedagogik. Sonne, L. (2020). Museums and Education in Norway. In Fristrup, T. (ed.). Museums and Education and in the North. Fornvårdaren 39. Jamtli Förlag. Östersund, p. 17–27. Sonne, L. (2022). Lærende frivillige på museer o gen ny profession i frivillighed- sledelse med focus på udvikling av en samfundsrelevant kulturarvspædagogik. Eksempler på et nyt paradigme i et europæisk perspektiv. In Fristrup, T. (ed.). Nordic Figurations of Volunteering. Fornvårdaren 41. Jamtli Förlag. Östersund, p. 85–112. Sonne, L. & Banik, V K. (2021). Kapitel 1. Livslang læring, sosial læring og museet som ramme for mellom, uformell og ikke-formell læring. In Wollentz et al (eds.). By Lasse Sonne, Anna Hansen, Vibeke Kieding Banik, Gustav Wollentz 161 Museet som ett socialt rum och en plats för lärande. Östersund: Jamtli förlag, p. 23–47. Tiller, T. (ed.). (2016). Aksjonsforskning. I skole og utdanning. Oslo. Cappelen Damm. Thorhauge, S. (2014). Interface learning. New goals for museum and upper secondary school collaboration, Aarhus University, Denmark. Torbert, W. (2006). Action Inquiry. The Secret of Timely and Transforming Leader- ship. San Francisco. Berrett-Koehler Publishers. Ulvik, M. et al. (2022). Å forske på egen praksis. Aksjonsforskning og andre til- nærminger til profesjonell utvikling i utdanningsfeltet. Bergen. Fakbokforlaget. Weick, K. E., Sutcliffe, K. M. & Obstfeld, D.. (2005) Organizing and the process of sensemaking. Organization Science 16, (4), p. 409–421. Wollentz, G. (2021a). Varför och hur kan vi utvärdera värdet av social interaktion på museer? PowerPoint-presentation. November 10, 2021. Wollentz, G. (2021b). Hur kan vi bättre förstå och utvärdera värdet av social interak- tion på museer? In Wollentz et al (eds). Museet som ett socialt rum och en plats för lärande. Östersund, Jamtli förlag, p. 71–82. Wollentz, G.; Djupdræt, M. B.; Hansen, A.; Sonne, L. & Banik, V. K. (eds). (2021). Museet som ett socialt rum och en plats för lärande. Östersund, Jamtli förlag. Wollentz, G.; Djupdræt, M. B.; Hansen, A.; Sonne, L. & Banik, V. K. (2022). The museum as a social space and a place for lifelong learning, Nordic museology (2), 2022: 23–42. DOI: https://doi.org/10.5617/nm.10069. Acknowledgement: a special thanks to Artur Ribeiro for language revision.