Article | DOI: 10.21307/eb-2018-002 Issue 2 | Vol. 2018 ffectiveandefficientcommitteework:ystematic overviewofmultidisciplinaryliteratures Author: Sandy Oliver, EPPI-Centre, UCL Institute of Education; and Africa Abstract Centre for Evidence, University of An era of financial constraints calls for effective and efficient commit- Johannesburg. tee work when making collective decisions. A systematic search iden- Katie Hollingworth, Thomas Coram tified research literatures in business administration, health research Research Unit, UCL Institute of and service development, and social psychology addressing deci- Education. sion making about highly technical issues by mixed groups of people. Rob Briner, School of Business Existing empirical and theoretical syntheses were drawn together to and Management, Queen Mary identify learning about the structure, processes and environment of University of London. committees and the characteristics of effective chairing. Committee performance depends upon the individuals involved, their attributes Catherine Swann, Public Health England. and relationships; and the time available for a committee to explore their knowledge to make choices or solve problems. In general, Kate Hinds, EPPI-Centre, UCL groups with six to twelve members tend to perform better than those Institute of Education. in either smaller or larger groups, especially when relying on virtual Chris Roche, Institute for Human communication. Diverse groups take account of a range of opinions Security and Social Change, La and enhance credibility and widespread acceptance and implemen- Trobe University, Australia. tation of decisions but may be more difficult to convene and manage Author contact: appropriately. However, where chairs manage conflict constructive-
[email protected]ly, more varied membership leads to better performance and more
[email protected]reliable judgements. These small-scale interactions reflect the larger
[email protected]scale institutional relationships, hierarchies and cultures which act
[email protected]as a backdrop to committee activities. These findings suggest that
[email protected]effective committee performance is enhanced by: appointing mem-
[email protected]bers from all key stakeholder groups who between them bring the appropriate range in educational and functional background, while keeping the group size close to 6-12; appointing committee chairs for their facilitation skills and generalist background rather than specialist knowledge; allowing sufficient time to allow all relevant knowledge to be shared and evaluated through discussion, especially when judge- ments need to be made by committees with members who vary in status; applying formal consensus development processes; and, par- ticularly when working virtually, considering the challenges of devel- oping trust and cohesion, and integrating divergent perspectives. Collective deliberation and decision-making processes advisory bodies, groups and committees in the public, involving people with differing backgrounds are cen- commercial and charitable sectors. Although deci- tral to most policy and governance decisions. Mak- sion-making groups vary in their terms of reference ing collective decisions commonly involves convening and their terminology, the generic terms ‘committee’ and managing small decision-making groups such and ‘board’ share similar meanings: a committee as committees or boards for the task. These include being a body of people ‘appointed or elected (by a © 2018 Australia and New Zealand School of Government and the authors. This is an 1 Open Access article licensed under the Creative Commons License CC-BY-NC-ND 4.0, https://creativecommons.orgw/licenses/by-nc-nd/4.0/ ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures society, corporation, public meetings etc) for some Finally, with an increased emphasis on transparency special business or function’; and a board being a of decision making in public life[1], committees must be body of people ‘officially constituted for the transaction diverse, well-informed and efficient. They should also or superintendence of some particular business, indi- take steps to ensure that key aspects of their process- cated by the full title as… ‘Board of Directors” (Oxford es are recorded in a way that is accessible and clear English Dictionary, 2015). Given the similarities of and leaves their work open to public scrutiny. This these definitions, in this report we use the term ‘com- may be achieved through holding meetings in public, mittee’ as the generic term, and the terms advisory publication of papers and minutes, or use of technolo- groups, bodies, panels or boards to describe specific gies such as electronic voting to ensure decisions are circumstances. precisely recorded. For instance, a more efficient (and Committee conventions to support decision- transparent) model has been proposed where group making include formal agendas, speaking through members make more contributions through a ques- the chair, voting and minutes of meetings. The pri- tionnaire and hold fewer meetings (Raine et al., 2005). macy of the expert group or committee for decision This review therefore synthesises empirical and making in public life can be seen across all sectors theoretical studies about groups engaging with highly and disciplines, from local authority planning deci- technical information and applying clear and efficient sions to national positions on scientific issues and processes that take into account diversity of mem- the development of legislation (Levy, 2007). Applying bership in order to make collective decisions. these conventions to decision-making for policy and practice in the 21st century introduces multiple chal- Existing reviews of the effectiveness lenges. The first challenge is, given the strong policy and efficiency of committees support for evidence-informed decision making, the requirement for group members to engage with high- Despite the centrality of the committee to so many ly technical information in order for decisions to be and to such far-reaching decisions, the evidence on based on high quality research. There is also a risk how to establish, run and govern such groups is sur- that, rather than critically evaluating options, groups prisingly fragmented. This may reflect in part the range may conform and make poor decisions through of different committee ‘models’ that exist, and also the ‘group think’ (Turner and Pratkanis, 1998). wide range of settings and sectors in which they oper- The second challenge arises from policy support ate. Evidence about methods and processes involved for involving a broad range of stakeholders, including in the application of the committee model to health professionals, users of public services and wider so- has been comprehensively synthesised (Murphy et al., ciety, all of whom may consider the issues from dif- 1998) and partially updated ten years ago (Hutchings ferent perspectives, who bring different knowledge, and Raine, 2006). It has also been given international expertise and beliefs to the processes and who will attention in a series of systematic reviews (see Oxman have varying degrees of familiarity with the technical et al., 2006a), but even the most recent of these re- information under consideration. The more inclusive views was published over a decade ago. a group is, the greater extent to which members ex- There is additional research about similar groups pertise will vary: expertise may be certified by profes- in commercial and charitable sectors. For instance, sional qualifications; it may be demonstrable through a study of the factors influencing the effectiveness of an individual’s ability to frame or solve problems; or research ethics committees by Schuppli and Fraser it may be accrued from daily experiences at home, (2007) identified aspects of committee composition, work and elsewhere – or it can be aggregated from dynamics, recruitment methods, motivation for join- across different domains (Blackmore, 1999). Hierar- ing, workload and member turnover. chies, mutual expectations and discussions can be The authors stated that one of the assumptions shaped by narrow and broad attitudes towards differ- behind the creation of research ethics committees is ent types of expertise (Stewart, 2007). that decisions made by groups are superior to those Thirdly, in an era of financial constraints, committees made by individuals. Their results, however, pointed must achieve these high quality, evidence-informed to some features of group decision-making – com- decisions as quickly as possible, with minimum wast- mittee structure, social influences and recruitment ed effort or resource: committees need to be both processes – which can lead to biases or polarisation. effective and efficient. In practice, this can mean improving efficiency by reducing time taken to reach decisions, for example, by replacing some face-to-face 1 https://www.gov.uk /government/topics/government- meetings and travel time with electronic interaction. efficiency-transparency-and-accountability. 2 There is a body of literature from the corporate NICE develops and manages the methods and and charitable sectors that focuses on the effective- processes through which evidence across health and ness of boards. A survey of charity boards concluded social care is brought to its committees for decision that board inputs and other characteristics are impor- making, but the committees themselves are comprised tant in explaining board effectiveness, namely: ‘the of independent experts. The transparency of method board has a clear understanding of its role and re- and process, and the independence of its committees, sponsibilities; the board has the right mix of skills and form the foundation for NICEs approach to guideline experience, and members had the time to the job development. Like many organisations who employ well; the board and management share a common expert committees or advisory groups in their decision vision of how to achieve their goals; and the board making, NICE continually review their methods and and management periodically review how they work processes: This review was designed to address prac- together’ (Cornforth, 2001, 225). Further studies ex- tical questions about the effectiveness and efficiency amine boards in relation to structure, size, gender of committees, first in terms of what works and how, and ethnic diversity. However, in many cases stud- and then in terms of frameworks, models and theories ies of these boards defined effectiveness largely in to understand how decision-making groups operate. terms of positive financial outcomes for the company (Boone et al., 2007; Coles et al., 2008; Linck et al., Research questions 2008; Upadhyay et al., 2014a,b). Recent psychology literature offers experimental Most of the research questions were identified by designs testing the effects of initial preferences and NICE which, like the rest of the public sector, must en- pressures of time, distraction and stress on group sure that it uses resources efficiently and effectively. decisions (Kelly and Loving, 2004). Electronic com- To maximise the learning for international readers, a munications, which might improve both inclusiveness question was added about contextual influences. The and efficiency, have advanced and the feasibility of overarching question was: online expert panels has been tested (Khodyakov et al., 2011). What does the evidence tell us about the effec- Concepts from practitioner literatures about team tiveness and efficiency of committee work? building, based on the principles of Tuckman’s model of teams forming, norming, storming, performing and This question included sub-questions about: adjourning, have since transferred into academic liter- atures (Bonebright, 2010). 1. Committee structure and environment a. The optimal composition (e.g. topic general- ists or specialists, past committee experience/ This review skills, demographics – gender, ethnicity, age) There are jokes about committees keeping minutes and size for decision-making committees, and and losing hours, but expert committees remain the the advantages and disadvantages of groups sine qua non for complex decision making in pub- of different compositions and sizes (i.e. impact lic life. This is because, despite risk of group biases on the outputs and of decision making) and dynamics impacting on decision making (see for b. The impact of environmental factors on com- example Tversky and Kahneman, 1974; Gigone and mittee work (e.g. layout, environment, acous- Reid, 1993), committee structures and processes tics, lighting, heating, air conditioning, spatial have the potential to support transparent production capacity). of good quality decisions, informed by multiple stand- 2. Chairing points – cornerstones of policy and legislation in the a. The most effective type of committee chair 21st Century. The challenge is to achieve these out- (competencies, skills e.g. topic specialists vs comes with ever-increasing efficiency. Knowing how, generalists). when and where to produce those efficiencies is key and a lack of recent syntheses from across multiple 3. Committee processes disciplines prompted the current review. a. The impact of meeting length, number and Since its inception in 1999, the United Kingdom’s timing on committee work National Institute for Health and Care Excellence (NICE) b. Effective and cost-effective processes and has used independent advisory groups or committees, structures for supporting committee decision formed of both professional (clinical, research or practi- making (for example, consensus techniques, tioner) and ‘lay’ (service user or third sector) members. decision support tools) 3 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures c. How use of different media (e.g. video con- • Empirical studies such as experimental stud- ferencing, email) for committee interaction ies, evaluations of interventions, surveys, case impact on decision making and costs studies, observational studies, longitudinal d. Equity considerations associated with differ- studies, and systematic reviews to elucidate ent committee structures and processes what works. e. The impact of training on technical and engagement issues for committee chairs, Only studies published in 1996 or later were committee members and secretariat. included, although reviews of research referred to earlier studies. This date corresponds with the date 4. Committee context of a systematic search for consensus development a. The influence of socio-economic context on studies by Murphy et al. (1998). how committees work and achieve their aims. Studies were excluded if they only addressed: This focus was added for the 2018 update. • determinants of group structures or proce- dures Methods • performance of the host organisation (e.g. fi- In addition to identifying the research questions, NICE nancial performance of the organisation, or guided the work to maintain the focus on practical implementation of guidelines) and not the perfor- challenges of running committees. We also consult- mance of the committee (e.g. decision-making ed three experts individually who, between them, had performance) experience of guideline development groups, audit • members drawn entirely from outside to advise committees and corporate or public sector boards. an organisation. Other input was invited from forums and networks debating related issues (Twitter, and two LinkedIn An iterative search was designed to capture stud- groups on public involvement in research). We drew ies from diverse literatures quickly: broad yet sim- on their responses when discussing the evidence ple searches that cut across academic disciplines available in the research literature. (Google Scholar and the library catalogue at the Evidence was eligible for inclusion in this review if it London School of Economics and Social Scienc- met at least one criterion in each of the following sets: es); searching a set of electronic sources each of which targeted particular contexts or approaches Populations to research; and searching for systematic reviews. A subsequent strategy of inspecting reference lists and searching for citations of eligible studies was chosen • Committees that make decisions about high- as a rapid way of identifying additional and similar ly technical matters and comprise a range of relevant studies. stakeholders, including people from outside of The original search was conducted in 2014. the organisation. Having recognised from this search the learning avail- • Facilitation of discussions and decisions about able from reviews in this area, and having chosen highly technical issues by mixed groups of our theoretical framework, we updated our review people. in 2018 by searching sources rich in systematic re- views of committees (Cochrane Effective Practice Outcomes and Organisation of Care review group, Health Sys- tems Evidence and the Campbell Collaboration); and • Committee effectiveness, in terms of perfor- searching for systematic reviews that cited systematic mance, including quality of decision-making. reviews included in our 2014 synthesis. • Committee efficiency, in terms of performance All reports were appraised for their relevance to the within time or resource limits. overall review question and where possible, matched to a specific sub-question. Their methods and find- ings were appraised for the type of evidence they of- Types of evidence fered. A range of evidence drawn from different types of reviews and primary studies was identified and • Frameworks, models or theories for under- used to address different questions. In general, when standing or assessing the performance of addressing each sub-question for this review we drew committees. on literature reviews for which we could discern how 4 authors had identified and selected studies to meet The reasons for exclusion are provided in Figure 1. clear or implied eligibility criteria. For evidence about Of the 106 studies addressing the overarching question models to understand or assess the performance driving this review, the most common focus was the of committees, we drew on theoretical syntheses of development of clinical guidelines (48 studies). There literatures. were also significant literatures investigating commit- The synthesis was conducted in two stages. tees within business administration (34), and the group The first stage synthesised findings for each review dynamics or facilitation practices (19) or psychology sub-question that had been reported by earlier system- (13) of group decision making. Studies of committees atic reviews. These findings offered evidence of what (rather than experimentally convened decision-making works and presented explanations for what works. groups) came from two different areas: guideline devel- The second stage synthesised findings from re- opment and business or public sector administration. search syntheses, some systematic, that offered This literature included 60 primary studies. Within frameworks for clarifying the meanings of ‘effective’ this set, the development of clinical guidelines was and ‘efficient’ when applied to committees, or mod- more often addressed by qualitative (14) and exper- els or theories to enhance understanding of decision- imental designs (6), whereas business administra- making groups. tion (such as corporate boards or audit committees) Iterative searching identified 1,320 items, 142 of was more often addressed by observational studies which were duplicates. The remaining 1,178 items (12). The most common studies addressing group were screened and 1,072 were excluded, leaving 106 dynamics or facilitation practice had experimental relevant items (See Fig. 1). designs (11). Figure 1: Flow of studies from initial search (in 2014) identifying titles to inclusion in the review. 5 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures There were also many literature reviews (47). Some models, and theories that have been developed from reviews were conducted systematically (28): 16 sys- synthesising studies to understand committee perfor- tematic reviews (one of these was underway at the time mance and decision-making. of this review); and 12 systematic reviews of systematic reviews. Ten reviews developed frameworks, models Reviews synthesising empirical or theories for understanding or assessing the perfor- evidence mance of committees (only two of these reported sys- tematic searches). The updating search in June 2018 Composition and size of decision-making identified 81 additional, potentially relevant reviews. We groups excluded 68 because they did not address committee performance, and two because we could not access There is little generalisable evidence for how the char- the full reports. The remaining 11 were added to those acteristics of participants and groups influence the originally identified four years earlier, giving a total of judgements produced in formal consensus develop- 39 systematic reviews and nine unsystematic reviews, ment methods (Hutchings and Raine, 2006). However, including 13 that developed a model or framework. multi-specialty groups tend to be recommended over With so many reviews available over the past single specialty groups in order to take account of a 20 years (Fig. 2), primary studies were excluded from wider range of opinion (Murphy et al., 1998; Hutchings subsequent analyses. Only those reviews synthesising and Raine, 2006). Similarly, larger groups offer empirical evidence systematically (see Table A1 in opportunities for more diverse membership which, appendix) and/or offering explanations through when managed well, lead to better performance frameworks, models or theories (see Table A2 in (Murphy et al., 1998). They also offer more reliable appendix) were included in further analysis. judgements but may be more difficult to manage The review findings are presented in two formats. and encourage equal participation. Below about six The first takes each review question above in turn and participants, reliability declines quite rapidly, with im- summarises the evidence of what works, followed provements in reliability subject to diminishing returns by studies offering explanations. The second takes with more than 10–12 participants (Murphy et al., a more holistic approach, drawing on frameworks, 1998), particularly for groups working virtually (Acai Figure 2: Systematically synthesised literatures (Key: Systematic reviews (deep blue); systematic reviews of systematic reviews (grey); systematic review with limited access to literature (light blue)). 6 et al., 2018). Larger groups allow the membership to leaders need to be proficient in using the technology reflect a broader range of background, skills and opin- and ‘effective in helping team members monitor and ions of the population of experts from which the par- manage their performance’ (Acai et al., 2018). ticipants are drawn (Murphy et al., 1998; Acai et al., 2018). This may also enhance credibility and wide- Timing of committee work spread acceptance and implementation of decisions (Murphy et al., 1998; Pagliari et al., 2001; Fretheim, In ‘laboratory studies’, where prior knowledge was 2006a; McCoy et al., 2012; Yassi et al., 2013). They also manipulated, groups tended to focus their discus- allow more varied membership which leads to better sions more on information that all members know at performance (more perspectives and considerations the outset, although this was less so when they had of alternatives), particularly for non-routine tasks, to choose among a small number of decision alterna- although conflict may arise between diverse partic- tives and were pressed for time (Reimer et al., 2010). ipants. The difference in performance and accept- ance of including experts in a committee versus them Effective processes and structures for inputting via consultation has not been explored. supporting group decision making Financial expertise, which positively correlates with the quality of audit committee’s outputs (Bilal and In general, the formal consensus methods tested, such Bushra, 2018; Velte, 2017), is moderated by independ- as Delphi studies, perform better than informal tech- ence and personal workload (Bilal and Bushra, 2018). niques but the reasons for this are not clear (Murphy Members are more likely to advocate familiar options et al., 1998). Some aspects that are likely to be impor- (which may come with a financial interest, although the tant include: ensuring that all members have a chance evidence for this is drawn largely from recommenda- to voice their views; ensuring that all options are tions made outside committees) (Murphy et al., 1998; discussed; providing feedback and repeating the Hutchings and Raine, 2006). These initial opinions judgement; and ensuring that individual judgements may affect the group process (Murphy et al., 1998). For are made confidentially. Qualitative research concludes instance, initial consensus may be followed by a shift that a good facilitator who can ensure that the proce- to a more extreme decision. Alternatively, following dure is conducted properly will enhance consensus an initial split view, members will either move towards development (Murphy et al., 1998). Providing guide- one another’s views or cohesive subgroups may line groups with the review literature results in deci- form to polarise views. Groups with similar compo- sions that are closer to the available research evidence sitions are likely to reach similar conclusions (Murphy (Raine et al., 2005). Effective group processes that et al., 1998). need to be supported include: challenging assertions, managing constructive conflict, teamwork, common Environmental factors sharing of goals, active engagement and openness (Jonsdottir, 2010). Conflicts of interest should be as- There is a lack of evidence about the impact of certained (Boyd and Bero, 2006) and the appointment environmental factors (such as room layout, décor, of group members should be based on objective and acoustics, lighting, heating, air conditioning, spatial explicit criteria (Ramsay et al., 2010). Delphi studies capacity) on group decision-making but research vary in their design and their definitions of consensus suggested that groups valued good working condi- (Diamond et al., 2014). Recommendations for con- tions (Oseland and Burton, 2012). ducting and reporting Delphi studies are now available (Jünger et al., 2017). A checklist for guideline develop- ment that includes size, membership, group interac- Competencies of effective chairs tions and decision-making has been developed from There is little rigorous evidence reported in system- guidance manuals (Schünemann et al., 2014) although atic reviews about what specific competences make the nature of the underlying evidence is unclear. an effective chair, although corporate board leaders are seen to have a crucial role in establishing inclusive Use of media for committee interaction working procedures and an atmosphere of open- and decision making ness, dialogue and trust (Jonsdottir, 2010). Facilita- tors can help groups to generate more ideas through Although formal processes appear to be better than encouraging members to express diverse opinions informal processes, there is no evidence to suggest and by delaying expressing their own opinion there are any major differences in the outcomes (Murphy et al., 1998). For groups working virtually, achieved between the effectiveness of the Delphi 7 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures method (used with geographically dispersed groups) tance for effective group functioning and individual and Nominal Group Technique (for face to face meet- skill building (Ward and Preece, 2012). Training and ings) (Murphy et al., 1998); rather, they may be more development for NGO Board members, essential for or less suitable for different purposes and circum- their understanding of Board responsibilities, ‘spans stances. For example, the former may prevent undue from orientation of a new director to training to im- influence by individuals, and the latter may provide prove teamwork, to ongoing training on topics related better opportunities for discussion. However, virtu- to governance’ (Shore, 2015, p. 98). Virtual groups al teams may struggle to develop mutual trust, team may benefit from training to enhance communication, cohesion without subgroups emerging), and integrat- developing trust and managing conflict, and regular ing divergent perspectives: they work well generating feedback about team performance (Acai et al., 2018). ideas through asynchronous communication but without immediate feedback afforded by face-to-face Reviews developing frameworks, conversation find bringing ideas together more diffi- cult (Acai et al., 2018). models and theories We found 12 reviews that developed frameworks, Issues of equity models or theories for understanding or assessing the performance of committees. We found these re- The commercial and public sectors may encourage views were able to provide explanations that helped diversity in boards, but there is little evidence that it make sense of the discrete pieces of evidence offered improves performance (Murphy et al., 1998; Kagzi by reviews of empirical evidence described above. and Guha, 2018). At consensus conferences, partic- A systematic review of corporate board evaluations ipation is uneven and related to status and expertise, identified 35 indicators for assessing performance so that people have participated actively and continu- across seven domains, five of which addressed pro- ously, intermittently, or little. In smaller groups, mem- cesses (Sajadi et al., 2013). The first process domain bers with higher status/expertise/initial position often was the members themselves: what they brought to exert more influence over the group. If members’ sta- the role, their understanding of, commitment to and tus is equal or similar, majority opinion influences the participation in that role; and their relevant external re- outcome of decisions that require judgement. Other- lationships. Other domains were: leadership strength wise, higher status members sometimes have greater and style; structure; processes (meetings, selection influence over judgements. For intellectual group and appointment, education, evaluation); and board tasks whoever finds the correct answer tends to have dynamics and relationships. These domains accord most influence regardless of their status. with many of the empirical findings presented above. The influence of women on corporate boards, However, more can be learnt from papers that synthe- which have traditionally been male dominated, was sised evidence framed by existing theoretical explana- limited unless they made alliances with the most tions of Board governance (Brown, 2005; Schoenberg influential members; women tended to have more in- et al., 2016) or conceptual models of board perfor- fluence when they were well prepared and employed mance (Minichilli et al., 2009), or developed models challenging questions as their main contribution about group decision making (Forbes and Milliken, (Jonsdottir, 2010). 1999; Vandewaerde et al., 2010; Brodbeck et al., 2007; Matta et al., 2016). Impact of recruitment and training Forbes and Milliken (1999) integrated the litera- ture about the effectiveness of boards of directors Public and voluntary sector boards place greater em- with the literature about group dynamics and work- phasis on initial selection and recruitment of board group effectiveness to develop a theoretical model members than on training and development (Ward of effectiveness and efficiency. This model offered and Preece, 2012). For NGOs, a clear organisation two criteria for board effectiveness: the ability of mission statement, well specified roles for board the board to perform its tasks effectively; and the members and clear recruiting procedures are essen- board’s ability to continue working together. In this tial for identifying, attracting and retaining effective model, cognitive conflict and board cohesiveness Board members (Shore, 2015). Good practices such are negatively related and board task performance as coaching, succession planning, support, devel- is reduced by too little or too much cohesiveness. opment and performance appraisals for members of Thus, the greater diversity of members’ occupational decision-making groups are often ad hoc and informal and educational background increases the knowl- despite recognition (but no evidence) of their impor- edge and skills available and cognitive conflict, but 8 simultaneously reduces the board’s cohesiveness with more negotiation than knowledge sharing and indi- and use of its knowledge and skills. Equally, the viduals evaluating their own knowledge highly (particu- finding that too little cohesiveness may also neg- larly if time is short); or discussion focuses on sharing atively impact on board task performance is im- knowledge, with time and good facilitation encourag- portant, highlighting the need for boards and their ing repetition and opportunities for validation of shared chairs to find a constructive way to manage dif- knowledge, to achieve mutual learning and better de- ference or conflict. A model for audit committees cisions. Much of the literature reviewed above (which (Mohiuddin and Karbhari, 2010) similarly notes that does not overlap with the literature used to develop the larger groups can encourage unnecessary debate model) provides supporting empirical evidence. and delayed decisions. A theory building review (Matta et al., 2016) re- Reviews addressing committee lated corporate boards’ roles, and highly technical contexts functions, to the mechanisms available to them: de- cision-making structures, formal processes, and Reviews drawing evidence from across the world raise communication approaches. Where the knowledge questions for readers about their relevance to specific brought by members is insufficient, which is likely in contexts. Two reviews specifically addressed develop- fast moving areas such as information technology, ing country contexts (George et al., 2015; Lodenstein they can co-opt or consult experts, collaborate with et al., 2017). Lodenstein et al. (2017) noted that social knowledgeable peers and improve information shar- accountability mechanisms, including health commit- ing within the organisation. tees, operate within cross-cutting power asymmetries By reviewing the relevant literature, Brown (2005) where professionals doubt the competency of lay investigated six dimensions of effective board per- members, and lay members may be chosen for their formance originally identified by primary research socio-economic or political status rather than knowl- of higher education boards with varying reputations edge, competence for making health decisions, or for effectiveness (Chait et al., 1991), and then tested interest in improving health services rather than their these with a survey of non-profit organisations. Higher personal access to services. The result is mecha- performing organisations were reported having nisms lacking legitimacy, and the interests of particu- high-performing boards across all dimensions. Howev- lar groups being overlooked either by their absence or er, it was the interpersonal dimension in particular (cre- by their reluctance to speak, or inability to influence. ating a sense of inclusiveness, setting goals for them- George et al. (2015, 159) offer a framework to help: selves, and grooming members for leadership) that provided a unique explanation of judgments of organi- delineate and organise… contextual factors into sational performance. Schoenberg et al. similarly found four overlapping spheres (community, health facil- that board performance was better where: Board- ities, health administration, society) with cross-cut- CEO relationships were collaborative and balanced ting issues (awareness, trust, benefits, resources, trust with distrust; and a board has ‘a sense of togeth- legal mandates, capacity-building, the role of polit- erness in achieving a long-term purpose’ (2016, 12); ical parties, non-governmental organisations, mar- and a positive boardroom climate allows open and kets, media, social movements and inequalities). informal discourse with ‘a level of constructive scepti- cism in a psychologically safe environment’ (2016, 15). A third review from Australia presented a frame- Minichilli et al. (2009) reviewed the literature about work that emphasised the contextual factors influenc- boards of directors (their tasks, characteristics and ing the performance of corporate boards: the type performance, and members’ background diversity, of organisation (e.g. public, commercial or voluntary commitment and critical debate), to develop a the- sector); the relevant legislative and societal frame- oretical model; testing the model with a subsequent works; the organisation’s constitution, history and survey of CEOs of the 2000 largest industrial com- strategic use of resources (Nicholson and Kiel, 2004). panies in Italy found that board members’ commit- Where studies do not offer explicit contextual ment, in particular, and cognitive conflicts and critical analyses, relevance of the evidence to particular con- debate were far more important for predicting board texts can be judged by: whether evidence has been task performance than was board demographics. generated in that context; and whether the phenom- The organisational and social psychology literature ena of interest are found in that context. For example, reviewed by Brodbeck et al. (2007) supported a model Table 1 presents the reviews offering evidence from whereby discussion either focuses on prior preferences, Australasia. 9 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures Table 1. Relevance of evidence to Australasia. Focus of evidence Studies from Australia Studies from New Zealand Guideline development Fretheim et al. (2006a) Fretheim et al. (2006a) Knai et al. (2012) Tan-Torres Edejer (2006) Consensus development Hutchings and Raine (2006) Hutchings and Raine (2006) Research agenda setting Oliver et al. (2004) Health and safety committees Yassi et al. (2013) Health boards Sajadi et al. (2013) Research ethics committees Schuppli and Fraser (2007) Schuppli and Fraser (2007) Corporate boards Jonsdottir (2010) Jonsdottir (2010) Schoenberg et al. (2016) Non-profit boards Shore (2015) Audit committees Mohiuddin and Karbhari (2010) Mohiuddin and Karbhari (2010) Velte (2017) Velte (2017) Bilal and Bushra (2018) Bilal and Bushra (2018) Discussion et al. (2011) considered Brodbeck et al.’s (2007) theories of information sharing and systemat- This review was commissioned to consciously draw ic processing applicable to the decision-making of upon a diverse literature and synthesise knowledge guideline development groups, especially when or- that could usefully inform the convening of committees ganisational culture encourages critical ‘norms that in an era of financial constraint. We found empiri- create an open, constructive atmosphere enabling cal evidence of groups with six to twelve members members to feel comfortable in airing their views’. tending to perform better than those either smaller However, they wondered whether the relatively large or larger. Groups with diverse backgrounds and spe- size of a group may hinder the processes. They also cialties take account of a range of opinions, with each noted the significance of group development and cit- member likely to advocate familiar options. Larger ed Wheelan and Kesselring’s (2005) stages of group groups reflect a broader range of outside interests, development: 1) group members looking to the lead- enhance credibility and widespread acceptance and er for direction; 2) the group develops norms, operat- implementation of decisions, but present difficulties ing procedures and goals (a stage characterised by for encouraging equal participation. However, if con- increased conflict); 3) increased trust and freedom flict is managed constructively, more varied member- to disagree and a consolidation of relationships; and ship leads to better performance and more reliable 4) high productivity and effectiveness. The time for a judgements. Corporate board leaders are important group to develop, socialise and negotiate norms was for establishing inclusive working procedures and an seen as relevant by a critical, but unsystematic, review atmosphere of openness, dialogue and trust. Chairs of the guideline development literature, combined with facilitate groups to generate more ideas through en- practical experience (Pagliari et al., 2001). Confirma- couraging members to express diverse opinions. tion came from a qualitative study of guideline devel- What is known about training is largely descriptive opment groups which found members valued oppor- (Ward and Preece, 2012; Shore, 2015) and reflective tunities to develop as a group, through the forming, (Gonski, 2015) rather than offering evidence of influ- storming, norming and performing stages described encing committee performance. Little is known about by Tuckman (Atkins et al., 2013). the impact of committees’ physical environment. All three contextual analyses (Nicholson and Kiel, How these factors influence committee perfor- 2004; George et al., 2015; Lodenstein et al., 2017) mance is explained by theoretical models. Hopthrow consider how the interactions between committee 10 members is influenced by the external experiences interest and willingness; offer time and commit- and the commitment they bring in terms of time or ment; actively participate; and behave appropriately motivation. They also all refer to influences of or- over external relationships, confidentially and con- ganisational structures, legal frameworks, and wider flicts of interest. social society, including politics and history. The rel- An important resource is the knowledge brought evance of the evidence to low and middle-income by individual members, which is unevenly distrib- countries is apparent not only from the contextual uted, or presented to them in committee papers or analyses available, but also from the growth of presentations. Educational and functional diversity formal partnerships between policymakers, stake- has given teams greater strategic clarity. In contrast, holders, and researchers, to enhance deliberative, demographic diversity has been seen as valuable in evidence-informed health policymaking (El-Jardali bringing different perspectives and a wider variety of et al., 2014). The relevance of the evidence to Austral- alternatives for consideration (Gonski, 2015) but does asia is evident not only from the studies conducted not necessarily improve committee performance there, but also from the structures established to sup- (Minichilli et al. 2009). Translating demographic diver- port committees with lay membership for developing sity into better committee performance may only be health guidance (National Health and Medical Re- possible when supported by holistic transformation of search Council, 2014), including The Best Practice human resource strategies (Alcázar et al., 2013). Advocacy Centre New Zealand (https://bpac.org.nz/ In addition to the knowledge and skills, the time guidelines/) which adapts NICE clinical guidance for available for a committee to explore that knowledge New Zealand, or for corporate boards (Australian In- to make choices or solve problems is important. Time stitute of Company Directors, undated; Financial Mar- for information processing during decision-making kets Authority, 2014). Our understanding of all these (left hand blue boxes, Fig. 3) allows more sharing of models combined is described briefly here and illus- knowledge; the more knowledge is shared during trated in Figure 3. discussion, the more it is subject to evaluation by Committee performance depends upon the indi- group members. When time is limited, less knowl- viduals involved (see top two blue boxes of Fig. 3), edge is shared and decisions are more the result of their attributes and relationships, specifically, negotiating between prior preferences, rather than members who: are aware of their tasks, roles and evaluation of shared knowledge. When tasks involve responsibilities; understand the wider context and judgements (rather than intellectual problem solving), culture; bring analytical and political competence, status within the group influences decisions. Figure 3: A model for effective and efficient committees, adapted from Brodbeck et al. (2007), and taking into account social and political influences. 11 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures With more time, greater facilitation skills to max- evaluated using the AGREE II instrument which em- imise sharing of knowledge, and greater mutual trust phasises the knowledge explicitly underpinning de- developed as committees mature and members cisions, criteria for its selection, how it is found, its get to know each other, more information about all strengths and limitations, and consideration of the options is revealed and available for evaluation. The ultimate implications of acting on this knowledge result is more sharing of ideas and individual learn- (Brouwers et al., 2010). Although AGREE II notes the ing, better quality decisions, more commitment to composition of the group, it asks little about group decisions by group members and wider acceptability interactions. of decisions within the group’s wider networks (right Drawing on multiple literatures allowed us to syn- hand blue boxes of Fig. 3). thesise research addressing different aspects of These small-scale interactions reflect the larger committee work and, simultaneously, draw on the scale institutional relationships, hierarchies and cul- methodological traditions dominating the literatures tures which interact as a backdrop to committee ac- of different disciplines: qualitative research address- tivities (outer triangle of purple boxes in Fig. 3) and ing clinical guidelines; experimental designs for clinical stakeholder engagement with decisions more widely guidelines and social psychology; and observational (Oliver et al., 2018). studies from business administration. Synthesising such a broad literature would not have been possi- Strengths and limitations ble without drawing on existing reviews and tolerating the different methodological expectations for reviews A key strength of this study was the iterative nature in different disciplines. Relevant reviews of empirical of its searching that revealed relevant bodies of liter- research were included where we could discern how ature addressing different contexts applying research authors had identified and selected studies, from clear conventions from different academic disciplines. The or implied eligibility criteria and search strategies. An scope and diversity of the literature mean that we additional criterion regarding appraisal of included cannot claim to have identified all relevant studies. studies (such as risk of bias) would have excluded This limitation is counteracted by our reliance on sys- learning from: Baltes et al. (2002) about computer- tematic reviews and theoretical syntheses drawing on mediated communication and group decision mak- extensive literatures. Evidence of ‘what works’ was ing; Boulkedid et al. (2011) about the Delphi Method drawn from systematic reviews of empirical studies of for Selecting Healthcare Quality Indicators; Jonsdottir guideline development, business administration com- (2010) about gender and corporate boards; Knai et al. mittees and social psychology. Their findings were (2012) about clinical guideline development for the confirmed and explained by theoretical syntheses management of chronic disease in Europe; Lundh et al. drawing on different sets of studies. (2013) about personal conflicts of interest and recom- Despite all this evidence and theoretical develop- mendations on medical interventions; Ramsay et al. ment, our understanding is limited because research (2010) on NHS Health Boards; Reimer et al. (2010) on has largely focused on the internal workings of com- discussion and decisions in small groups; Ward and mittees, and less on the socio-political history and Preece (2012) on housing sector boards; and Yassi context of their host organisations which will influence et al. (2013) on health and safety committees. how the committee works as much or more than the More fundamentally, reviews without a system- individual members. atic search strategy were included if they developed a framework, model or theory. Without these expla- Reviewing multidisciplinary literatures nations from the business administration literature, the following issues would have been lost: the impli- This systematic review found that evaluations of cations of committee size (Mohiuddin and Karbhari, health boards have emphasised the individuals in- 2010; Minichilli et al., 2009; Brown, 2005), committee volved, their attributes and relationships, specifically, composition (DeZoort et al., 2002; Minichilli et al., members who: are aware of their tasks, roles and 2009; Forbes and Milliken, 1999; Brown, 2005), struc- responsibilities; understand the wider context and tures and processes (DeZoort et al., 2002; Forbes culture; bring analytical and political competence, and Milliken, 1999; Brown, 2005), committee chairs interest and willingness; offer time and commitment; (Vandewaerde et al., 2010), timing (Brown, 2005), actively participate; and behave appropriately over decision-making in fast moving areas such as infor- external relationships, confidentiality and conflicts mation technology (Matta et al., 2016), and contex- of interest (Sajadi et al., 2013). This is very different tual influences of corporate boards (Nicholson and from how guideline development groups have been Kiel, 2004). The most useful model for explaining the 12 empirical evidence, from social and organisational to explore issues requiring judgements where psychology, would also have been excluded committee members vary in status. (Brodbeck et al., 2007). Only with this literature was the review able to report on the findings as a cohe- Implications for substantive research sive synthesis of evidence. The review also identified important gaps in the evi- Conclusions dence on committee effectiveness and efficiency that would benefit from new primary studies in commu- The findings of this review have the following practical nication science, ergonomics, social psychology and implications for the organisation and management of education. committees. First, primary studies have not addressed the impact of environmental factors (e.g. layout, décor, acoustics, lighting, heating, air conditioning, spatial Implications for Committees capacity) on committee performance. 1. Having members representing the full range of Second, while development activities are consid- stakeholders could bring the full range of relevant ered important for effective group functioning, they knowledge to discussions, although increasing are often poorly evaluated in current studies. Future the size of a committee above 12 members has monitoring or evaluation specifically of decision-mak- diminishing returns, particularly for groups work- ing processes should consider the quality of group ing virtually. decisions in terms of: the degree of consensus with- 2. Given that members’ views tend to favour in the group; the attitude of the group towards the their own specialist areas, and that good de- processes and the decisions; and the implications cisions arise from constructive conflict, effec- of decisions in terms of organisational performance tive chairs are more likely to be generalists with (governance, effective and efficient service, pub- good facilitation skills to help members share lic confidence). More evidence is required about the their knowledge; manage hierarchy and con- training of committee members. flict constructively; and develop an atmosphere Within many types of organisations, committees of inclusiveness, openness and trust. Particular are likely to remain the sine qua non for complex effort should be made to reveal knowledge ini- decision making. However, the results of our review tially held by individual, rather than all, members suggest there is still much to be learnt about how to especially if their status is not high. maximise their efficiency and effectiveness. 3. Time is required to allow knowledge brought to the meeting to be shared and evaluated before Implications for systematic review decisions are made. methods 4. Formal consensus methods are recommended, with guideline groups given the relevant technical Finally, this review also provides lessons for review- literature to inform their decisions. ing research on substantive issues that transcend 5. Distance working reduces the influence of indi- academic disciplines. An early challenge is ensuring viduals, but also opportunities for discussion. access to a broad scope of research. While open ac- Unfamiliar technology discourages participation, cess publishing is increasing, it is not yet sufficient to lowers quality of contributions and members’ sat- support reviews without institutional access to all rele- isfaction. Members more easily disengage when vant journals and libraries. Instead, access to reports feedback is delayed through asynchronous com- is provided by the institutional affiliations of multidisci- munication such as email or discussion boards. plinary teams. Accessing the reports’ content is also Working virtually may enhance committee per- easier with such teams, between them, being familiar formance by allowing more diverse membership. with a broader range of languages and traditions. However, it may present challenges to develop- As methodological approaches and standards vary, ing trust and cohesion (with sub-groups emerg- each discipline offers different empirical and theoret- ing,) and integrating divergent perspectives. ical contributions. Applying only review methods de- 6. Demographic diversity is valued for bringing dif- veloped in one discipline risks precluding the learning ferent perspectives and a wider variety of alter- that is available from others. In this case, applying natives for consideration. Educational and func- methodological criteria developed in health services tional diversity has given teams greater strategic research would have excluded empirical studies and clarity. More time and effort may be required scholarship from business administration, including 13 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures the theoretical development that provided a coherent practice in creating clinical guidelines. Journal of framework to understand the contributions of dispa- Health Services Research & Policy 4(4): 236–48. rate empirical studies. Thus, the appropriate methods Blackmore, P 1999. Mapping professional exper- for a systematic scholarship are determined by what tise: old tensions revisited. Teacher Development 3(1): can justifiably be learnt from the relevant literature. 19–38. Bonebright, DA 2010. 40 years of storming. Human Acknowledgements Development Resources International 13(1): 111–20. Boone, AL, Field, LC, Karpoff, JM and Raheja, CG The original report was commissioned in 2014 by the 2007. The determinants of corporate board size and Research and Development team of NICE as part of its composition: an empirical analysis. Journal of Financial Research Support Unit. We are grateful to the seven Economics 85(1): 66–101. stakeholders whose views helped inform our discussion Boyd, EA and Bero, LA 2006. Improving the use of of the literature. Many thanks also to the following for research evidence in guideline development: 4. Man- their extremely helpful comments on draft versions of aging conflicts of interests. Health Research Policy and this report: (i) Prof. Sarah Garner, Moni Choudhury, Systems 4(1): 16. Paul Levay and colleagues at NICE; and (ii) Professor Boulkedid, R, Abdoul, H, Loustau, M, Sibony, O and David Gough, Director of the Research Support Unit at Alberti, C 2011. Using and reporting the Delphi method the EPPI-Centre, UCL Institute of Education. for selecting healthcare quality indicators: a systematic review. PLoS ONE 6(6): e20476. Brodbeck, FC, Kerschreiter, R, Mojzisch, A and References Schulz-Hardt, S 2007. Group decision making under conditions of distributed knowledge: the information Acai, A, Sonnadara, RR and O’Neill, TA 2018. Get- asymmetries model. Academy of Management Review ting with the times: a narrative review of the literature 32(2): 459–79. on group decision making in virtual environments and Brouwers, M, Kho, ME, Browman, GP, Burgers, implications for promotions committees. Perspectives JS, Cluzeau, F, Feder, G, Fervers, B, Graham, in Medical Education 7: 147–55. ID, Grimshaw, J, Hanna, S, Littlejohns, P, Makarski, Alcázar, FM, Fernández, PMR and Gardey, GS 2013. J and Zitzelsberger, L 2010. Next Steps Consortium Workforce diversity in strategic human resource man- AGREE II: advancing guideline development, reporting agement models: a critical review of the literature and and evaluation in healthcare. Canadian Medical Asso- implications for future research. Cross Cultural Manage- ciation Journal 182: 839–42. ment: An International Journal 20(1): 39–49. Brown, WA 2005. Exploring the association be- Atkins, L, Smith, JA, Kelly, MP and Michie, S 2013. tween board and organizational performance in non- The process of developing evidence-based guidance in profit organizations. Nonprofit Management & Leader- medicine and public health: a qualitative study of views ship 15(3): 317–39. from the inside. Implementation Science 8(1): 101–13. Chait, RP, Holland, TP and Taylor, BE 1991. The Ef- Australian Institute of Company Directors. undated. fective Board of Trustees, Macmillan, New York. viewed 9 August 2018: https://aicd.companydirectors. Coles, J, Daniel, ND and Naveen, L 2008. Boards: com.au/~/media/cd2/resources/director-resources/di- Does one size fit all?. Journal of Financial Economics rector-tools/pdf/05446-5-4-mem-director-rob-board- 87(2): 329–56. committees_a4-web.ashx. Cornforth, C 2001. What makes boards effective? Baltes, BB, Dickson, MW, Sherman, MP, Bauer, CC An examination of the relationships between board and LaGanke, JS 2002. Computer-mediated commu- inputs, structures, processes and effectiveness in nication and group decision making: a meta-analysis. non-profit organizations. Corporate Governance: an Organizational Behavior and Human Decision Process- International Review 9(3): 217–27. es 87(1): 156–79. DeZoort, FT, Hermanson, DR, Archambeault, Bilal, SC and Bushra, K 2018. Audit committee DS and Reed, SA 2002. Audit committee effective- financial expertise and earnings quality: a meta-analysis. ness: a synthesis of the empirical audit committee liter- Journal of Business Research 84: 253–70. ature. Journal of accounting literature 21: 38–75. Black, N, Murphy, M, Lamping, D, McKee, M, Diamond, IR, Grant, RC, Feldman, BM, Pencharz, Sanderson, C, Askham, J and Marteau, T 1999. PB, Ling, SC, Moore, AM and Wales, PW 2014. De- Consensus development methods: a review of best fining consensus: a systematic review recommends 14 methodologic criteria for reporting of Delphi studies. Kelson, M, Akl, EA, Bastian, H, Cluzeau, F, Curtis, Journal of Clinical Epidemiology 67(4): 401–9. JR, Guyatt, G, Montori, VM, Oliver, S and Schünemann, El-Jardali, F, Lavis, J, Moat, K, Pantoja, T and HJ 2012. Integrating values and consumer involvement Ataya, N 2014. Capturing lessons learned from in guidelines with the patient at the center: article 8 in In- evidence-to-policy initiatives through structured reflec- tegrating and coordinating efforts in COPD guideline de- tion. Health Research Policy and Systems 12(2): 1–15. velopment. An official ATS/ERS workshop report. Pro- Financial Markets Authority. 2014. Corporate Gov- ceedings of the American Thoracic Society 9(5): 262–8. ernance in New Zealand: Principles and Guidelines, Kelly, JR and Loving, TJ 2004. Time pressure and Wellington, New Zealand Government Press. group performance: Exploring underlying processes Forbes, DP and Milliken, FJ 1999. Cognition and in the attentional focus model. Journal of experimental corporate governance: understanding boards of direc- social psychology 40(2): 185–198. tors as strategic decision-making groups. Academy of Khodyakov, D, Hempel, S, Rubenstein, L, Shekell, Management Review 24(3): 489–505. P, Foy, R, Salem-Schatz, S, and Dalal, S 2011. Con- Fretheim, A, Schünemann, HJ, and Oxman, ducting online expert panels: a feasibility and exper- AD 2006a. Improving the use of research evidence in imental replicability study. BMC Medical Research guideline development: 3. Group composition and con- Methodology 11(1): 174. sultation process. Health Research Policy and Systems Kötter, T, Schaefer, FA, Scherer, M and Blozik, E 4(15): 1–6. 2013. Involving patients in quality indicator develop- Fretheim, A, Schünemann, HJ and Oxman, ment – a systematic review. Patient Preference and AD 2006b. Improving the use of research evidence in Adherence 7: 259–68. guideline development: 5. Group processes. Health Knai, C, Brusamento, S, Legido-Quigley, H, Saliba, V, Research Policy and Systems 4(17): 1–4. Panteli, D, Turk, E, Car, J, McKee, M and Busse, R 2012. George, A, Scott, K, Garimella, S, Mondal, S, Ved, Systematic review of the methodological quality of clini- R and Sheikh, K 2015. Anchoring contextual analy- cal guideline development for the management of chronic sis in health policy and systems research: a narra- disease in Europe. Health Policy 107(2-3): 157–67. tive review of contextual factors influencing health Kunz, R, Fretheim, A, Cluzeau, F, Wilt, TJ, committees in low and middle income countries. Qaseem, A, Lelgemann, M, Kelson, M, Guyatt, G and Social Science & Medicine 133: 159–67. Schünemann, HJ 2012. Guideline group composition Gigone, D and Reid, H 1993. The common knowl- and group processes: article 3 in Integrating and co- edge effect: information sharing and group judgment. ordinating efforts in COPD guideline development. An Journal of Personality and social Psychology 65 (5): 959. official ATS/ERS workshop report. Proceedings of the Gonski, D 2015. I gave a Gonski, Hawtorne, Viking. American Thoracic Society 9(5): 229–33. Hopthrow, T, Feder, G and Michie, S 2011. The role Levy, G 2007. Decision making in committees: trans- of group decision making processes in the creation of parency, reputation and voting rules. The American clinical guidelines. International Review of Psychiatry Economic Review 97(1): 150–68. 23(4): 358–64. Linck, JS, Netter, JM and Yang, T 2008. The deter- Hutchings, A and Raine, R 2006. A systematic minants of board structure. Journal of Financial Eco- review of factors affecting the judgements produced by nomics 87(2): 308–28. formal consensus development methods in health care. Lodenstein, E, Dieleman, M, Gerretsen, B and Journal of health services research policy 11(3): 172–9. Broers, JE 2016. Health provider responsiveness to so- Jonsdottir, T 2010. The Impact of Gender Demography cial accountability initiatives in low- and middle-income on Male and Female Role Interpretations and Contribu- countries: a realist review. Health Policy and Planning tions: A Qualitative study of Non-Executive Directors of 32(1): 125–40. Icelandic Boards, Bedford: Cranfield University. Lundh, A, Jørgensen, AW and Bero, L 2013. As- Jünger, S, Payne, SA, Brine, J, Radbruch, L and sociation between personal conflicts of interest and Brearley, SG 2017. Guidance on Conducting and recommendations on medical interventions (Proto- REporting DElphi Studies (CREDES) in palliative care: col). Cochrane Database of Systematic Reviews 6(40): recommendations based on a methodological system- 1–24. doi: 10.1002/14651858.MR000040. atic review. Palliative Medicine 3(8): 684–706. McCoy, DC, Hall, JA and Ridge, M 2012. A system- Kagzi, M and Guha, M 2018. Board demographic atic review of the literature for evidence on health facil- diversity: a review of literature. Journal of Strategy and ity committees in low- and middle-income countries. Management 11(1): 33–51. Health Policy and Planning 27(6): 449–66. 15 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures Matta, M, Cavusoglu, H and Benbasat, I 2016. Un- include. Health Research Policy and Systems/ BioMed derstanding the board’s involvement in information Central 4(19): 1–7. technology governance: theory, review and research Oxman, AD, Schünemann, HJ and Fretheim, agenda. https://ssrn.com/abstract=2778811 or http:// A 2006c. Improving the use of research evidence dx.doi.org/10.2139/ssrn.2778811. in guideline development: 16. Evaluation. Health Minichilli, A, Zattoni, A and Zona, F 2009. Mak- Research Policy and Systems/BioMed Central 4(28): ing boards effective: an empirical examination of board 1–6. task performance. British Journal of Management 20(1): Pagliari, C, Grimshaw, J and Eccles, M 2001. The 55–74. potential influence of small group processes on guide- Mohiuddin, MD and Karbhari, Y 2010. Audit com- line development. Journal of Evaluation in Clinical Prac- mittee effectiveness: a critical literature review. AIUB tice 7(2): 165–73. Journal of Business and Economics 9(1): 97–125. Raine, R, Sanderson, C and Black, N 2005. Devel- Murphy, MK, Black, NA, Lamping, DL, McKee, oping clinical guidelines: a challenge to current meth- C, Sanderson, CFB, Askham, J and Marteau, T 1998. ods. British Medical Journal 331(7517): 631–3. Consensus development methods and their use in Ramsay, A, Fulop, N, Fresko, A and Rubenstein, S clinical guideline development. Health Technology As- 2010. The Healthy NHS Board: A Review of Guidance sessment 2(3): 1–88. and Research Evidence, Leeds: NHS National Leader- National Health and Medical Research Council. ship Council. 2014. Annual Report on Australian Clinical Practice Reimer, T, Reimer, A and Czienskowski, U 2010. Guidelines, National Health and Medical Research Decision-making groups attenuate the discussion bias Council, Canberra. in favor of shared information: a meta-analysis. Com- Nicholson, GJ and Kiel, GC 2004. A framework for munication Monographs 77(1): 121–42. diagnosing board effectiveness. Corporate Govern- Sajadi, HS, Malek, M, Ravaghi, H, Hadi, M, Hasan- ance: An International Review 12(4): 442–60. zadeh, H and Tahmasby, B 2013. The indicators of Oliver, S, Clarke-Jones, L, Rees, R, Milne, R., board evaluation in healthcare organizations: a review Buchanan, P, Gabbay, J, Gyte, G, Oakley, A and of evidence. Life Science Journal 10(8s): 92–8. Stein, K 2004. Involving consumers in research and Schünemann, HJ, Fretheim, A and Oxman, AD development agenda setting for the NHS: developing 2006a. Improving the use of research evidence in an evidence-based approach. Health Technology guideline development: 10. Integrating values and Assessment 8(15): 1–148. consumer involvement. Health Research Policy and Oliver, S, Roche, C, Stewart, R, Bangpan, M, Dickson, Systems/BioMed Central 4(22): 1–6. K, Pells, K, Cartwright, N, Gough, D and Hargreaves, D Schünemann, HJ, Oxman, AD and Fretheim, A forthcoming. Stakeholder engagement for development 2006b. Improving the use of research evidence in impact evaluation and evidence synthesis. Paper for the guideline development: 6. Determining which outcomes Centre for Excellence in Development Impact and Learn- are important. Health Research Policy and Systems/ ing, Department for International Development. BioMed Central 4(18): 1–6. Oseland, N and Burton, A 2012. Quantifying the Schoenberg, G, Cuskelly, G and Auld, C 2016. The impact of environmental conditions on worker per- role of intragroup dynamics in nonprofit governance formance for inputting to a business case to justify models: a systematic quantitative literature review. enhanced workplace design features. Journal of Build- Managing Sport and Leisure 2(11): 1–22. ing Survey, Appraisal & Valuation 1(2): 151–64. Schünemann, HJ, Wiercioch, W, Etxeandia, I, Falavig- Oxford English Dictionary. 2015. Board of Directors, na, M, Santesso, N, Mustafa, R, Ventresca, M, Brignardel- Oxford: Oxford University Press, Viewed 24 May 2015: lo-Petersen, R, Laisaar, KT, Kowalski, S, Baldeh, T, Zhang, http://www.oed.com/view/Entry/20731?result=46& Y, Raid, U, Neumann, I, Norris, SL, Thornton, J, Harbour, rskey=-oBjEYt&. R, Treweek, S, Guyatt, G, Alonso-Coello, P, Reinap, M, Oxman, AD, Schünemann, HJ and Fretheim, A Brozek, J, Oxman, A and Akl, EA 2014. Guidelines 2.0: 2006a. Improving the use of research evidence in systematic development of a comprehensive checklist for guideline development: 2. Priority setting. Health a successful guideline enterprise. Canadian Medical Asso- Research Policy and Systems 4(14): 1–6. ciation Journal 186(3): 123–42. Oxman, AD, Schünemann, HJ and Fretheim, A Schuppli, CA and Fraser, D 2007. Factors influencing 2006b. Improving the use of research evidence in the effectiveness of research ethics committees. Journal of guideline development: 7. Deciding what evidence to Medical Ethics 33(5): 294–301. 16 Shore, B 2015. A model of governance practices Upadhyay, A and Zeng, H 2014b. Gender and ethnic for a nonprofit organization’s board of directors: a diversity on boards and corporate information environ- systematic review and meta-ethnographic synthesis ment. Journal of Business Research 67(11): 2456–2463. (Order No. 10008885). ProQuest Dissertations & Vandewaerde, M, Voordeckers, W, Lambrechts, Theses Global. (1765446712), https://search.proquest. F and Bammens, Y 2010. The board of directors as a com/docview/1765446712?accountid=14511. team: investigating the influence of shared leadership on Stewart, R 2007. Expertise and Multi-Disciplinary board task performance. in Panka, E and Kwiatkowska, Training for Evidence-Informed Decision Making, Insti- A (Ed.), Proceedings of the 6th European Conference on tute of Education, University of London, London. Management, Leadership, and Governance, Academic Tan-Torres Edejer, T 2006. Improving the use Conferences Limited, Reading: 1–455. of research evidence in guideline development: 11. Velte, P 2017. The link between audit committees, cor- Incorporating considerations of cost-effectiveness, porate governance quality and firm performance: a litera- affordability and resource implications. Health ture review. Corporate Ownership & Control 14(4): 15–31. Research Policy and Systems 4(23): 1–6. Ward, C and Preece, D 2012. Board member Turner, ME and Pratkanis, AR 1998. Twenty-five development in the public, voluntary and social hous- years of groupthink theory and research: lessons from ing sectors. European Journal of Training and Develop- the evaluation of a theory. Organizational Behavior and ment 13(6): 630–45. Human Decision Processes 73(2-3): 105–15. Wheelan, SA and Kesselring, J 2005. Link between Tversky, A and Kahneman, D 1974. Judgement un- faculty group: Development and elementary student der uncertainty: heuristics and biases. Science 185 performance on standardized tests. The journal of ed- (4157): 1124–31. ucational research 98(6): 323–330. Upadhyay, AD, Bhargava, R and Faircloth, Yassi, A, Lockhart, K, Sykes, M, Buck, B, Stime, B SD 2014a. Board structure and role of monitoring and Spiegel, JM 2013. Effectiveness of joint health and committees. Journal of Business Research 67(7): safety committees: a realist review. American Journal 1486–1492. of Industrial Medicine 56(4): 424–38. 17 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures Appendix Table A1. Characteristics of reviews synthesising empirical evidence systematically. First author Substantive Included Search Type of Findings for (year) literature designs strategy synthesis questions Reviews about or to inform clinical guideline development Murphy et al. (1998) Guideline development Any Systematic Thematic All except training Black et al. (1999) Group dynamics/ summaries facilitation Psychology Baltes et al. (2002) Group dynamics/ Comparison Systematic Statistical Communication facilitation groups meta-analysis media Psychology Timing Hutchings and Guideline development Controlled Systematic Thematic Committee Raine (2006) studies summaries composition Communication media Structures/processes Oxman et al. Guideline development Systematic Systematic Thematic Communication (2006a) #2 reviews summaries medium Structures/ processes Fretheim et al. Guideline development Systematic Systematic Thematic Committee size (2006a) #3 reviews summaries Committee composition Committee chair Structures/processes Equity Boyd and Bero Guideline development Systematic Systematic Thematic Structures/processes (2006) #4 reviews summaries Fretheim et al. Guideline development Systematic Systematic Thematic Committee chair (2006b) #5 reviews summaries Communication medium Structures/processes Physical environment Schünemann Guideline development Systematic Systematic Thematic Committee et al. (2006b) #6 reviews summaries composition Structures/processes Oxman et al. Guideline development Systematic Systematic Thematic Structures/processes (2006b) #7 reviews summaries Schünemann Guideline development Systematic Systematic Thematic Committee et al. (2006a) #10 reviews summaries composition Communication medium Structures/processes Tan-Torres Edejer Guideline development Systematic Systematic Thematic Committee (2006) #11 reviews summaries composition Structures/processes Oxman et al. Guideline development Systematic Systematic Thematic Structures/processes (2006c) #16 reviews summaries 18 Boulkedid et al. Guideline development Evaluation Systematic Descriptive Committee size (2011) statistics Committee composition Communication media Structures/processes Kelson et al. (2012) Guideline development Systematic Systematic Thematic Committee reviews summaries composition Equity Knai et al. (2012) Guideline development Appraisal Systematic Thematic Committee of clinical summaries composition guidelines Communication media Kunz et al. (2012) Guideline development Systematic Systematic Thematic Committee size reviews summaries Committee composition Committee chair Communication medium Structures/processes Equity Schünemann Guideline development Guidelines Systematic Thematic Committee size et al. (2014) for guideline summaries Committee development composition Structures/processes Jünger et al. (2017) Guideline development Qualitative Systematic Thematic Structures/processes and summaries quantitative Oliver et al. (2004) Patient and public Any Systematic Framework Communication involvement synthesis media Timing Structures/processes Equity Other health sector reviews Lundh et al. (2013) Drug, device or Any Systematic Statistical Structures/ medical imaging meta-analysis processes (authors’ products recommendations) Kötter et al. (2013) Research committee Any Systematic Thematic Committee (patient and public summaries composition involvement) Communication media George et al. Health committees in Empirical Systematic Developing a Committee context (2015) LMICs studies framework Acai et al. (2018) Promotions Empirical Systematic Thematic Communication committees in studies summaries medium undergraduate and postgraduate medicine. Reviews about business administration Jonsdottir (2010) Business management Any Systematic Thematic Committee summaries composition Committee chair Structure/processes Equity 19 ffectiveandefficientcommitteework:asystematicoverviewofmultidisciplinaryliteratures Ramsay et al. Business management Any Systematic Thematic Committee size (2010) (health boards) summaries Committee composition Committee chair Structures/ processesEquity Yassi et al. (2013) Business management Empirical Systematic Realist Committee size (workplace safety) data review Committee composition Timing Structures/processes Training McCoy et al. (2012) Business management Any Systematic Framework Committee development composition Equity Oseland and Ergonomics and Experimental Limited Statistical Physical environment Burton (2012) design design systematic meta-analysis search Ward and Preece Business management Any Systematic Thematic Training (2012) (housing) summaries Sajadi et al. (2013) Business management Any Systematic Thematic Committee (health boards) summaries composition Developing a Committee chair framework Structures/ processes Shore (2015) Non-governmental Studies Systematic Meta- Committee Organisation Boards of U.S. ethnography composition non-profit Structures/processes organizations Schoenberg et al. Non-profit Boards Empirical Systematic Developing a Structures/processes (2016) studies framework Velte (2017) Business Empirical Systematic Thematic Committee administration studies summaries composition Bilal and Bushra Business Studies Systematic Statistical Committee (2018) administration that report meta-analysis composition t-statistics, p-values, z-scores, and chi-square statistics. Reviews of social psychology Reimer et al. (2010) Psychology Comparison Systematic Statistical Timing groups meta-analysis Structures/processes Diamond et al. Social psychology Delphi Systematic Thematic Structures and (2014) studies summaries processes 20 Table A2. Characteristics of reviews presenting frameworks, models or theories. First author Substantive Included Search Type of Findings for (year) literature designs strategy synthesis questions Forbes and Business Unspecified Unspecified Developing a model Committee composition Milliken (1999) management Structures/processes Group dynamics DeZoort et al. Business Unspecified Unspecified Developing a model Committee composition (2002) management Structures/processes Nicholson and Corporate Unspecified Unspecified Developing a model Committee context Kiel (2004) Boards Brown (2005) Business Unspecified Unspecified Developing a model Committee size management Committee composition Timing Structures/processes Brodbeck et al. Social and Unspecified Unspecified Developing a model Structures/processes (2007) organisational psychology Minichilli et al. Business Unspecified Unspecified Developing a model Committee size (2009) management Committee composition Mohiuddin and Business Unspecified Unspecified Developing a model Committee size Karbhari (2010) management Vandewaerde Business Unspecified Unspecified Developing a model Committee chair et al. (2010) management Sajadi et al. Business Unspecified Systematic Thematic summaries Committee composition (2013) management Developing a Committee chair (health boards) framework Structures/processes Matta et al. (2016) Corporate Unspecified Unspecified Developing a Committee composition Boards framework Schoenberg et al. Non-profit Empirical Systematic Developing a Structures/processes (2016) Boards studies framework Lodenstein et al. Social Unspecified Systematic Realist review Committee context (2017) accountability 21