(PDF) From process improvement to learning organisations

SOFTWARE PROCESS IMPROVEMENT AND PRACTICE Softw. Process Improve. Pract. 2006; 11: 239–249 Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/spip.267 From Compliance to Business Success: Improving Outsourcing Service Controls by Adopting External Practice Section Regulatory Requirements ´ Biro´ 1 *† , Csilla De´ak2 , J´anos Ivanyos2 and Miklos Richard Messnarz3 1 Corvinus University of Budapest, Veres P´aln´e u. 36. H-1053 Budapest, Hungary 2 MEMOLUX, Th¨ok¨oly ut 137, H-1146 Budapest, Hungary 3 ISCN, Florence House, 1 Florence Villas, Bray Co. Wicklow, Ireland The new generation of general models that refer either to IT or Internal Controls, like COBIT or COSO, are presented with an executive management perspective. Practice shows that this opening is solely not enough to reach a breakthrough, since models became so complicated that they could only be applied with difficulties. The best catalysts of improvement programs are the mandatory rules being issued, mainly from the financial reporting area. The Sarbanes–Oxley Act (SOX) for US SEC registrants and its affiliates, and the 8th Directive on company Law in the EU require strict internal controls for reporting processes. In this article we concentrate on the successful application of these rules in a situation where IT-enabled services have a major effect on the compliance of the user organization. We investigate the effects of a high maturity level on compliance for both the service and the user organizations. The article refers to the applicability of the well-known capability models CMM and eSCM, and some other sources like COSO, BSC, and SAS 70. For presenting implementation practices of the general risk–based control model via key control processes, effectiveness measurement and innovative technologies were used, including the knowledge management platform created in earlier software process improvement experiments. Copyright  2006 John Wiley & Sons, Ltd. KEY WORDS: process improvement; business objectives; control frameworks; independent audit; IT-enabled service; outsourcing; risk analysis; knowledge management 1. INTRODUCTION ∗ Correspondence to: Miklos ´ Biro, ´ Corvinus University of Software process improvement models and prac- Budapest, Veres P´aln´e u. 36. H-1053 Budapest, Hungary tices have become more and more accepted and † E-mail:

incorporated not only by the international stan- Copyright  2006 John Wiley & Sons, Ltd. dards like ISO 15504 (ISO/IEC 15504) and ISO Practice Section M. Biro´ et al. 9001 series (1994), but also by the more business 2. EXPERIENCES WITH SPI AT MEMOLUX management-oriented control frameworks such as COBIT (Information Systems Audit and Control Memolux, established in 1989, is a Hungarian pri- Foundation, IT Governance Institute), the open vate SME company with professional experience as standard of Control Objectives for Information and a service provider in finance and public accoun- related Technology. COBIT Management Guidelines tancy, management organization, software devel- opment and information system engineering. In provide the tools to help IT managers improve Hungary, Memolux is ranked after the ‘Big Four’, IT performance and link the IT objectives to busi- the four greater advisory firms in public accoun- ness objectives, which consist of Maturity Models, tancy. Memolux is a member of several economic critical success factors (CSFs), key goal indicators chambers (AMCHAM, BCHH, CCCH) and profes- (KGIs) and key performance indicators (KPIs). This sional organizations (IIA, EOQ). Its payroll and concept delivers a significantly improved frame- accounting service lines are represented among the work, responding to the management’s need for biggest ones provided by an independent Hungar- control and measurability of IT by providing the ian SME, with about 150 clients in the Hungarian management with the tools to assess and measure market. their organization’s IT environment against 34 IT Memolux has been implementing software pro- processes identified by COBIT. In addition, to help cess improvement practice for more than 15 years. focus on performance management, the principles The ICT department achieved maturity level of the balanced business scorecard (Kaplan and 3 of Bootstrap methodology (www.bootstrap- Norton 1992) were used. institute.com) and ISO 9001 certification in 1998 and The new generation of general models that refer has been successfully participating in EU research to either IT or Internal Controls, like COBIT or projects. Memolux was the prime user and contrac- COSO (The Committee of Sponsoring Organiza- tor of the PASS project, which was the first Central tions of the Treadway Commission (COSO)), are and Eastern European ESSI Process Improvement extended with a business perspective of having Experiment (FP4 PIE) project directly supported by the objective to attract attention of the top man- the European Commission (Biro´ et al. 2000). Mem- agement. However, experience has shown that this olux was a co-developer of the Media-information opening solely is not enough to achieve a break- sans Fronti´eres (NQA-based teamwork) system and through since models became so complicated that was the technical coordinator of the Media-ISF Best Practice (FP5 IST Take-up) project (Baksa et al. 2002). they could only be applied with difficulty. The The company built its success on the accounting best catalysts of the improvement programs are and payroll outsourcing needs of Hungarian and mainly the mandatory rules being implemented foreign start-up companies following the social nowadays in the financial reporting area. The Sar- and economic transformation in the 1990s. The full banes–Oxley Act (SOX) for the US SEC (Security time professional staff, the nimble organization, the and Exchange Commission) registrants and its affil- innovative culture, and their strong IT foundation iates, and the 8th Directive on company Law in the enabled Memolux to maintain a stable growth and EU, require strict internal controls and the executive to adapt quickly to changing market requirements. management has to draw conclusions on effective- Owing to a conscious and consistent integration ness. of business and technology development efforts Compliance and maturity issues have begun to in the company’s strategy, Memolux was able to be viewed by the management as the huge cost build and maintain a competitive advantage in its of compliance-readiness projects draws attention to markets. the sustainability and the added business value of such efforts. 3. LESSONS LEARNT FROM USING In this article the authors discuss their experi- CAPABILITY MODELS ments at a Hungarian SME where both IT-enabled outsourcing services and software and business pro- One of the major criticisms of ISO 9000:1994 was cess improvement projects were run for more than that its introduction became a burden because of the 15 years. overwhelming ISO bureaucracy, which was only Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 240 DOI: 10.1002/spip Practice Section Improving Outsourcing Service Controls meant to control the production and was not ready • Verification and accounting of the transferred to adapt to the permanent change of processes, knowledge resources are critical issues for the technology, and customer demands. sustainability of IT-enabled outsourcing. These business issues were highly relevant in cen- On the basis of the lessons learnt, Memolux is tral and eastern Europe at the end of the 1990s since focusing on how its process improvement skills an efficient use of all resources became increasingly and the adapted technology that has been devel- critical. Hungary played, in general, a major role oped for many years can be utilized in professional in the involvement of central and eastern Euro- outsourcing services, especially in managing con- pean companies in software process improvement trol activities that support the internal and external initiatives and the creation of channels for their pre- compliance requirements. sentation (Biro´ et al. 1998), as well as contributed to the global understanding of business motivations for software process improvement (Biro´ and Tully 4. EXTERNAL REGULATORY 1999, Biro´ and Messnarz 2000). The publication of REQUIREMENTS AND CUSTOMIZING the basic concepts of SPI and the business moti- INTERNAL CONTROL MODELS FOR vations in the form of book chapters accessible in COMPLIANCE Hungarian language was a major milestone as well External legal regulations regarding control of (Biro´ 1998, 1999). business processes such as SOX and the new 8th A further issue that is highly relevant in emerging Directive on public financial reporting drew our countries is the consideration of the differences attention towards knowledge management support in cultural value systems when introducing new and technologies for internal controls. The related management processes. This issue is discussed in external assurance requirements in the United the context of SPI in (Biro´ et al. 2002). In another States, EU and its Member States are summarized article on ‘Stages of software process improvement in the discussion article (F´ed´eration des Experts based on 10 year case studies’ (Biro´ et al. 2004), Comptables Europ´eens (FEE) 2005) issued in March the authors describe how the above mentioned 2005 by the FEE – the representative body of the global processes drove Hungarian companies to European accounting profession. orient their further process improvement initiatives The US SOX of 2002 provides for new corporate towards their business needs. Memolux, whose governance rules, regulations, and standards for core business is providing payroll and accounting specified public companies, including SEC regis- services, introduced and published the eSourcing trants. The US Securities and ExSEC have mandated Capability Maturity Model for IT-enabled Service the use of a recognized internal control frame- Providers (escm ) (Hyder et al. 2002). work. The SEC in its final rules regarding the SOX There were four main lessons learnt (The Institute made specific reference to the recommendations of of Internal Auditors (The IIA)) from using the The Committee of Sponsoring Organizations of the escm model to assess the Memolux outsourcing Treadway Commission (COSO). capability, which were as follows: By the definition of the COSO framework, inter- nal control is a process effected by an entity’s board of directors, management, and other personnel, and • High capability level (3) practices cannot be designed to provide a reasonable assurance regard- achieved without the support of external process ing the achievement of objectives. By adopting improvement. the models and experiments of software process • The high capability levels cannot be kept up improvement, Memolux has developed a usable without running a knowledge management knowledge management platform for supporting system. the design, implementation, and measurement of • Practices of the escm framework are well adapt- internal controls. For the customizing process we able to any virtual organization model as the used the public resources of the IIA (The Insti- high capability level outsourcing cooperation of tute of Internal Auditors (The IIA)) and ISACA service clients and providers implements a real (Information Systems Audit and Control Associa- knowledge-based virtual organization. tion (ISACA)) web sites. Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 DOI: 10.1002/spip 241 Practice Section M. Biro´ et al. The IT Governance Institute recently published This model does not indicate that the whole an article on ‘IT Control Objectives for Sar- organization and the key process areas should be at banes–Oxley (The importance of IT in the design, high maturity levels to achieve successful business implementation and sustainability of internal con- results. It refers only to the return of investment trols over disclosure and financial reporting)’ (Infor- regarding the compliance-readiness efforts of the mation Systems Audit and Control Foundation, IT specific external regulation. Governance Institute 2004) in order to advice on the From the viewpoints of sustainability and added satisfactory compliance and sustainability require- business value, the following level 5 practices imply ments through implementing maturity practices that the internal resources are used effectively and defined by the COBIT framework. efficiently: This article provides a direction for IT profession- als to meet the challenges of the SOX. Compliance • An enterprise-wide control and risk manage- is not a stand-alone process as it must be integrated ment program exists such that controls and within the overall business-led compliance process. procedures are well documented and contin- However, even these should be based on business uously re-evaluated to reflect major processes or (financial reporting) requirements, signed off by the organizational changes. business houses, and not left to the IT provider. • A self-assessment process is used to evaluate the This is especially true when IT is outsourced. For design and effectiveness of controls. IT application controls, the business, not IT, should • Technology is leveraged to its fullest extent define the control requirements, especially for finan- to document processes, control objectives and cial systems that are often complex in nature from a activities, identify gaps, and evaluate the effec- business process perspective. tiveness of controls. When an organization uses external service orga- With this conclusion for the discussion so far, the nizations to perform outsourced services, these rest of the article concentrates on risk management, services still remain a part of the former’s over- effectiveness measurement, and related technology all operations and responsibility, and need to be issues. considered in the overall internal control program. Organizations should review the activities of the service organization in arriving at a conclusion on 5. IMPLEMENTING ENTERPRISE RISK the reliability of its internal control. Documentation MANAGEMENT BY USING KEY CONTROL of the service organization’s control activities will PROCESSES be required for attestation by the independent audi- tor, so an assessment of the service organization is The COSO has issued an enterprise risk management required to determine the sufficiency and appro- (ERM) – Integrated Framework (COSO 2004), as well priateness of evidence supporting these controls. as a detailed practical application guide in 2004. Traditionally, audit opinions commonly known as Designed to offer organizations a commonly Statement on Auditing Standards (SAS) 70 reports accepted model for evaluating risk management (American Institute of Certified Public Accountants efforts, the framework expands on internal control (AICPA)) have been performed for service organi- concepts by providing a more robust focus based on zations. the broader subject of ERM. Detailing the essential The IT Governance Institute provides the matu- components of an effective ERM process, the rity profiles for internal control design and effective- framework provides guidance to help organizations ness models, demonstrating the stages of control build effective programs for identifying, measuring, reliability that may exist within organizations. prioritizing and responding to risk. For the purpose of establishing internal control, Embedded within an organization’s strategies some organizations may be willing to accept and objectives, ERM’s value is maximized when controls that fall short of stage 3. However, given the a balance is reached among growth, returns, risks, SOX’s requirement for an independent attestation uncertainties and opportunities. How much risk of controls by external audit, controls will more the entity is prepared to accept is inherent in than likely require the attributes and characteristics the capabilities of the ERM, which encompass the of stage 3 or higher for key control activities. following key components: Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 242 DOI: 10.1002/spip Practice Section Improving Outsourcing Service Controls • Aligning risk appetite and strategy. that the key control is attempting to achieve. In fact, • Enhancing risk response decisions. when properly formulated, minimum standards are • Reducing operational surprises and losses. derived on the basis of the control failure risks that • Identifying and managing multiple and cross- the key control is attempting to prevent. enterprise risks. The implementation of key control process con- • Seizing opportunities. sists of the following steps: • Improving utilization of capital. • Customizing the generic (e.g. financial) control In addition, the new framework presents a standard objectives to be specific to the organization type, definition of risk and ERM and provides a direction size, etc. to enhance risk management, including criteria for • Reviewing and documenting the ‘performance’ use by companies to determine whether or not their reporting processes. risk management is effective, and if not, what is • Searching for missing controls. needed. • Testing identified controls. Considering activities at all levels of the organi- – Developing ‘minimum standards of control’. zation, the ERM framework views entity objectives – Reviewing key controls and control excep- at the entity, division, business-unit, and subsidiary tions. levels, in four key categories: strategic, operations, – Verifying against the established minimum reporting, and compliance. At the same time, the standards of control. framework focuses on eight interrelated compo- Figure 2 presents an example of a key control nents: internal environment, objective setting, event process of financial reporting in an EU-funded identification, risk assessment, risk response, con- multi-partner (co-sourcing) project experienced by trol activities, information and communication, and Memolux: monitoring. Customized control objectives for financial Key controls (Vorhies 2004) are those significant reporting: controls within the business processes, which, if operating correctly, will both ensure that the 1. Authorization – Financial reports are appropri- organization is achieving its key business objectives ately authorized by project partner/financial and assure us of the same. We use key controls manager. concept to simplify the implementation process of 2. Accuracy – Reported costs are actual, economi- COSO ERM as presented in Figure 1: cal, and necessary for the implementation of the Minimum standards of control should ensure that project. the key control is completed in a way that achieves 3. Valuation – Reported costs are determined in our control objectives in a complete, timely, and accordance with the usual accounting principles accurate manner. Therefore, minimum standards of the contractor. of control are actually related to control objectives 4. Completeness – All requested (periodic/final) reporting forms and evidences are provided. 5. Classification – Cost model instructions for direct and indirect costs are appropriately used. 6. Existence – There is proper justification for the resources deployed by each contractor, linking them to activities implemented. 7. Timeliness/cutoff – All reported costs are incur- red during the project/period. 8. Safeguard assets – All the original documenta- tion that are likely to be examined by the auditors are available. 9. Segregation of duties – External audit certificate is provided. Figure 1. Enterprise risk management and key control Example of minimum standards of control for process financial reporting: Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 DOI: 10.1002/spip 243 Practice Section M. Biro´ et al. Figure 2. Key control process of financial reporting in an EU-funded co-sourcing project (Certified European Project Manager HU/B/03/F/PP-170028 Project) 1. Financial reports are submitted by all the project tested (i.e. a common scope and objectives built partners in time (scheduled min. 2 weeks before around identified business risks). official deadlines). • Identify the key controls, including both their 2. Actual templates, formats, and calculations are manual and automated elements. used (e.g. provided by teamwork tool). • Document, assess, and test only those identified 3. Consistency with the actual periodic/final key controls. activity report and with the reported deviations from project plan ensured. The conclusion drawn from the discussion of risk 4. Access to original documentation provided. and opportunity management is that key controls 5. Submission of reports, change requests, external have to focus on those events in which the reviews, and approval by the project manager related business objectives are really measurable. are documented (e.g. via teamwork submis- Therefore, we use the balanced scorecard (Kaplan sion tool). and Norton 1992) model for presenting a possible effectiveness measurement. This example of a key control process has relevance for the practices in many types of virtual organiza- tions and co-sourcing cooperation/service controls 6. MEASURING CONTROL as well. EFFECTIVENESS BY USING KEY CONTROL Integrated audit principles regarding co-sourcing & SAS 70 BALANCED SCORECARDS models have the following considerations: • Understand the business risks/opportunities The balanced scorecard concept was created by over which the controls are to be assessed and Robert S. Kaplan and David P. Norton in 1992 on Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 244 DOI: 10.1002/spip Practice Section Improving Outsourcing Service Controls the basis of the simple premise that ‘measurement Table 1. Key control scorecard motivates.’ Today it is being utilized by thousands Key control scorecard of corporations, organizations, and government agencies worldwide. User orientation Corporate contribution The balanced scorecard allows organizations to How do the users view the How does management view Key Control process? the Key Control process? rapidly and effectively implement strategy by inte- grating measurements with the management sys- Mission Mission To meet compliance To obtain a reasonable tem. It allows one to assess a detailed set of requirements and to improve business contribution from objectives and activities on an ongoing basis, as well user satisfaction KC process as to measure links between incentive compensa- Objectives Objectives tion and individual performance. An organization • Independent audit • Control of expenses for should build its specific key control scorecard on the performance compliance basis of the four elements of the framework: Finan- • User satisfaction • Maximum effect on the business perspective cial/Corporate Orientation, User/Customer Orien- tation, Operational Excellence, and Future/Growth Measures Measures • Acceptance rate of audit • Actual versus budgeted Orientation. A generic key control scorecard applied results expenses/efforts by Memolux is shown in Table 1. • Score on user satisfaction • Actual versus planned By focusing on measuring internal control effec- survey income tiveness of co-sourcing partnership, we investigated • Increase of business value the applicability of SAS 70 audit procedures as key Operational excellence Future orientation control processes. SAS No. 70 (American Institute of How effective is the Key Is organization positioned to Certified Public Accountants (AICPA)) is an inter- Control process? meet future Key Control nationally recognized auditing standard developed challenges? by the AICPA. A SAS 70 audit or service audi- Mission Mission tor’s examination is widely recognized because it To ensure and provide To develop opportunities that represents that a service organization has been put assurance that the answer future challenges organization is achieving its through an in-depth audit of their control activities, related key business objectives which generally include controls over information Objectives Objectives technology and related processes. In today’s global • Mitigate high impact risks • Skilled and motivated staff economy, service organizations or service providers • Leverage high impact • Applicable innovative must demonstrate that they have adequate controls opportunities technologies and safeguards when they host or process the data • Apply and develop • Process improvement that belong to their customers. In addition, the minimum standard requirements in Section 404 of the SOX of 2002 Measures Measures • Achieved versus intended • Skills assessment make SAS 70 audit reports even more important value of impact and • Timely identification and for the process of reporting on effective internal probability (efficiency) analysis of technological controls at service organizations. • Actual versus planned opportunities SAS No. 70 is the authoritative guidance standard frequency of verification • Maturity-based evaluation that allows service organizations to disclose their • Control failures/deficiencies (capability assessment) • Preventive and corrective control activities and processes to their customers actions and customers’ auditors in a uniform reporting format. A SAS 70 examination signifies that a service organization has had its control objectives obtains services from another organization (service and control activities examined by an independent organization). Service organizations that provide accounting and auditing firm. A formal report such services could be application service providers, including the auditor’s opinion (Service Auditor’s bank trust departments, claims processing centres, Report) is issued to the service organization at the Internet data centres, or other data processing conclusion of a SAS 70 examination. service bureaus. SAS No. 70 is generally applicable when an In Table 2, we provide a balanced scorecard of auditor (user auditor) is auditing the financial a sample SAS 70 audit applicable for IT-enabled statements of an entity (user organization) that outsourcing service organizations. Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 DOI: 10.1002/spip 245 Practice Section M. Biro´ et al. Table 2. SAS 70 scorecard costs and provide a higher confidence on the service SAS 70 scorecard organization. From the service organization’s viewpoint, the User orientation Corporate contribution SAS 70 audit, just as any other type of quality How do the users view the How does management view certification process, can be handled as a key control audit process? the audit process? process incorporating relevant aspects to measure Mission Mission the achievement of its specific business objectives. To meet audit requirements of To obtain a reasonable users and to improve user business contribution from satisfaction audit process Objectives Objectives 7. APPLICABLE SPI TECHNOLOGIES • Audit performance • Control of expenses for audit SUPPORTING CONTROL EVALUATION • User satisfaction • Maximum effect on the business perspective In January 2005, The IIA Research Foundation Measures Measures published a survey of the work by ‘Sarbanes–Oxley • Acceptance rate of SAS 70 • Added value for the user Section 404 – Looking at the Benefits’ (The IIA audit results organizations Research Foundation 2005). The survey identifies • Score on user satisfaction • Positive effect on the service survey management control improvements that have taken place as a • Actual versus budgeted direct result of SOX evaluations and the lessons expenses learned that could improve the efficiency and effectiveness of control evaluations in the future. Operational excellence Future orientation Regarding the enhanced documentation and How effective is the audit Is organization positioned to process? meet future audit challenges? control evidence, there are two components of improved documentation that were mentioned by Mission Mission Effective audit process Develop opportunities to the respondents in the survey: answer future challenges • Documentation of the processes, workflow, and Objectives Objectives controls. • Improvement of audit • SAS 70 audit training and process education of service personnel • Documentation of the evidence that the controls • Efficient account audit and user contact persons are working on. • Efficient audit result • Monitoring audit presentation requirement revisions Improvement in the documentation of controls • Efficient management of • Internal and external and processes is not surprising because it was audit findings benchmarking research mandated by regulation and auditing standards. In Measures Measures completing the readiness effort, organizations have • Audit maturity level • Educational budget as better implemented not only the process flow and • Number of successful percentage of total audit associated controls, but also updated the associated account audits budget • Rate of accepted audit • Percentage of service staff policies, procedures, handbooks, job descriptions, reports and user contact persons and other pertinent documents. • Number of failures to involved in training and Respondents believed that the development of manage nonconformities in education activities adequate documentation would pay future divi- time • Percentage of budget spent dends in areas such as training new employees, on audit requirement revision monitoring enabling backfill and succession planning for key • Number of successful positions, and identifying process improvement renewal projects initiated by opportunities. Many respondents mentioned that research team the improved documentation is an important con- trol from a global control perspective. A major finding is that there was little documen- The provision of an SAS 70 audit report is tation or evidence to show that existing controls not meant as a key control for the user organi- were working. For example, how would an orga- zation as it can be implemented in the Service nization determine that there was a proper review Level Management or Third party Management of an exception report, or a proper reconciliation, if processes. However, it can reduce the total audit there was no documentation that the review or the Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 246 DOI: 10.1002/spip Practice Section Improving Outsourcing Service Controls reconciliation was performed? Respondents noted an improvement after documenting the evidence of supervisory reviews and approvals, management committee actions and decisions, and the investiga- tion and resolution of un-reconciled or outstanding items. The need to properly and clearly develop evidence of the operation of each key control has been established and has become a more common practice. These issues addressed by the survey are strongly connected to the conclusions about compliance and maturity presented in Chapter 4 of this article. The key control scorecard of the future orientation measures shown in Table 1 reflects these experiments. Here we identify some applicable Figure 3. Process assessment technologies that resulted from earlier software process improvement developments, which were also practiced by Memolux: driving force for prioritizing improvements to the processes. The NQA teamwork (Messnarz et al. 2001) environ- Process capability determination is concerned ment. A combination of methodology, technology, with analysing the proposed capability of selected and social skills to run project administration, processes against a target process capability quality management, and internal control pro- profile in order to identify the risks involved cesses as a teamwork over the Internet. in undertaking a project using the selected NQA is highly configurable and adaptable. Com- processes. The proposed process capability may panies can define their own project administra- be based on the results of relevant previous tion structure, and by configuring the scripts process assessments, or on an assessment carried the user interface can be adapted. Companies out for the purpose of establishing the capability can use their documentation (so far developed) of the proposed process. by inserting their documentation guidelines in a ISO/IEC 15504-2 (2003) defines a reference model template pool. of processes and process capability that forms Teamwork is highly emphasized by the underly- the basis for any model to be used for the ing methodology (role-based work flow models) purposes of process assessment. The reference and the assignment of team members to roles, model comprises a two-dimensional approach and the structuring of workflows are done by the to the evaluation of process capability – one quality administrator through a menu system. dimension defines the processes to be assessed, NQA is pre-configured for scenarios supporting the other describes the scale for the measure- international guidelines and standards. ment of capability. Any model(s) compatible The process assessment portal. Within a process with the reference model may be used for improvement context, process assessment pro- assessment, and the results of any conformant vides the means of characterizing the current assessment can be translated into a common practice within an organizational unit in terms of base. the capability of the selected processes. Analysis Each process in the reference model is described of the results in the light of the organization’s by a statement of the purpose of the process, business needs identifies strengths, weaknesses, which includes an outline of the intended out- and risks inherent in the processes. This, in turn, comes of process implementation. leads to the ability to determine whether the The skills assessment portal (Biro´ and Messnarz processes are effective in achieving their goals, 2004). The skills assessment portal is a skills por- and to identify significant causes of poor quality tal that is configured with the skills card and or overruns in time or cost. These provide the supports the steps of browsing the required skills, Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 DOI: 10.1002/spip 247 Practice Section M. Biro´ et al. self-assessment, formal assessment, evidence col- REFERENCES lection, generation of skills profiles, and learning recommendations. American Institute of Certified Public Accountants (AICPA). 1992. Statement on Auditing Standards (SAS) By integrating the Capability Adviser Process No. 70, service organizations. Solutions and Capability Adviser Skill Card Solu- tions with the web-based training portal Moodle Baksa C, Ivanyos J, Ziaja Z, Messnarz R. 2002. Measuring and with the NQA Teamwork Portal System, the e-business effectiveness: adaption of process driven approach in web-publishing. EuroSPI 2002, Nurenberg, result will be a system available for assessment, Germany, September 18th–20th 2002. learning, and joint development of knowledge and training. Biro´ M. 1998. The quality of software products and software processes. In Chapter in the book entitled for Businessmen about Readiness, Szilv´assy E Dr (ed.). Qualika: Budapest, Hungary, ISBN 96304 95171. (in Hungarian). 8. CONCLUSIONS Biro´ M. 1999. Software Quality. In Chapter in the Outsourcing and co-sourcing partnerships have a book entitled Quality Management and Informatics, Toth ´ T major effect on the implementation of ERM-based (ed.). Technical Publishing House∼Muszaki konyvkiad ¨ o: ´ control systems. By using the maturity practices Budapest, Hungary, ISBN 96316 30471. (in Hungarian). of the COBIT Management Guidelines, we pre- Biro´ M, Messnarz R. 2000. Key Success Factors for Busi- sented the key control concept as a sustainable ness Based Improvement, Vol. 2, No. 2. Software quality interpretation of risk and opportunity manage- professional, ASQ∼American Society for Quality: 20–31. ment in order to measure the effectiveness of (http://www.asq.org/pub/sqp/past/vol2 issue2/biro. the related business objectives. The innovation in html). achieving software process improvement skills and Biro´ M, Messnarz R. 2004. ManagEUr – success criteria the experiments performed by Memolux from the for EU project management. In EuroSPI’2004 Industrial beginning of the 1990s resulted in an appropriate Proceedings, Messnarz R, Christiansen M, Konig S (eds.). capability to provide knowledge-based skills and Norvegian Technical University: I1-A.15–I1-A.22, ISSN- the technology to support the design, implemen- NO 1503-416X. tation, and measurement of internal controls for Biro´ M, Tully C. 1999. The software process in the context outsourcing and co-sourcing projects. Having been of business goals and performance. In Chapter in the a member of both the software process improve- book entitled Better Software Practice for Business Benefit, ment and business process outsourcing communi- Messnarz R, Tully C (eds.). IEEE Computer Society Press: ties for many years, Memolux is developing a new Washington, DC; Brussels, Belgium; Tokyo, Japan, ISBN 0-7695-0049-8. business service that is applicable for internal con- trol of outsourcing, and co-sourcing activities as Biro´ M, Ivanyos J, Messnarz R. 2000. Pioneering Process well. Improvement Experiment in Hungary, Software Process: This new service will be set-up by implement- Improvement and Practice, Vol. 5, no. 4. John Wiley & Sons: ing the same knowledge management platform 213–229. (http://www3.interscience.wiley.com/cgi- bin/abstract/76503384/START). that is already used for software quality processes. The uniqueness of the proposed service is that, Biro´ M, Messnarz R, Davison AG. 2002. The Impact of being assessable by independent auditors, it sup- National Cultural Factors on the Effectiveness of Pro- ports adequate internal control processes of both cess Improvement Methods: The Third Dimension, Vol. 4, the service providing organization and the user No. 4. Software Quality Professional, ASQ∼American Society for Quality: 34–41. September 2002, organization. (http://www.asq.org/pub/sqp/past/vol4 issue4/biro. For further extension of the experiments pre- html). sented in this article, the authors are considering setting up of an ISO/IEC 15504 conformation pro- Biro´ M, Balla K, Ivanyos J, Messnarz R. 2004. Stages cess reference model and measurement framework of software process improvement based on 10 year case studies. In EuroSPI’2004 Industrial Proceedings, that is applicable to the internal audit community Messnarz R, Christiansen M, Konig S (eds.). Norvegian assessing the control effectiveness of outsourcing Technical University: I2-B.7–I2-B.18, ISSN-NO 1503- and co-sourcing business processes. 416X. Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 248 DOI: 10.1002/spip Practice Section Improving Outsourcing Service Controls Biro´ M (moderator), Gorski J, Stoyan YuG, Loyko MV, ISO 9001. 1994. Quality systems. Model for quality Novozhilova MV, Socol I, Bichir D, Vajde Horvat R, Roz- assurance in design, development, production, man I, Gyork¨ os ¨ J. 1998. Software process improvement installation and servicing; ISO 9001: 2000 Quality in central and Eastern Europe. Software Process Newslet- management systems. Requirements. ter, no.12. IEEE Computer Society Press, Spring: 19–21, http://members.iif.hu/birom/spn no12.pdf/. ISO/IEC 15504. 2003. Information technology – process assessment. Certified European Project Manager HU/B/03/F/PP- 170028 Project. 2003–2006. Carried out with the financial ISO/IEC 15504-2. 2003. Information technology – process support of the commission of the European communities assessment – Part 2: performing an assessment. under the Leonardo da Vinci Programme. Kaplan RS, Norton DP. 1992. The balanced scorecard: Committee of Sponsoring Organizations of the measures that drive performance. Treadway Commission (COSO). 2004. Enterprise risk management – integrated framework. Messnarz R, Nadasi G, O’Leary E, Foley B. 2001. Experience with teamwork in distributed work F´ed´eration des Experts Comptables Europ´eens (FEE). environments. EuroSPI 2001, Limerick, Ireland. 2005. Risk management and internal control in the EU, March 2005. Haase W, Messnarz R, Koch G, Kugler HJ, Decrinis P, BOOTSTRAP: Fine tuning process assessment, In IEEE Hyder EB, Kumar B, Mahendra V, Sieges J, Heston KM, Software, July 1994. www.bootstrap-institute.com. Gupta R, Mahaboob H, Subramanian P. 2002. eSourcing Capability Model (escm) for IT-enabled Service Providers The Committee of Sponsoring Organizations of v1.1, Technical Report no. CMU-CS-02-155, School the Treadway Commission (COSO). 1992. Internal of Computer Science, Carnegie Mellon University, control – integrated framework. Pittsburgh, PA 15213-3890, 21 October 2002. The IIA Research Foundation. 2005. Sarbanes–Oxley Information Systems Audit and Control Association section 404 work – looking at the benefits, January 2005. (ISACA). 2005. www.isaca.org. The Institute of Internal Auditors (The IIA). 2005. Information Systems Audit and Control Foundation, IT www.theiia.org. Governance Institute. 2000. COBIT – control objectives for information and related technology. Vorhies JB. 2004. Key controls: the solution for Sarbanes-Oxley internal control compliance. 2004, Information Systems Audit and Control Foundation, The Institute of Internal Auditors Research Founda- IT Governance Institute. 2004. IT control objectives for tion, http://www.theiia.org/bookstore.cfm?fuseaction= Sarbanes–Oxley – the importance of IT in the design, product detail&order num=489/. implementation and sustainability of internal control over disclosure and financial reporting. Copyright  2006 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., 2006; 11: 239–249 DOI: 10.1002/spip 249