Advancing internal audit analytics

Advancing internal audit analytics

• 1.
  Turning the corner Advancingthe use of analytics within internal audit July 2016
• 2.
  2 Turning thecorner Advancing the use of analytics within internal audit As businesses build and adapt for a more complex global marketplace, the expectations placed on internal audit are increasing. Data analytics can fundamentally change the way internal audit responds to those expectations and can lead to efficiency, richer business insights, and enhanced monitoring. To realize those benefits, internal audit has to move beyond merely using analytics to automate fieldwork and fully integrate analytics capability throughout the audit life cycle. The ultimate goal is to have an internal audit function composed of critical thinkers who are enabled by analytics in every aspect of their work.
• 3.
  3 Rising expectations ofanalytics During the past decade, internal audit has gained a strong appreciation for the potential that data analytics offers. In fact, three years ago, in PwC’s 2013 State of the Internal Audit Profession Survey, the vast majority of chief audit executives (CAEs) and internal audit directors told us they understood the importance of data analytics for quantifying issues, strengthening audit coverage, and gaining a better understanding of risks. Many organizations decided to invest in building their internal audit analytics capabilities—but experienced varying degrees of success. i Institute of Internal Auditors, Pulse of Internal Audit 2016: Time to move out of the comfort zone. Some have reaped significant benefits from their investments, but others have found their programs stalling, and they’re struggling to incorporate analytics throughout their audits and departments. Across the profession, interest in analytics continues to remain high (Figure 1). In the Institute of Internal Auditors’ Pulse of Internal Audit 2016 report, 37% of respondents indicated that data mining and analytics skills are very essential or extremely essential to their internal audit function’s ability to perform its responsibilitiesi. Figure 1: The importance and use of internal audit analytics continues to increase
• 4.
  4 Turning thecorner Advancing the use of analytics within internal audit “ Data analytics provide new and better options for internal audit, such as allowing for testing greater populations. While investments in these tools and technologies can be significant for an internal audit organization, the returns are also expected to be significant in terms of risk management, productivity and auditor job satisfaction. - James S. Hunt, Audit Committee Chair for Brown & Brown, Inc. and PENN Mutual life Insurance Co. Inc. Pressure from stakeholders is also driving the need for successful analytics programs. With the changing business and system landscape and increased regulations, stakeholders’ expectations of audit are expanding while the complexity of the business increases. PwC’s 2016 State of the Internal Audit Profession Survey found that 62% of stakeholders expect more from internal audit, including almost half of those already receiving significant valueii. Stakeholders want internal audit to provide deeper insights and root cause investigations and to partner with the business with a view to create sustainable value. In such an environment, analytics can help internal audit generate greater value and allow a company to respond better to risks by looking at wider populations and sifting out important issues from less critical ones. ii 2016 State of the Internal Audit Profession Study. The integration of analytics can also strengthen auditors’ skills and business acumen, providing auditors with more-rewarding experiences, which in turn leads to increased talent appeal in the marketplace. Whether beginning, redirecting, or advancing an internal audit analytics program, the importance and benefits of fully incorporating analytics into the audit life cycle, the methodologies, and the function as a whole are clear. This paper explores how internal audit can avoid common pitfalls in the analytics journey and more rapidly achieve the sustained value leadership anticipates.
• 5.
  5 Avoiding the pitfalls Asinternal audit builds or revisits its data analytics strategy, PwC’s experience shows that there are six common points of breakdown that can ultimately hold internal audit back from realizing the analytics promise (Figure 2). Figure 2: Focusing on these areas can help restart a stalled analytics program
• 6.
  6 Turning thecorner Advancing the use of analytics within internal audit Pitfall #1: Embarking on the journey without a defined strategy First and foremost, a documented strategy that is tied to a value proposition is a precursor to success. Many functions begin without a fully defined data analytics strategy and simply dive in by asking resources to “make it happen.” Some lead the effort with a technology implementation, but many analytics initiatives are not accompanied by a comprehensive organizational plan. Advancing the use of analytics requires a shift in thinking. Simply acquiring a tool or gaining access to technology will not lead to success unless accompanied by the integration of a supporting infrastructure and comprehensive capabilities across the internal audit function. The absence of a strategy may also prevent data analytics from being seen as a serious initiative that warrants funding and attention from the business. A strategy that documents the vision and roadmap with a time horizon of 18 to 24 months is necessary to ensure internal audit has in place a comprehensive data analytics goal and an execution plan so that internal audit can build what is required for success. The roadmap prioritizes initiatives and sets an achievable timeline to meet specific milestones that will help ensure successful execution of the strategic plan. One large manufacturer started down an analytics path without a strategy, and as a result the department lost focus on what it had intended to do. The auditors failed to progress along the route set by their intended roadmap, resulting in regular intervals of stopping and then starting over the course of two years. They lost key team members and struggled to gain significant traction in the organization, which led to the perception that analytics was a hobby, as opposed to a strategic initiative. Those events ultimately stalled efforts, and the company had to go back to basics by defining the strategy and clearly outlining its short- and long-term objectives by reconsidering all of the factors necessary to succeed. • Establish realistic milestones • Combine knowledge of audit, data, enterprise resource planning, and business processes and operations • Build a team of resources with the right combination of skills • Develop capabilities and reusable analytics across the business • Choose the right tools and technology • Demonstrate continuous quick wins • Establish a communication plan, and assess it against success criteria • Incorporate analytics throughout the audit life cycle • Coordinate with other lines of defense (e.g. business, compliance) and relevant departments (e.g. information technology) • Define and assign analytics governance procedures and other program-level procedures (e.g. analytics-candidate selection, build methodology) Guiding principles for a successful strategy What to do Whether internal audit is just starting to build its data analytics strategy or is in the midst of implementation, perform a strategy review to ensure that the strategy is inclusive of the plan for the people, the data, the technologies, the capabilities, the communication, the business value, and the methodology.
• 7.
  7 “ Working withour IT department was initially challenging, but once we had a framework for collaboration in place they helped accelerate the incorporation of analytics as part of a continuous monitoring process. Multiple company departments have benefited from the analytics the IA department created as we were able to identify fraud and develop an on going monitoring dashboard. - Rene Herrera, Sr. IT Audit Manager, Office Depot Pitfall #2: Going it alone If internal audit’s data analytics strategy does not include the leveraging of connection points (such as with information technology [IT], compliance, or operations), then internal audit will fail to benefit from the lessons learned from others in the organization. By making those connections across the organization, internal audit can learn where others are in the data analytics process, can benefit from their knowledge, and can potentially take advantage of resources, technologies, and processes that already exist. One internal audit function took eight months to get a technology license in place and to get an IT environment operational. If early in the strategy execution internal audit had used connection points and communicated with the appropriate resources, the function could have accelerated the process or piggybacked on another initiative. A different internal audit department had been using analytics for two years but consistently found itself dealing with a high rate of false-positives. Because its analytics effort was being led in isolation, the team didn’t realize that the company had statisticians accessible who could have helped refine the analytics so as to reduce false-positives through more-advanced analysis techniques. Making sure that connection points are getting identified is a particularly important issue if a company has a strong compliance arm (which is often the case in highly regulated industries such as pharmaceuticals or financial services) because multiple risk assurance and compliance functions may be in the midst of an analytics journey. Communicating with the wider organization can also aid in acquiring necessary data from across the business, which for most companies is a common challenge. Many firms are investing in big data technology to meet this challenge and internal audit should leverage these investment. Furthermore, identifying departments with similar goals can help gain organizational buy-in and lead to potential cost savings or increased cost-effectiveness. And establishing up front how internal audit will collaborate with others such as IT will serve to smooth and expedite collaboration on individual audits. What to do Circulate the data analytics strategy with senior leadership to make known what internal audit intends to do. Identify opportunities for collaboration, and incorporate feedback. Seek input from others in the organization who regularly apply data analytics, and then adjust the plan based on their experience.
• 8.
  8 Turning thecorner Advancing the use of analytics within internal audit Pitfall #3: Underestimating the power of organizational culture One of the greatest obstacles to a successful data analytics implementation can be corporate culture. By managing the process only as complementary to what has been done before rather than as a fundamental change, people and organizational change management sometimes get overlooked and then present a significant challenge down the line. If it isn’t easy for employees to embrace a strategy, the strategy can easily fail. Upon the implementation of analytics, there will be new roles and modified job descriptions; and for most resources, the ways they do their work will change. Internal audit must consider how every aspect of the function could be affected by analytics—from resource scheduling to assessing risk—and how to create an environment in which to execute the analytics strategy. The objective is to encourage and increase adoption by helping people feel that the use of analytics is the new normal and an expected part of every audit. By considering the influence of corporate culture on the strategy to be implemented, internal audit’s data analytics strategy is far more likely to gain acceptance among staff. For example, a company that places high priority on performance reviews may want to build the use of analytics into its goal setting or the employee performance review process. For an internal audit department of team members with lengthy tenures or, conversely, members with only limited experience, new approaches may require more incentive for change. One strategy can be to empower junior team members to encourage the rest of the team to adapt to the change. In all cases, finding ways to motivate resources in the department’s specific environment is essential. The wider culture of an organization will also affect the internal audit model. Awareness of potential cost constraints or sensitive business data can dictate the success of the model selected. In some instances, outsourcing or offshoring may not be feasible. What to do Consider the impact of your unique corporate culture on your data analytics strategy and roadmap. Then find ways to motivate employees to change, and create a safe environment for learning, wherein people feel empowered and unafraid to ask questions or try something different.
• 9.
  9 Pitfall #4: Fallinginto the automation trap Many internal audit functions embark on analytics by incorporating them into fieldwork to automate testing. If fieldwork is the only area where analytics get deployed, internal audit is simply automating the work it has always done; it is not transforming the audit or the core of how internal audit responds to risk. And though such limited application might generate some level of efficiency, it will fall short of capitalizing on the more robust risk assessment, the richer insights, and the data-driven discussions that are possible when analytics are applied to their full potential. PwC’s 2015 State of the Internal Audit Profession Survey showed that just 48% of CAEs reported that they use analytics for scoping decisions and only 43% leveraged data to drive their risk assessmentiii. Internal audit should look comprehensively at the pieces of the audit process to see where analytics can be applied, such as finding answers to questions in the following areas. Risk assessment: How does data help identify where risks reside in the organization? How can analytics help determine where to focus internal audit’s efforts? How can data be applied to evolve into continuous risk assessment? Can external data such as external risk scores, reputation issues, industry trends, or hot spots enrich the information available to internal audit? Audit planning: How can data profiling become part of audit planning so as to help the audit team understand where it should spend time or to determine whether auditing is needed at all? Where can broad process-level key risk indicators or key performance indicators be used to attain the right breadth and depth of coverage? Fieldwork: How can data be applied to enable the field team to react and go deeper into places the team might not otherwise have gone—or to pull back the amount of time it spends? This application of data requires audit team flexibility to both expand and contract relative to initial plans. Reporting: How can data become part of the reporting cycle to better demonstrate coverage, quantify risk, and then communicate findings, recommendations, and value? Continuous monitoring and auditing: How can we create an environment that allows continuous auditing based on the continuous monitoring of key events? iii 2015 State of the Internal Audit Profession Study. What to do As part of the roll-out strategy, identify a select number of audits to pilot using analytics throughout the audit life cycle from audit planning to reporting. This demonstrates to the team how analytics change the dynamics of the audit as well as their role. Success then helps to build momentum for more widespread end-to-end adoption. One global financial services company incorporated analytics into its risk assessment to identify major focus areas for initial walkthrough and control assessment. When the company applied that technique during the audit of a global product control function, the analytics- driven approach reduced the timeline by at least 60% and resulted in significant cost savings.
• 10.
  10 Turning thecorner Advancing the use of analytics within internal audit Pitfall #5: Seeing analytics as just a bolt-on to existing audit procedures In the same way that analytics should be incorporated throughout the scope of internal audit’s work, the impact of analytics must be considered throughout internal audit’s methodology. Analytics should be transformative to the audit process, not additive. For instance, if there are typically 10 steps in an internal audit methodology, the addition of analytics should not result in 10 traditional steps and, say, five analytics steps; instead, the existing steps should get altered to account for the analytics. When embedded into the audit methodology, analytics render the process and the auditors more efficient. Consider all of the decision points throughout the internal audit life cycle. When analytics is embedded in the methodology those decisions become automated rather than onerous manual processes. This allows internal audit to generate far more insights into business risks and paves the way for the implementation of continuous monitoring of controls to protect the business. Integrating analytics into the full audit process requires a review and a modification of the end-to-end audit methodology (Figure 3). Figure 3: Fully incorporating analytics into the internal audit methodology transforms and accelerates the audit process 02 • Project-level risk assessment • Audit pre-planning • Audit scoping and planning • Risk attribute sampling 01 • Enterprise risk management • Annual internal audit risk assessment • Risk monitoring • Business unit or site-level profiling 04 • Audit and executive reports • Issues dashboard • Compliance metrics • Analytics validation review 03 • Multi-unit auditing • Data-driven testing • 100% coverage • Process, control, and results validation • Root cause identification Embedded and sustainable analytics
• 11.
  11 For instance, considerthe following examples of areas where methodology may have to be updated before the use of analytics. Preplanning: A preplanning process may have to be introduced in advance of the implementation of standard audit planning activities. Preplanning enables both timely data acquisition and the development of the analytics to be used during the audit. Preplanning can have broad impacts on resource scheduling, auditee notification and involvement, and so on. Result validation: Using analytics enables internal audit to test 100% of the population. That level of coverage can generate a high volume of results from each test. The methodology associated with the analytics development process may need a refinement step to identify ways of reducing false-positives. In addition, internal audit might have to develop a method to document false- positives or provide a framework and criteria for auditors to use in deciding which results require follow-up actions. All of this becomes part of internal audit’s documented policies and procedures so that the consideration and application of the results get incorporated into the audit methodology and do not get viewed separately from the existing process. Documentation and review: If analytics are being created using programming languages—such as SQL, ACL, Python, or R—and visualization technologies, staff and managers must validate that the analytics are operating correctly. And yet many team members won’t have the necessary knowledge required to review and assess the technical accuracy of the analytic scripts. Therefore, alternative review procedures may have to be established to make sure there is an appropriate review. This may mean implementing a peer review, or having a designated data analytics manager perform the technical review, or training auditors to conduct data-level walk-throughs. In all cases, new procedures are warranted. What to do Revisit your end-to-end audit methodology to make sure that appropriate revisions have been made to integrate analytics efficiently and effectively such that analytics accelerate the overall process, not bog it down.
• 12.
  12 Turning thecorner Advancing the use of analytics within internal audit Pitfall #6: Isolating the analytics team The entire internal audit team—whether the members possess technical data skills or not—has to have a level of analytics acumen for building critical-thinking skills with regard to how, when, and where to use data to solve problems. A common mistake often occurs when internal audit is at the stage of determining the operating model for its analytics program and decides to keep its technical and data resources together in a siloed team. A siloed structure makes it difficult for technical analytics resources to engage with the broader internal audit department and raise the analytics IQ of everyone in the function. A combination of skills is required, including technical or data expertise, business process familiarity, and internal audit background. Typically, it is not common for all of those skills to reside within one resource, though that’s changing as analytics becomes more prevalent and colleges change their curricula. For that reason, it’s usually necessary to compensate for one resource’s lack of combined skills by teaming up several resources together rather than having them operate in silos (Figure 4). A suggested operating model consists of internal audit’s establishment of a center of excellence that has analytics specialists, or data artists (those who can translate the data into a “story”), working in conjunction with analysts staffed on audit teams. One global financial services company found that creating a three-tiered resource model within internal audit itself helps the company effectively blend a deep understanding of data together with a deep understanding of the business. The tiers consisted of a group of power users with deep analytics skills, a group of data analytics champions on each business and technology audit team, and a group of business auditors who, though not as technically skilled, are becoming more and more comfortable with applying analytic toolsiv. iv 2015 State of the Internal Audit Profession Study. Figure 4: The internal audit team should have a combination of skills with a shared analytics mindset
• 13.
  As we sawin the prior pitfall, successful analytics are embedded across the audit life cycle, and as a result, some level of analytics skills should extend into all internal audit team members. Internal audit should develop a training curriculum that enables the full team to think more analytically about their audits. Ultimately the goal is to help the staff become better auditors (not technologists) by approaching audits as problems with a critical thinking mindset and asking the right questions from the outset. The use of data to support analytics is then a natural extension of this foundation. What to do Include in the strategic plan the training and development plan for all staff, whether data analytics specialists or field auditors. Focus on critical thinking and problem solving in addition to technical skills. Raise the analytics IQ of everyone on the team. • Operating a model of a center of excellence versus core capability throughout the audit function • Developing skill sets through training, internal recruiting, external recruiting, or cosourcing with a provider • Training of the broader internal audit team to develop an analytics mind-set and understanding of analytics methodology • Roles and responsibilities during development and during audit procedures Key considerations
• 14.
  14 Turning thecorner Advancing the use of analytics within internal audit Finding the path forward Many internal audit leaders hit pitfalls in their analytics journeys regardless of where they are on the path to advancing their analytics programs. Even though each internal audit function’s approach to analytics should tailor its analytics use based on the company’s perception of risk, the company’s audit mandate, and other company and industry-specific factors, every approach should contain a strategic plan that defines how internal audit will execute the transformation. As internal audit leaders consider the level of success that ensues from their analytics initiative, we encourage them to recheck their strategies, keeping all of the pitfalls in mind. Avoiding pitfalls will accelerate the achievement of analytics objectives, thus enabling internal audit to excel at using data to:  Identify audits based on risk  Dictate the areas that require additional auditing  Understand the why and the how behind the what so as to deliver valued insight  Separate important issues from less critical ones If the benefits expected from data analytics are not materializing, it could be time to go back to basics: make sure there is a documented strategy that is taking advantage of lessons learned by others in the organization and that contains an operating model, steps for organizational change, specific methodologies, and a roadmap for expanding analytics beyond fieldwork. By following those steps, internal audit can accelerate its progress and begin to see the benefits anticipated.
• 15.
  © 2016 PwC.All rights reserved. PwC refers to the US member firm or one of its subsidiaries or affiliates, and may sometimes refer to the PwC network. Each member firm is a separate legal entity. Please see www.pwc.com/structure for further details. Contact us To have a deeper conversation about how to start or refresh your internal audit analytics initiative, please contact: Jason Pett Partner (410) 659-3380 jason.pett@pwc.com Seth Rosensweig Principal (646) 471-6762 seth.rosensweig@pwc.com Shane Foley Principal (646) 471-0516 shane.p.foley@pwc.com Robyn Conlon Director (973) 222-2084 robyn.conlon@pwc.com