Risk analysis in long term audit planning and audit preparation - part II

You can find part I of this article here.

Phase 2 - Analysis at the level of the area of responsibility (auditable area)

Internal audit’s responsibility for the proper application of risk based analysis in the preparation of its audit activities does not end with the multi-year audit planning or its actualisation. Each area of responsibility which has been selected for audit needs to go through a risk based analysis at the operational level as part of the specific audit planning phase.
This second phase assumes there is no structural risk management application present which outputs could be used as input for audit preparation purposes.

Executing the operational risk analysis

We survey all collaborators (“responsibles, accountable and consulted” in the context of the RACI matrix) within the selected auditable area. We use the 80 statements developed in the Risk Identification Model. Note this is not simply executing the initial entity wide risk analysis again. The scope of the assessment is both more narrow and deeper, as it only covers the auditable area but covers it in-depth. In addition, not all collaborators were (necessarily) involved in the execution of the entity wide risk analysis which was used for purposes of multi-year internal audit planning. Remember, the decision to involve them was at the discretion of the person accountable for the auditable area.

These collaborators are asked to judge each of the statements as to relevance and current risk exposure. Remember that risk exposure is a function of their assessment of impact, likelihood of occurrence and current level of risk management and leads to the development of a risk control matrix.

Risk analysis results

The risk analysis results, correctly represented in a risk control matrix, are but one input in the preparation of the audit approach. Of course internal audit remains independent in its assessment and can call on other information to complement the risk analysis. However, note that this, both from the point of view of the IIA’s standards and the effort of the organisation should be considered as a major specific audit planning input. What’s also interesting is that these results allow internal audit to make a comparison with the initial assessment of middle management.

The results also provide us with more information as to which aspects of the auditable area are considered to be important by the collaborators intimately involved with the process. In essence, it’s an appreciation of sorts of their understanding and knowledge of the problems they can be confronted with.

Perhaps counterintuitively, our priority audit areas are not those areas of high risk and low control. We want to ensure that these are subject to a risk mitigation or monitoring plan which is in the process of being implemented. These risks are actually the responsibility of the risk management function, if it exists. When auditee and auditor agree on the existence of a problem and the auditor has gathered enough audit evidence to confirm the existence and the scope of the problem, his audit activities need to end. He can then write this up in his audit report.

Internal audit’s responsibility is to provide assurance. Hence, we look at those areas which are considered to be high risk but under control. We assure the board, the audit committee and management that, based on our assessment, these risks are in effect under control.

Building the audit workprogram

Those areas which are considered to be those high risk high control areas constitute, together with the appreciation of the auditor based on prior experience, the basis for the development of the core of the audit workprogram. Note that the build of the audit workprogram consists of a significant change in the approach from risk to process.

During the risk analysis the auditor focuses on risks. Risks and their appreciation by collaborators in the process are central to the approach. However, once the auditor starts the development of the audit workprogram, all risks with an influence in an auditable area are gathered and covered in the audit workprogram. The process becomes the central aspect and the entire audit workprogram is structured according to the processes covered in the auditable area.

This has a very logical but sometimes difficult to accept consequence for auditors. If a certain process within an auditable area is not linked to one or more risks, if no indication other than the analysis exists that there are risks related to that process, it should no longer necessarily be covered by an audit activity.

Executing the audit

After the audit workprogram has been developed and validated by the CAE, the actual audit can start. The auditors execute all activities planned and described in the audit workprogram. Note that multiple risks can be evaluated at the same time, depending on the results from the audit activities which are executed. In case for example accuracy and completeness of a transaction are to be evaluated, running a test-batch of information through the process can be a test functional for both objectives.

Results

In addition to the standard audit dispositions which need to be reached at the end of an audit activity in the audit workprogram this approach allows us to assess the understanding of risks and relevance of the current risk management measures. Especially those situations where there is a significant discrepancy between the assessment of the accountable people and the assessment of the responsible people. This may be an indication of deeper underlying issues.

Risk analysis in long term audit planning and audit preparation - Part I

The IIA standards require us to develop a risk based internal audit planning. There is however little material available on how organisations actually perform this. Organisations with good risk management systems can provide information from these approaches or systems to internal audit. However, if there are no risk management systems available, you will need to do a lot of the work yourself. And even when these systems are present, that does not necessarily mean that you will be able to easily repurpose the output for internal audit planning.

We’ve developed a two phase approach. In the first phase we execute the analysis at the level of the entire organisation. In the second phase we prepare the specific audit at the level of the auditable entity or activity (process, subprocess).

Today, I will cover phase 1, the organisation-wide analysis. A next post will cover the analysis at the level of the auditable entity. Let’s kick off phase 1.

Phase 1 - Organisation-wide analysis

Analysis coverage

Any activity within the audit universe needs to be subject to the risk analysis. If your organisation includes decentralized entities, these need to be included as well. We query all people accountable for an activity. You need to read accountable as it’s meant in the RACI matrix. For us, this is middle management, including all local representatives of our organisation. However, each accountable can ask as many collaborators as they want to identify the questionnaire. At the least the middle manager needs to answer, at the most everyone involved in the activity can answer. Our systems provide us with enough flexibility to treat that volume.

Analysis frequency

The analysis is to be executed at least once during the duration of the management agreement which covers our activities. It should take place at the start of the agreement. If the agreement runs for a longer period that five years, a new risk analysis needs to be executed which remains valid until the signing of the new agreement or for a period of five years, whichever comes first.

Additional analyses need to be executed on part of the audit universe if there are significant changes in that part of the audit universe. For example, were we to take on new responsibilities which are not explicitly covered in our current responsibilities but were foreseen in the management agreement, we need to execute an additional risk exercise on this responsibility. In the case of significant adaptations or alterations to activities or processes, a new risk exercise needs to be executed on that activity and all downstream activities depending on that specific activity or process. Finally, in case an adaptation in the management agreement would lead to a significant adaptation in roles or responsibilities, our function or our structure, we need to execute a new risk exercise on the activities impacted.

Analysis execution

The analysis is executed by means of an online survey. Participants are asked to judge about 80 statements on risks to their current responsibilities. In case someone is accountable for more areas of responsibility as defined in the audit universe, they need to pronounce themselves on each of these areas of responsibility through a separate survey.

Participants are asked to judge the relevance of the statement for their areas of responsibility. In addition, they are asked to judge their risk exposure over a period of five years, both in terms of impact of the risk, likelihood of occurrence and current level of risk management.

The people accountable for a process or a function are asked to execute this analysis within three months after signature of the new management agreement.

Translating the results of the risk analysis into a long term audit planning

The information gathered in the survey is translated into a risk control matrix, which is proposed for validation to the management committee. The purpose of the risk control matrix is not to develop a detailed and nuanced view on the relative proportions of the risks. Rather, we want to create a clustering of risk exposure levels to develop a prioritisation in the auditable activities (hence, the audit universe).

However, internal audit retains its independence with respect to the results of the risk analysis, which is a subjective perception on risk exposures by the people accountable for the processes. We combine the information gathered in the risk analysis with other information, such as total budgetary spend over a period (historical and forward looking) and prior audit experiences. In order to remain fully transparant, proposed changes to the priorities as derived from the risk analysis need to be motivated by internal audit.

A theoretical example: Let’s imagine for a moment that risks related to types of cash transactions are considered to be high exposure. This is based on the experience of the accountable people. However, internal audit knows and has confirmed that the number of cash transactions is significantly being reduced in the organization because of initiatives taken to cover this risk. At that moment internal audit may motivate and reduce the risk exposure level.

As risks have already been linked to auditable areas (the audit universe) since the accountable collaborators need to fill out the survey for each of these areas of accountability, we can easily prioritise based on the risk control matrix. For each area of accountability, be it an activity, a process or a subprocess, we can now calculate an overall risk exposure level. This prioritisation along the areas of responsibility (the audit universe) allows us to determine the frequency within the audit cycle of five years an audit of this area needs to be executed.

Audit coverage

Areas of responsibility with a high risk exposure level will be covered twice each audit cycle. This in effect may be two full audits, an audit and an elaborate follow-up audit, or even an audit by internal audit followed by a coverage by the court of auditors. Areas of responsibility with an average risk exposure level are covered once every audit cycle, while areas with a low risk exposure level are covered if adequate resources are available.

Audit coverage includes audits executed by our external auditors or the Court of Auditors. In order to ensure an adequate level of execution we will, at least once over the duration of the management agreement execute an audit on audit, an audit peer review of the work of the external auditors and the Court of Auditors. This peer review will allow us to assess whether the quality level of the work executed by these external parties is adequate to provide us with a reasonable assurance on the adequacy of governance, controls and risk management in the activities covered by their audits. We will use the IIA’s peer review approach. These audits will not influence the independence of the external auditors or the Court of Auditors.

Planning frequency

The long term internal audit planning is based on the risk analysis executed once over the duration of the management agreement or each five years, whichever duration is the shortest. The continued relevance of the risk analysis is questioned each year by re-introducing the assessment for validation to the management team. In case of adaptations to the risk analysis by the management team, the internal audit planning is reassessed and if necessary re-introduced to the audit committee for validation.

Other adaptations to the audit planning are only possible in case of changes within our organisation or in its operating environment which require a full or partial re-execution of the risk analysis.

You can find part II of this article here.

Reducing the effort of risk based internal audit planning

Risk based internal audit planning

The IIA's standards require us to prepare a risk based internal audit planning. However, if risk assessment and management is not (yet) embedded in your organization, it requires a concerted effort from the auditees to provide you with the relevant information. Given this is not necessarily a priority to them, are there more efficient ways to gather more relevant information you need for risk based planning without overburdening your auditees?

Defining the auditable space

In the end, our assurance role as internal auditor is to provide assurances to the audit committee, the board and management. We developed the risk control matrix to properly segregate the responsibilities of management and the responsibilities of internal audit:

• internal audit is responsible to provide assurance in the high risk areas where management considers the risk management measures to be adequate;
• internal audit is responsible to assess the relevance, appropriateness and effectiveness in the low risk areas where management may have provided too many risk management measures;
• management is responsible for developing actions plans for high risk areas where risk management measures are considered inadequate;
• management is responsible for monitoring issues in low risk areas where risk management measures are low, to ensure timely identification and management of emerging risks;

The risk control matrix is a good concept, but how do we ensure completeness of identification of all elements that need to be included in the matrix? In talking with the both the actual and the ad-interim head of internal audit at the Belgian federal government service Mobility & Transportation, we came up with the following ideas.

Identifying risks related to action plans

Action plans are developed when management deems specific risk management measures inadequate. Action plans are prioritized, ideally as a function of the risks they aim to cover. Hence, the identification of risks in quadrant I comes down to the identification of which risks the current action plans aim to cover. A good approach would therefore be to either ask management which risks they aim to cover with a specific action plan. An alternative would be to read the action plan and identify the risk which should at least be referred to in that action plan.

I am aware completeness of identification is not assured if the budgets are not adequate to fund all required action plans. I would at least expect management to have developed a list of future actions to be taken, which can be traced back to the risk we need to identify.

The assurance function of internal audit in this risk control matrix quadrant is limited. We can assess the relevance and adequacy of action plans, however, given it is the discretion of management to manage the business, and given they know there are issues, our assurance contribution would be limited. We can act in an advisory capacity, as long as this does not influence our independence and objectivity now and in the future.

Identifying risk related to measures deemed adequate by management

Quadrant II and III of the risk control matrix is where the core assurance function of internal audit is situated. Again the question occurs how we can best (as complete as possible with minimal disruption of day-to-day activities) identify the relevant risks? A suggested solution to bring the questioning our of the theoretical realm of risk to the level of day-to-day operations is to ask management to provide us with a list of risk management measures they deem adequate. The measures need to be linked to processes (elements of the audit universe) in order to allow for development of risk based, process related audit programs. We would identify risk by asking management to explain why they have taken these measures. The why is often the relevant response to which risk a control aims at covering.

Our assurance function then needs to focus on both assessing the adequacy of the risk management measure as it relates to the risk as well as the completeness of risk coverage. But how are we sure that all relevant risks under responsibility of the different members of management have been appropriately identified, assessed and covered?

Closing the risk gap

Based on the above, we now know which risks management covers with its action plans. These are reactions to risks the consider inadequately covered. We also know which risks they consider relevant and adequately covered as they offer these to us for auditing. But what about the risks not identified.

Here, we need to revert to the risk identification model, but not as a full-blown identification tool, but rather as a trigger list. A trigger list is a list which a manager reviews on a regular basis to assist him in jogging his memory on exposures known but not formally identified. If by going through the risk trigger list a manager would discover a risk not formally identified in the prior assessment, there are a couple of possible outcomes:

1. The risk is known, managed, but not formally identified. This is an issue linked to formalization which does not necessarily leads to a specific exposure.
2. The risk is known, not formally identified and not managed. This could indicate an exposure to be managed. Risk severity will impact the urgency.

Conclusion

Rather than having management and their collaborators go through a theoretical exercise each year, we can use information generated by them in the course of their day-to-day activities as a good basis for risk identification and prioritization. This would allow us to reduce the effort required from management in risk identification as well as reducing the effort we need to put in risk assessment for audit purposes.

This approach does not alllow for identification of the so-called Black Swans. I am a taker for any good solution that would not influence the efficiency of my audit planning process.