It has been 13 years since the original COSO ERM Framework and eight years since ISO 31000:2009 was published. The updated COSO ERM Framework was an opportunity for COSO to “leap forward”. But did it?
I have been working on this for a while. I wanted to be fair to COSO and PwC; several of my friends were involved in the update project on the COSO Board and as advisers. I respect them all.
To perform a detailed assessment, I used the 12 questions I developed to assess the exposure draft with two additional questions at the end. Each is scored on a scale of 1-10, where 10 is best.
But first I need to step back and address whether my wishes and expectations for the update were the same as COSO’s. Then I can give my overall recommendation and then the detailed assessment.
I don’t think they were the same.
I started from the point of view that risk management today is far too often ineffective and needs a catalyst to spark a change.
As I said in my comment letter to COSO on the exposure draft, “surveys, notably by Deloitte, have found a huge disconnect between those leading risk management and the executives and directors who should be obtaining value from it. Only a small percentage said that risk management had made a significant contribution to their setting and execution of strategies.”
According to the surveys, executives see risk management as a compliance exercise. It is not seen as essential to running the business day-to-day – when in fact it can and should be critical to success, not just avoiding failure. In my comment letter, I set out a number of reasons.
It’s 13 years since the original COSO ERM Framework, 8 years since ISO 31000:2009 was published. Who knows when the next COSO update will be, and the news from the teams working on the ISO update is discouraging.
This was an opportunity for COSO to “leap forward” and “transform how organizations are run, from the setting of the mission, objectives, strategies and plans to the daily operation of the business: how it performs in practice through intelligent and informed decision-making at levels of the extended enterprise”.
Enhancements to the COSO framework would be ‘nice’, but when risk management in practice is failing to be seen as vital to success, ‘nice’ is insufficient.
A leap change rather than incremental change is necessary – and something needs to be a catalyst for that change.
I hoped that would be the COSO ERM update.
It is not.
It is my opinion that COSO and PwC did not seek to incorporate leading thinking and practices. I made sure they had a copy of World-Class Risk Management and was assured that they had read it. I also suggested (in the comment letter and in calls with leaders of the update project) that they involve thought leaders.
They appear to be satisfied with modest improvements, incremental changes that, in my opinion, will not change practices to any great extent. This is their news release.
Leading risk management practitioners are already ahead of what COSO ERM 2017 suggests.
Yes, they have made progress:
- Eliminated the cube
- Stressed the need to consider risk (what might happen) when selecting strategies
- Mentioned (without detail) the needs to enable decision-making and to address bias (more normally called cognitive bias) and culture
- Said that risk management is more than the periodic review of a list of risks
They have also introduced diagrams that purport to show the relationships between strategy, performance, risk, risk capacity, and risk tolerance. Sorry, but I don’t think the diagrams are more than sound in theory. I doubt they work in practice and I question whether they are even theoretically sound as they suggest that you can aggregate all forms of risk and risk capacity, which you cannot in the real world.
So, bottom line, where am I?
The update is an interesting contribution to the world of risk management guidance. But…
- It is insufficient to describe or support the effective management of risk.
- It is also insufficient as a basis for the assessment of risk management.
However, it is worth buying, reading, and considering – along with ISO 31000:2009, my book (which is way ahead of COSO I am afraid), and other guidance.
Now for the 14 questions and my detailed assessment:
|1. Does the update provide useful guidance that will help leaders of the organization define the mission, objectives, strategies, and plans that will deliver optimal value to stakeholders?
||Rating: 7/10 COSO has made a significant improvement in its discussion of the need to embed the consideration of risk into both strategy-setting and execution. I particularly like the reference to scenarios and the need to consider what might happen under each strategic option.
Principle 8 is The organization evaluates alternative strategies and potential impact on risk profile. It talks about evaluating strategies and making sure that they are aligned with the mission, vision, and values of the organization.
However, there is no discussion of the possibility that the mission or vision is sub-optimal, or that (for reasons such as poor information or not involving the appropriate people) the strategy is not the best.
Risks to setting strategy are important and this is a gap in the COSO update.
The update mentions assumptions, but not the possibility that the assumptions are incorrect. Mature organizations should understand that and assess the likelihood of an error that would be significant to the achievement of the strategy or objective; actions should be taken where necessary.
I find the discussion of risk appetite, profile, and strategy somewhat confusing. Tt recognizes that some will set appetite before selecting strategy and others will do the reverse; this is a reasonable point to make. However, when discussing the setting or risk appetite and defining risk profile, it assumes strategies and objectives are defined. When selecting strategies, it assumes risk profiles and appetite are in place. I think this could have been written better and as a result I am unsure how people will be able to interpret and use the guidance.
What I find lacking is any discussion of the need to assess the likelihood and extent of all potential consequences in a disciplined and systematic fashion. In other words, use similar methods when considering the benefits of a strategy as when assessing potential harms.
Further, there is no discussion of the need to take all the potential effects, both good and bad, into account when selecting a strategy. On balance, do the potential benefits outweigh the potential harms? Instead, there is a focus on the list of harms (the risk profile) and risk appetite.
|2. Does the update provide useful guidance when it comes to executing against the defined mission, objectives, strategies, and plans? Is there sufficient guidance on effective decision-making, and will it move the practice of risk management away from only reviewing, periodically, a list of risks? Will it lead to organizations practicing risk management continuously?
||Rating: 2/10 While the Executive Summary talks about decision-making, there is really no guidance on this. There are no principles and no practical guidance on how decisions should be made, considering all potential consequences.
This is critical, as this is where risk is taken in the real world.
The section on Performance is all about risks – potential harms.
In real life, as distinct from the world of standards and frameworks, people at all levels across the extended enterprise are taking or not taking risk every day. They do this through decisions.
Every decision creates or modifies risk.
The key to the effective management of risk is having decision-makers take the desired amount of the right risk.
This is simply not covered.
It is simplistic to think that you take risk only as the result of a risk assessment activity.
As a result, I have great concern as to whether the COSO update will influence risk-taking in practice, in the real world.
|3. Will the guidance still lead people to only identify, assess, and address potential harms? Will risk reporting still be focused on the level of risk rather than the likelihood of achieving each objective?
||Rating: 2/10COSO and I are on totally different pages.
They see events or situations (or decisions) having either a positive (opportunity) or negative (risk) opportunity.
In the real world, events or situations have not only multiple potential effects, but each is a range and not a point.
The framework asks that people identify opportunities as well as risks, but not the combination of good and bad that is likely to follow from an event or situation.
Even then, COSO insists on a risk profile (a list of potential harms) and assessing whether risks are within risk appetite, without any consideration of the positive that may accompany a negative.
Further, there is only a suggestion to include the effect on objectives as one of the bases for prioritizing risk.
If it is all about achieving objectives and fulfilling strategies, then the focus needs to be there and not on risk.
The management of risk needs to be far more than maintaining a risk profile. It has to be about taking the right level of the right risks with every business decision.
|4. Does the guidance explain clearly and help decision-makers understand and then evaluate all the potential effects of uncertainty?
||Rating: 1/10 As discussed above, this is not covered and is a serious problem IMHO.|
|5. Will the update provide decision-makers with the structure/process they need to decide whether to ‘take the risk’ because of the potential for reward?
||Rating: 1/10 As discussed above, the criteria for determining whether to take a risk does not include any reference to the potential for reward, only the appetite for risk.|
|6. Will the update lead to providing decision-makers with the guidance they need if they are to make the decisions management and the board want them to make?
|Rating: 1/10 COSO says that risk appetite is cascaded down to decisions-makers but provides no practical guidance or examples.|
|7. Does the update provide sufficient guidance on how to assess and then correct, as necessary, the culture of the organization?
||Rating: 4/10 While the update mentions risk culture and emphasizes its importance, there is no practical guidance.
|8. Does the update provide sufficient guidance on each stage of the risk management process, including identifying, assessing, evaluating, and treating risk and opportunity? Does it provide sufficient guidance on communications and monitoring, including continuous improvement?
||Rating: 5/10 This is thin, but so is ISO 31000:2009.|
|9. Is the updated COSO guidance on risk appetite and risk tolerance useful? Does it mirror and enable effective decision-making in real life? Does the guidance help to establish not only the upper limit of ‘risk’ that should be taken, but the lower level as well?
||Rating: 2/10 The update fails to address the points made in the question (1stcolumn).
COSO has introduced new charts that purport to show the relationships between levels of performance and the level of risk that needs to be taken to achieve each level of performance, the risk capacity of the organization, the risk tolerance, and the level of variability in performance that is acceptable.
But are these charts more than simply interesting?
Are they reflective or real life? Are they practical guidance?
IMHO, they are flawed.
Thought leaders have questioned the concept of risk appetite, and this section from the update is telling.
|10. Will it be possible to assess the effectiveness of risk management in practice using the updated version?
||Rating: 1/10 The recommended approach by COSO is significantly flawed.
No guidance is provided on how to assess whether the principles are present and functioning. Compare this to the COSO Internal Control – Integrated Framework, where such guidance is provided: (a) internal control can be considered effective if there is reasonable assurance that risks to objectives are at acceptable levels, and (b) the principles are present and functioning if there are no “major” weaknesses – and the latter is where the weakness means that there is a lack of reasonable assurance that risks to objectives are at acceptable levels.
Further, there are no principles or practical guidance on decision-making – which is where risk is actually taken day-to-day.
Arguably, the principles can be assessed as present and functioning, yet executive management and the board still sees risk management as failing to make a significant contribution to both the setting and the execution of strategy.
|11. Will the guidance provide sufficient guidance to enable the board and/or a committee of the board to provide effective oversight?
||Rating: 1/10 See #10, above.|
|12. Is the updated document consumable? Is it too long? Will it be read, understood, and acted on by all levels of the organization?||Rating: 2/10 It appears long but the practical guidance is short.
While it may be read and understood, the valuable comments are terse and few.
Much is missing from the guidance in terms of what effective risk management really is – from strategy-setting through execution through decision-making.
|13. Will the updated product help the busy executive or board member understand what risk management is all about, that it is not simply a compliance exercise but can improve the likelihood of quality decisions and the achievement of the right objectives?||Rating: 1/10 It is not persuasive that risk management will help an organization succeed. At best, it might avoid a level of failure.
I would not provide the busy executive or board member a copy of the Executive Summary.
|14. Is the 2017 product a sharp improvement on the 2004 version?
||Rating: 3/10 There are improvements, as reflected in my comment letter. For example, there is language (even if the guidance is thin) on culture, decision-making, cognitive bias, and risk capacity.
It simply is well behind leading thinking on risk management and I would not recommend that any organization embrace it and believe that is sufficient.
ISO 31000:2009 is not perfect either. Is it better? Perhaps. It is also thin in a number of areas.
At minimum, everybody interested in COSO ERM should also read and consider ISO 31000:2009. In some respects, they complement each other.
But there is more to risk management, and I in all modesty believe the guidance in my book is superior.
 Exploring Strategic Risk reported that “Only 13% of [C-level] respondents believe their risk management processes support, at a high level, the ability to develop and execute business strategies”