A measure of success

Would you measure your sales effectiveness by looking at whether or not you had a sales force in place and whether or not they were trained? Would you measure the effectiveness of your logistics by simply analyzing the design and operation of current processes? Would you measure the effectiveness of your manufacturing by inspecting the machines that created the products that you sell?

Of course not. You measure effectiveness (really performance) by analyzing the outcomes that these processes generate for the business. But how do you go beyond legal effectiveness and truly measure the performance of your governance, risk and compliance (GRC) processes? In this article, I introduce a proposal for what that performance scorecard should look like.

For each of these areas, an organization should develop a number of indicators, baseline the measurement of these indicators and establish future targets so that progress and performance can be measured.

Measurement of GRC is in its infancy and so it is understandable that most proposals for evaluating performance involve long lists of candidate indicators. Unfortunately, my proposal is no exception. Fortunately, however, it provides a number of indicators that have been vetted by a community of over 14,000 professionals.

There are a number of ‘first order’ outcomes that a solid GRC approach should deliver including:

1. Culture
Does the program inspire a principled culture of performance, accountability, trust, and open communication?

Sometimes considered ‘touchy feely’ and nebulous, culture has been a challenge for many executives (especially executives from an accounting, audit or legal background) to measure. Fortunately, research conducted over the past decade provides a solid foundation on which we can build a fact-based understanding of culture.

Ask individuals about how they personally feel with regard to the organization, its direction, their role and whether they feel prepared to fulfill their obligations. Some of these indicators are typically captured in an ‘Employee Engagement’ survey conducted by human resources. This helps to provide context for other perceptions that an individual may have.

Things to know are whether the employee understands the organizational mission, vision and direction. Probe for whether they ever feel pressured to compromise ethics, integrity or to violate policies or the law. Are they comfortable discussing ethics and integrity with people outside of the organization?

Get employees to discuss how they view their direct supervisor and their peers. Do they think the supervisor sets a good example for ethics and demonstrates consistency (not hypocrisy)? Are they comfortable broaching the subject of ethics with colleagues? It is also key to learn whether they feel an atmosphere of non-retaliation.
 
2. Preventing noncompliance
Does the program actually prevent violations, noncompliance and unethical conduct?

Ask individuals about observed misconduct and violations in the workplace over a specific timeframe. Keep in mind that these indicators are perceptions and may not necessarily reflect fact (eg, a person may observe something that they perceived is wrong or illegal, but may, in fact, not be illegal). That said, the fact that someone perceives wrongdoing is in and of itself an important (even more important) indicator.

So, how do you substantiate misconduct and weaknesses? It’s crucial to look at the frequency and types of violations and weaknesses that were not prevented to understand how well the program is actually performing. A program will not be able to prevent all violations. However, over time, the program should get better and better at preventing, detecting and responding to similar types of issues.

To develop a good picture of overall performance, measure reported violations rate per employee; substantiated violations rate per employee; number of actual violations or weaknesses identified via ongoing or periodic monitoring; and frequency of violation types.

While it is very difficult to prove a negative, the following indicators are good proxies for understanding if the program is preventing violations: employees’ willingness to seek advice; comfort calling a helpline or seeking advice when they are not sure what actions to take; helpline usage; and the number of weaknesses in the program detected during an evaluation.

3. Preparing for actual misconduct
Does the program actually prepare the organization to address key risks, noncompliance and unethical conduct?

Employees need the right skills to understand what compliance and ethics issues present operational risks and how to deal with them, up to and including reporting them, if appropriate. One way to understand this is to ask individuals about their perceived level of skill.

Do the employees believe they have the necessary skills to recognize ethical and compliance challenges? Do they think they might not be prepared? Do they know how to seek advice if they have a question? Do they know how to report a violation?

Skills assessment and performance appraisals can be used to help establish a baseline. All of the same questions and indicators that are listed above continue to apply; only now, employees are tested for actual competence versus merely asked about their own perceptions of their competence.

4. Protecting the organization
Does the program adequately protect the organization from negative consequences if or when noncompliance and unethical conduct materializes?

A critical aspect of every program is to provide adequate coverage of risks including legal and regulatory risks. These are typically part of an effectiveness evaluation and represent the ‘legal effectiveness’ of the program. To this end, areas to measure are critical risks that have control and accountability in place; critical risks that are addressed by federal sentencing requirements or specific laws and regulations, critical risks and requirements that have multiple layers of control in place like policies, training, preventive controls, detective controls and workforce controls including compensation and performance appraisal incentives.

Supply chains also pose their own problems. Do key suppliers have contractual commitments aligned to risk tolerance? Are   there supplier audit provisions, indemnifications and limits in your contracts? Finally, are those controls that are in place operating as designed?

5. Detecting non-compliance
Does the program actually detect non-compliance and unethical behavior when it occurs?

Workforce perceptions represent important indicators that the organization has sound detection in place. Useful perceptions to consider are: whether employees think violations and misconduct are actually detected and whether they observed violations and actually reported the violation. Can you capture the reason that people did not report? Was the issue resolved? Did the employee not want to get someone fired? Did they think it was not their responsibility? Did they fear retaliation? Did they not know who to contact?

Process excellence will also help management and the board understand how long issues linger before they are detected. Categories that could help flesh out the reasons are: detection lag time and the number of days it took for organization to actually detect a violation.

Proactive detection rate can be another key piece of data. Were there weaknesses discovered by internal staff before they were exploited? Were there weaknesses discovered by internal staff or via a reporting hotline after a violation actually occurred but still via planned activities or mechanisms? And what percent of weaknesses did external auditors, authorities or other external source discover before the organization was able to discover it?

6. Responding to non-compliance
Does the program appropriately, consistently and quickly respond to events once they are detected?

As with detection, it is important to understand workforce perceptions of how well the organization responds to misconduct and violations – both the cycle time as well as the quality of response after reporting a violation. What is the perceived speed of response? The perceived quality of response? The perceived consistency of response?

Actual process metrics can help management and the board understand exactly how quickly the company responds to and resolves issues such as response lag time, the number of days it takes to respond to a reported violation and the cause of response lag. While every issue will have its own set of circumstances and facts, similar issues should be compared.

7. Improving the program
Once the organization detects and responds to a weakness, is the weakness actually fixed so that similar events do not materialize in the future? Analysis of repeat issues is the single best measure. Consider whether substantiated violations are similar to previous violations.

8. Reducing loss
Does the program reduce the tangible and intangible damage that is the result of non-compliance and unethical behavior?

Losses associated with non-compliance and other violations can mount. Ideally, an organization should track direct costs associated with the investigation and ultimate resolution of an issue to better understand how much the organization lost because of the event/issue. Areas of expense to consider are: internal investigation cost; internal and external resources spent to investigate issues; fines and penalties; impairment of assets; market cap reduction; workforce turnover; number of staff who voluntarily left citing compliance and ethics issues; and business interruption.

There are intangible expenses as well which can be difficult to quantify. However, you can try to gauge reputational loss; change in customer or supplier confidence in the organization; and negative media coverage.

9. Optimizing ROI
Does the organization continuously optimize its resource allocation to deliver similar or improved outcomes? Does it balance proactive and reactive costs? Does it balance strategy with operation? What are the direct costs? Are you counting fully-loaded staff costs and expenditures on technology and other assets that are part of the program? And don’t forget planning costs; preventative costs incurred in developing a code of conduct, policies, procedures and other controls; the price of training; the price of designing and implementing workforce incentives and other controls; insurance costs; and the hotline/helpline expenses, among others.

10. Enhancing stakeholder perception of value
In the end, organizations must meet the expectations of a number of stakeholders including shareholders, ratings agencies, creditors and other business partners. Some  organizations may consider other stakeholders as well, such as the community and ‘the environment’ itself. 

The key categories to assess are: a) Stock performance: financial performance compared to industry, geographic and other peers as defined by the organization; b) Governance ratings and the percentage changes in ratings from outfits like Governance metrics (GMI), Institutional Shareholder Services (ISS); Audit Integrity; and the Corporate Library; and c) Credit ratings and their percentage changes from agencies like Standard & Poor’s and Moody’s.

There is also the issue of media coverage. Media serves as a proxy for and helps to drive public opinion. Favorable vs. unfavorable media can affect the perceptions of customers, suppliers, partners, employees and regulators. Therefore, measuring change in media coverage (percent change in favorable vs. unfavorable media coverage) is crucial.

Open Compliance and Ethics Group (OCEG) has begun the work of developing an approach to metric analysis in its ‘Measurement and Metrics Guide’ (OMG). The organization has also undertaken a benchmarking study of more than 150 entities entitled ‘Proving the Value of GRC: Measures and Metrics’.