Skip to content


Harry Cassin
Publisher and Editor

Andy Spalding
Senior Editor

Jessica Tillipman
Senior Editor

Bill Steinman
Senior Editor

Richard L. Cassin
Editor at Large

Elizabeth K. Spahn
Editor Emeritus

Cody Worthington
Contributing Editor

Julie DiMauro
Contributing Editor

Thomas Fox
Contributing Editor

Marc Alain Bohn
Contributing Editor

Bill Waite
Contributing Editor

Russell A. Stamets
Contributing Editor

Richard Bistrong
Contributing Editor

Eric Carlson
Contributing Editor

The perils of measuring compliance program ‘effectiveness’

A new report recently published by The Ethics and Compliance Initiative (ECI), the 2019 Global Business Ethics Survey, addresses the state of E&C programs across ten countries — USA, Brazil, UK, Spain, France, Italy, Germany, Russia, India, and China. Findings from the report are intriguing. They tell us something about the state of E&C practices in key countries, but they also show how difficult it is to measure program effectiveness.

Five areas of measurement are highlighted in the report — whether E&C goals are part of the organization’s strategy; the organization’s effectiveness in identifying and managing risk areas; senior leadership conduct; means for reporting unethical behavior and misconduct; and use of policies that address ethical violations in the organization.

About 200 senior managers from organizations with 5,000 or more employees in each of the ten countries were invited to assess the five areas. Participants’ ratings were used to gauge the state of current E&C practices in each country. In addition, they were also scored to evaluate the maturity of the E&C program of the organization each senior manager was referring to in his or her ratings on a scale from “Underdeveloped” (i.e., program or program area presents only a few high-performing elements) to “Optimizing” (i.e., program or program area presents all or nearly all high-performing elements).

What We Know and What We Don’t Know

According to the report, most senior managers believe that E&C is central to their organization’s strategy. For example, 89 percent of respondents indicated that E&C objectives are included in the overall strategy and nearly as many agreed that senior leaders champion the importance of E&C plans (84 percent) and engage in communications that highlight E&C practices (81 percent).

Yet, ratings fail to clarify whether senior leaders talk about the E&C plan in relation to the larger business plan and related business goals in a generic or specific way. They also say nothing about how the E&C plan fares with respect to day-to-day business practices, in particular in situations in which employees are faced with ethics and business goals that conflict with each other.

In the area of risk management, the study’s findings are also intriguing. For example, 87 percent of senior managers reported that senior leaders “identify and mitigate risks related to E&C,” but only 75 percent agreed that “there is a good understanding of key E&C risk areas across the organization.”

Either senior leaders work in a bubble and fail to propagate their E&C-related efforts or something here is not adding up.

Risk management can be viewed as a good acid test for these ratings. Take, for example, the finding that “according to most senior managers surveyed in the study, their organization encourages, protects, and values reporting of ethical issues.”

If, in fact, most organizations have multiple ways to report misconduct (87 percent), all employees who report misconduct are treated with consistency and fairness (79 percent), and leaders are trained on the organization’s policy on retaliation (81 percent), shouldn’t we find a higher understanding of E&C risks across the organization? This seems especially true given that 80 percent of respondents agreed that their organization uses metrics to detect risk areas.

It is encouraging that 91 percent of senior managers reported that their organization has defined the ethical conduct it expects from its senior leadership and 81 percent reported that senior leaders “walk the talk.”

Similarly, it is also a step forward that, according to 89 percent of respondents, their organization expects employees to uphold the organization’s values and holds accountable those who don’t. Accountability is, indeed, a key component of an ethical culture, and unquestionable clarity around policies regarding how the organization treats unethical behavior and the principles of conduct to which these policies refer helps raise the salience of ethics and increase baseline ownership across all stakeholders.

ECI Insights Against CPI and FCPA

Yet, some of the findings in the current study are at odds with other data points. In particular, the 2018 Corruption Perceptions Index and data showing FCPA-related enforcement actions across the 10 countries tell a different story when it comes to ethics and compliance maturity in each country. The Corruption Perceptions Index “ranks 180 countries and territories by their perceived levels of public sector corruption according to experts and businesspeople” on a scale from 0 (i.e., highly corrupt) to 100 (i.e., very clean). As such, it offers a particularly apt comparison term: corruption in the public sector is a good proxy for corruption in business practices at large given that the “corruptors or offenders” are very likely to operate in the private sector.

According to the ECI’s study, Brazil, China, and India are at the “managing” stage — that is, they have achieved high maturity — in the area of embedding E&C goals in the organization’s business strategy. India is also at this high-maturity stage in the area of risk identification and mitigation. In these domains, for example, the study finds that 96 percent of senior managers in Brazil and China reported that their organization integrates E&C objectives into the business strategy, and 90 percent of senior managers in India indicated that senior leaders in their organization have a good understanding of what the E&C risk areas entail.

The three countries are also at the “managing” level with respect to senior leadership conduct. Furthermore, India and Brazil seem to excel in the area of reporting unethical behavior and misconduct. Finally, in the area of holding employees accountable for ethical violations, Russia joins India, China, and Brazil as one of the countries with the most mature performance in this very complex E&C area. Interestingly, the study shows that 89 percent of senior managers in China and Russia indicated that “consistent disciplinary action is taken when ethics violations are substantiated” in their organization — a surprising finding that also raises the question of how little we know about how strongly or leniently disciplinary actions are enforced in each of these geographies.

These findings stand out not only because Brazil, China, India, and Russia score low on the Corruption Perceptions Index — 39, 35, 41, and 28, respectively — but because all four of them are also associated with a high number of FCPA-related enforcement actions — 21, 59, 21, and 17 respectively. Furthermore, when we compare the ratings by senior managers in these four countries with the ratings by senior managers in Germany, a country that scores 80 on the Corruption Perceptions Index, we find that the reports provided by the latter group are much less positive compared with the reports provided by the former group.

Strengthening Program Effectiveness Measurement

While it is imperative to measure program effectiveness and, in many cases, self-reported ratings are the best, if not the only, way to do so, we have to keep in mind the perils and hurdles of using subjective measures. Self-reports are always colored by a certain amount of self-serving bias. Such bias can be even stronger when it comes to ethics, especially if we ask those stakeholders who are responsible for enforcing and/or implementing ethics and compliance practices.

Thus, not only should we use adequately probing measures to assess program effectiveness if we want to curb bias, but we need very specific questions if we wish to move away from a check-the-box approach. It’s not sufficient that an E&C program presents certain components. It is also important to take into account how these components are designed, implemented, and linked to other business practices, particularly if we want the program to help shape the organization’s culture in meaningful ways.


Caterina Bulgarella, Ph.D., pictured above, is the co-founder of Be Thread, a platform of tools for human-centric work environments, an SAI Global influencer, and a member of Ethical Systems’ core team. She’s a culture architect and ethics expert who advises senior leaders on culture change and ethical challenges. She can be contacted here.

Share this post


Comments are closed for this article!