Debbie Troklus (debbie@troklusconsulting.org) is President of Troklus Compliance Consulting, Louisville, KY. Sarah M. Couture (sarah.couture@ankura.com) is Managing Director, Ankura Consulting, Chicago.
If you are new to compliance, you may have noticed that compliance officers talk a lot about compliance program effectiveness. That is for good reason. According to the Federal Sentencing Guidelines, which provide the reasoning and framework for compliance programs, effectiveness is the expectation.[1] That’s the goal! An effective compliance program can help to “mitigate the ultimate punishment of an organization.” It is not enough to simply have a compliance program. That compliance program must be effective. So, what does that mean? According to Merriam-Webster, when something is effective, it produces “a decided, decisive, or desired effect.”[2] In a compliance program, the desired outcome is preventing and detecting fraud, waste, and abuse. That is the whole purpose of a compliance program. If a compliance program is not effective, that is, it does not effectively prevent and detect fraud, waste, and abuse; it will not meaningfully decrease organizational culpability as outlined in the Federal Sentencing Guidelines. In other words, your compliance program must work!
So, how do you know your compliance program is effective? Before 2017, there was plenty of discussion in the compliance profession about effectiveness, and it was obvious what ineffective compliance programs looked like. But what the government thought about compliance, how it viewed and defined it, and what it should look like in practice were a little more amorphous. Since compliance program effectiveness is the expectation of the government and the measurement against which a compliance program will be judged, it was challenging to know whether a compliance program would be up to par with the seemingly subjective goal. But in 2017, the U.S. Department of Health & Human Services Office of Inspector General (OIG), in collaboration with the Health Care Compliance Association (HCCA),[3] as well as the U.S. Department of Justice (DOJ)[4] published guidance documents that gave us insights into how the government viewed effectiveness. Since then, the DOJ guidance has been updated twice, in 2019 and in 2020, and the compliance profession has had more of a glimpse into how the government views effectiveness. While these guidance documents provide insights into compliance program effectiveness expectations, determining whether a program works is not a simple or one-dimensional endeavor. True effectiveness cannot be confirmed with just a checklist.
The OIG guidance specifically states: “This is not a ‘checklist’ to be applied wholesale to assess a compliance program. An organization may choose to use only a small number of these in any given year. Using them all or even a large number of these is impractical and not recommended. The utility of any suggested measure listed in this report will be dependent on the organization’s individual needs. Some of these suggestions might be used frequently and others only occasionally. The frequency of use of any measurement should be based on the organization’s risk areas, size, resources, industry segment, etc. Each organization’s compliance program and effectiveness measurementprocess will be different” (emphasis added).[5]
We must use the guidance documents and other government insights and perspectives, while also adding “soft” elements—including, but not limited to, perceptions, progress toward goals, program evolution, risk prioritization, and culture—that cannot be measured merely by a checklist. Instead, and in the spirit of the OIG’s admonition, we must tailor and prioritize measuring our program’s effectiveness and progress toward that goal in a way that is appropriate and customized according to our organization’s size, resources, areas of business, risk profile, and other company specifics.
This article discusses why effectiveness is important, why it can be challenging to achieve, the hallmarks of effective compliance programs, methods and tools to help gauge effectiveness, and practical pointers as you continue chasing the goal of compliance program effectiveness.
Why does prioritizing and pursuing compliance program effectiveness matter?
Aside from the obvious answer—mitigating the ultimate punishment of an organization (e.g., potentially lower fines, potentially avoiding a corporate integrity agreement)—there are numerous benefits for an organization that has an effective compliance program. Effective compliance programs can find, address, and fix issues before they grow too large or out of control. The longer the issue exists, the more difficult it is to fix it. Effective compliance programs have educated and informed staff that know how to do their jobs in a compliant and effective way and how to spot and report concerns so they can be addressed. Effective compliance programs have cultures of transparency, where team members are comfortable reporting concerns, and managers and leaders appreciate and act on the reports. Organizations with effective compliance programs are more likely to bill correctly and keep reimbursement dollars, instead of paying back reimbursements when future audits identify overpayments. Evidence of an effective compliance program can help result in favorable business deals, such as in mergers and acquisitions or helping secure better insurance rates. Organizations with effective compliance programs have better communication and better cultures, run more efficiently, provide higher-quality care, and are more likely to protect their brand and reputation by preventing front-page news stories about wrongdoing.
Why can compliance program effectiveness be so hard to achieve?
If compliance program effectiveness is the expectation, and it is what compliance programs are pursuing and compliance professionals are talking about, and if it is so beneficial to organizations in multiple ways, why does it sometimes seem so hard, and at times even elusive, to attain? The simplest answer is that program effectiveness is multifaceted, that there is no formula for attaining it or evaluating, and that an effective program looks a little different at every organization because every compliance program is—or should be—tailored and unique to the specific organization (e.g., size, risk profile, scope, nature of services, geography). Because of the seeming subjectivity of compliance program effectiveness, it can be challenging to paint a picture of exactly what it looks like, how to evaluate it, and how to pursue it. For this reason, this article does not offer a flat, one-dimensional approach to effectiveness, but rather offers a multidimensional perspective on effectiveness; we offer building blocks that can be considered as your unique organization seeks to pursue effectiveness.
On the road to evaluating and pursuing effectiveness, there can be multiple obstacles that prevent it. In our years of compliance program effectiveness assessment work with a variety of clients, we have seen multiple challenges that can short-circuit such attempts.
Board and senior leadership understanding, engagement, and buy-in
In many ways, the ultimate success of your program rises and falls on the compliance commitment of the board and senior leadership. This support helps to build the strong foundation that is needed for a compliance program to be effective. If the board and CEO do not understand or care about compliance, and therefore do not prioritize compliance, there cannot be program effectiveness. The compliance program may exist in some weak form but will likely be a paper program with lame-duck authority and minimal ability to truly prevent and detect fraud, waste, and abuse.
Culture
A fearful or toxic culture will prevent compliance program effectiveness. Without widespread organizational commitment to doing the right thing or transparency that results in reporting and addressing issues, the compliance program cannot be effective.
Compliance reporting structure: CCO independence and authority
The chief compliance officer (CCO) must be given appropriate independence and authority. Without proper independence and authority, the compliance program cannot be effective at preventing and detecting fraud, waste, and abuse. Seating the CCO with senior leadership and ensuring the CCO directly reports to the CEO and reports to or has a dotted line to the board helps give the CCO the authority needed to fulfill the responsibilities of the program. When a CCO is not seated with senior leadership and does not have direct access to the CEO or board, it sends the signal that compliance is not as important as other functions in the organization (i.e., those that do report to the CEO). It is also essential for compliance to be independent of operations, finance, legal, and other business functions. Ensuring the CCO reports directly to the CEO/board guarantees that compliance is not subordinate to legal, finance, or operations, thus preserving independence. Compliance should also be careful not to be in charge of operations or operations functions, as compliance programs cannot objectively audit that for which they are operationally responsible. Compliance officers should not act as management; it is an operational function.
Adequacy of risk assessment and its impact on the compliance program
Ensuring thorough, collaborative, and ongoing risk assessment and prioritization is key to ensuring compliance program effectiveness. To best prevent and detect fraud, waste, and abuse, a compliance program must understand the highest-risk areas and then devote appropriate resources to those risks. Programs that are not laser-focused on risk can end up spending inordinate resources addressing low-risk concerns instead of effectively addressing high risks. According to the DOJ, “prosecutors may credit the quality and effectiveness of a risk-based compliance program that devotes appropriate attention and resources to high-risk transactions, even if it fails to prevent an infraction.”[6]
Compliance resource sufficiency and management
To be effective, compliance must be given sufficient resources (i.e., budget, staffing) to be able to carry out the compliance program. The compliance program should base its work on ongoing risk assessment and ensure compliance program resources, including time, are spent appropriately on higher-risk areas. An adequately resourced program is an indicator of an effective program.
Structure and maturity of each of the seven elements
Each of the seven elements of an effective compliance program should be appropriately designed and implemented and should evolve with the compliance program. When one or more of the seven elements is not built well or working like it should, the program may not be as effective as it would otherwise be (see “Hallmarks” section discussing DOJ and OIG guidance for specific elements, discussed later).
Documentation and tracking
Potentially as concerning as a paper/shelf program is a program that has much activity, but that activity is not adequately documented. In the clinical world, a well-known adage is, “If you did not document it, you did not do it.” The same can be said for a compliance program: if you did not track it or document it, you cannot provide evidence that you did it. It is essential to track all compliance program activity (e.g., reports, guidance requests, audits, investigations, education and training, meeting minutes, agendas) and ensure thorough documentation and document retention.
Access to, use of, and integration of data
Compliance must be given access to all company data to appropriately help prevent and detect fraud, waste, and abuse. Additionally, the compliance program must learn to intelligently use data, as data analysis can be used to both assess risk and measure effectiveness of risk mitigation work and compliance program effectiveness itself. A compliance program cannot be effective without access to all company data (and people, for that matter), and program effectiveness can be enhanced with smart data analysis and integration.
Operations engagement in compliance
Compliance cannot be successful in a vacuum or on an island. Compliance must engage operations and help operations understand their compliance responsibilities. Compliance is everyone’s job. It takes a village.
Leveraging risk partners
To be effective, compliance must collaborate with other risk partners when appropriate. This may include legal, risk management, internal audit, human resources, and potentially other internal and external partners. Ineffective compliance departments try to operate alone without bringing in appropriate partners.
Use of outside expertise
There may be times where your compliance program staff do not have sufficient experience or expertise to appropriately handle a certain concern. Whether for a specialized investigation, an audit, or to gain a fresh, objective perspective on a specific issue or your compliance program generally, consider how to best leverage appropriate expertise for the task at hand, whether that expertise is within your department or external. Know when you don’t know.
Hallmarks of compliance program effectiveness
While there are many challenges that can weaken attempts to achieve effectiveness, there are also signs of life and vibrancy that, when taken together, can signal to your organization, the public, and regulators alike that your organization has an effective compliance program.
Engaged board and leadership
Effective programs are those in which board members and organizational leaders understand their compliance obligations, have regular meaningful interactions with the compliance officer around compliance matters, and are clearly committed to doing the right thing no matter what. Compliance officers should have a regular seat at board meetings.
Culture of compliance
Effectiveness can be evidenced when employees know how to function compliantly, identify aberrancies, and report concerns without fear of retaliation, and managers and leaders encourage transparency and then remedy identified issues.
Culture of continuous improvement and evolution
Acknowledge that the compliance program can always become better and more effective. Be intentional in evaluating and planning for program effectiveness and ensure the program evolves over time to best align with the organization’s needs and risk profile.
Ongoing risk awareness, assessment, and prioritization
Ensure that compliance program work, resources, time, audits, work plan, and seven elements are based on prioritized risk that is being continually assessed in collaboration with operations partners.
Root cause analysis and incorporation of lessons learned
When something does go wrong, an effective compliance program analyzes the situation to learn lessons, then adapts the program according to those lessons. This can lead to more effective prevention and detection of noncompliance.
Humility, transparency, and feedback
Acknowledging that there is no perfect organization, no perfect compliance program, and no perfect compliance officer will help ensure humility and flexibility in the program. Facilitate and ask for transparency; ask for feedback from your team, leadership, and operations; and then make appropriate changes based on that feedback.
Intentional compliance program effectiveness plan, including assessment
Develop a plan for ensuring compliance program effectiveness, including how it will be developed, pursued, reported on, and evaluated. Best-practice programs prioritize ongoing, at least annual, documented self-assessments and outside, objective effectiveness assessments every two to three years.
Test and analyze controls that are in place in compliance and in operations
Do the controls that are in place work? Do the things that we think are controlling the risk work? Test the controls, document outcomes, and implement changes if needed.
Follow-up
Ensure corrective action plans are developed by management, implemented, and working over time. Compliance should monitor the corrective action plans.
Data-driven decision-making
Great business intelligence can be found in data analysis. Identify a partner to help you think through leveraging business data to improve compliance program effectiveness and decision-making around risks.
Not operating in a silo
Collaborate with operations, legal, risk, quality, internal audit, human resources, outside agencies/consultants, and professional organizations to ensure appropriate perspectives and expertise.
Well-engaged operations that understand their compliance responsibility
Effective compliance programs empower operations to own compliance. Operations leaders and managers should ensure appropriate internal controls are in place (i.e., policies and procedures, education and training, monitoring of high-risk functions) so that their operational departments and teams can function compliantly.
Documentation, tracking, analyzing trends, and reporting
Effective compliance programs set expectations through systems, protocols, and templates to ensure compliance program reports and activities are tracked, thoroughly documented, and reported to leadership and the board as appropriate.
Available, approachable, and knowledgeable staff
Effective compliance programs have compliance officers and staff that are personable, collaborative, knowledgeable, and approachable. Effective compliance staff work to build rapport with operations partners, and ensure they always have an open door and are available.
Getting outside help or a fresh perspective when needed
There may be times where your compliance program does not have the necessary expertise or experience to conduct an audit, investigation, assessment, or other project. Seek external help to ensure appropriate expertise or a fresh perspective. This may help ensure effective and thorough audits or investigations.
Seven-element development and implementation
Develop your program elements so they are pragmatic, applicable, and can flex and evolve as the organization evolves (e.g., in size, risk profile). Are your program elements well designed, implemented, and do they work? We recommend reviewing your elements against the specific guidance and suggestions from Evaluation of Corporate Compliance Programs and Measuring Compliance Program Effectiveness: A Resource Guide, as these guidance resources provide perspectives on the effectiveness of specific program elements.
Methods and tools to help evaluate compliance program effectiveness
As we have discussed, every effective compliance program looks different based on the size, complexity, and risk profile of its organization, and there is no one way to assess its effectiveness. In this section, we offer a few methods and tools to help you assess your program’s effectiveness. You cannot rely on just one of these tools, but we encourage you, based on the uniqueness of your organization and program, to use a blend of these methods and tools to help develop a tailored approach to assessing, then pursuing, effectiveness.
Guidance documents
As previously mentioned, government guidance documents offer both specific expectations around compliance programs as well as perspectives and questions to help assess effectiveness. Familiarize yourself with these documents and use them as you develop your own approach to an effectiveness assessment.
-
Federal Sentencing Guidelines, Chapter Eight
-
OIG Compliance Program Guidance[7]
-
HCCA-OIG Measuring Compliance Program Effectiveness: A Resource Guide
-
Practical Guidance for Health Care Governing Boards on Compliance Oversight[8]
-
DOJ Evaluation of Corporate Compliance Programs
-
HCCA compliance program resources and other benchmarks[9]
-
The Institute of Internal Auditors’ Three Lines model[10]
Benchmarks
Benchmarks are a great way to compare your progress to the progress of peer organizations. While the availability of benchmarks has increased over the years, this is still an area of development for our profession. HCCA has benchmarks for resources like staffing and budgeting, as well as salary benchmarks. Other organizations and some vendors track and provide specific benchmarks that may be relevant to your organization. Research what is available and use benchmarks where you can to understand your program’s effectiveness.
Surveys
Conducting surveys is an excellent way to gauge knowledge and perceptions of your employees. Consider adding specific compliance survey questions to your regular employee survey to better understand the impact your program is having on your culture and employees. Ensure that you follow up with employees, notifying them of the correct survey responses, where appropriate, as this can be a form of education and training. Measuring Compliance Program Effectiveness: A Resource Guide has many survey ideas for consideration. Also, ensure exit interviews occur to identify risk and opportunities for enhancing program effectiveness.
Rounding
Spend time in various operations departments getting to know staff, observing work environments and activities, and talking to employees. Consider a simple rounding checklist with a small number of questions that you can ask employees and record as you observe. This will allow you to understand your program’s impact on specific areas, identify areas where your program can enhance communication or consider a review, and will also allow you to benchmark departments against each other and trend progress over time.
Audits
Review recent internal and external audit results to identify areas for improvement. Look at auditing as a chance to enhance controls and incorporate lessons learned.
Data analysis
Trend and analyze compliance activity data to look for opportunities to enhance effectiveness and employ data analysis to review risk areas more efficiently.
Dashboards
Dashboards are a great way to not only communicate a large amount of information in a graphic way, but also to trend your progress and effectiveness over time. Develop dashboards to communicate what is going on in your program, track progress, and analyze trends to identify opportunities to enhance effectiveness.
Develop your own!
What other tools or assessment methods can give you insights into the effectiveness of your program? Be creative and collaborate with your staff and compliance committee to develop pragmatic assessment tools.
Conclusion
Just as there is no one-size-fits-all compliance program, there is no-one-size-fits-all way to evaluate or pursue effectiveness. Take the resources and methods described in this article to tailor a custom approach to evaluating your program’s effectiveness. Develop a plan for ongoing effectiveness assessment and pursuit. Be intentional and focused on a culture of continuous program improvement; your organization is evolving and so should your program. Discuss program effectiveness and your ongoing strategy to get there with your board, leadership, compliance committee, and compliance staff. Stay engaged with operations. Be creative with resources, ensuring risks are prioritized. Be flexible and look for lessons learned and incorporate them. Know that the work is never done; your compliance program should always be moving and evolving as you are chasing the goal of compliance program effectiveness.
Note: Ankura is not a law firm and cannot provide legal advice.
Takeaways:
-
All compliance programs should be intentionally pursuing effectiveness.
-
Effective compliance programs have many other benefits besides the primary purpose: mitigating the ultimate punishment of an organization.
-
Organizations face a variety of challenges to achieving effectiveness, including, but not limited to, support, resources, authority, engagement, and culture.
-
There is no one way to assess effectiveness; rather, programs should use a variety of guidance documents, methods, and tools to develop an approach to evaluation of effectiveness.
-
Evaluation of effectiveness should not just happen once but instead be an ongoing assessment and improvement process.