Michael Edenzon
January 27, 2024
•
10
min read
When it comes to compliance, it’s obvious. If you say something is done, show the proof. In the first part of our multi-part series on Automated Governance, we’re going to discuss the topic of proof and how it can be used to drive a successful Automated Governance implementation within your company.
Everywhere you look regulations and liability are shaping the way that software is shipped. No longer is it sufficient to thoroughly test your code, mitigate vulnerabilities in a timely manner, and track changes to production. Those practices, and more, must now be extensively documented with a preponderance of evidence for the sole purpose of mitigating liability and satisfying regulatory oversight. That’s compliance, and it has become the intersection of engineering, security, quality, and risk management departments across nearly every large organization that builds software.
As focus shifts toward the DevOps pipeline as a critical piece of the software supply chain it has become the epicenter of software compliance. Recent high-profile breaches have drawn further attention to build and release automation, which until only recently, was largely seen as a non-production system, advantageous in that it allowed developers flexibility to modify its behavior to powerful ends with minimal overhead and toil. However, in recent years regulated organizations have spent a great deal of time and money clawing back ownership of their DevOps automation in an effort to consolidate these processes into something that resembles an enterprise pipeline library. The reasons for this are varied — some view it as a necessary refactoring of sprawling patterns that evolved from rapid adoption of CI/CD during their DevOps transformation, others are mobilized by increased scrutiny from auditors and regulators toward GRC practices for continuous delivery. Regardless of the impetus, pipeline consolidation requires layers of abstraction that ultimately lead to centralization of ownership that typically belongs to the platform engineering team.
A common benefit of a centralized pipeline is the ability to break builds based on non-compliance. In this context, a build occurs in the continuous integration (CI) pipeline, when code is compiled and published. We refer to a “broken build” as an intentional failure of this process, preventing software from publishing to an artifact repository before it can be deployed. To be clear, this practice is as old as the pipeline itself. Nonetheless, it still elicits a visceral reaction from both software and platform engineers. Let’s assume that whether you’re in favor of breaking builds or against it, you’re reading this article which means that you believe that compliance should be enforced with automation regardless of when the enforcement actually occurs.
Proponents of breaking builds believe that a non-compliance event should stop the pipeline at first sign of trouble, citing the “shift-left” mantra as a foremost justification. Opponents favor enforcement at some later point in the process, such as deployment. This approach doesn’t admonish the “shift-left” approach, rather it shifts enforcement to the leftmost point in the delivery process and often cites developer productivity as a justification for doing so.
Regardless of your orthodoxy, the goal is achieving compliance through automated enforcement.
At its core, compliance is the act of adhering to a set of policies designed to protect a business process from risk. Ensuring the compliance of the software development process is something that cannot be achieved unilaterally. It requires measurements and evaluations from a variety of tools that collectively determine the status of the software in question as it is compared against policy.
So let’s get back to proof.
Compliance requires two points of proof: the first is proof of measurement, the second is proof of adherence. For a given policy, you must prove, first, that the process in question was measured and, second, that the measurements, when evaluated against policy, met or exceeded policy requirements. If you can prove both points, then you have proof of compliance.
Procedures outline how policy compliance is proven and enforced.
A common misconception is that policy enforcement alone is sufficient proof of compliance. This is a logical fallacy in that it asserts that the primary evidence for adherence to policy is the lack of enforcement. In layman’s terms, it claims that because a process was not stopped to enforce policy it is therefore compliant.
For example, consider a shipbuilder tasked with building a nuclear submarine. The process of procuring the steel from suppliers introduces the risk of accepting the wrong grade of steel and welding it into the ship’s hull. If the engineering schematics require that the submarine’s pressure hull consist of HY-80 steel, the consequences of welding HY-100 or H-130 steel could be catastrophic.
To mitigate the risk of welding the wrong steel, the shipbuilder’s policy states that metallurgy reports shall accompany each shipment of steel. Procedures outline the process for validating the supplier’s metallurgy report, either through a third party or in-house testing, and dictates that non-compliant steel must be returned to the supplier before it reaches the shipyard.
When audited, the shipbuilder points to their procedures and claims that the pressure hull of the submarine consists only of HY-80 steel because procedures don’t allow the incorrect steel to reach the shipyard.
No reasonable auditor would accept such a claim because the existence of a procedure is not proof of compliance.
Yet, such claims are commonplace in many regulated CI/CD pipelines where procedures are largely automated. Regardless of when enforcement occurs in the process of software development, pipeline owners claim that successful execution of procedures — a merged pull request, completed build, or successful deployment — constitutes compliance because a non-compliance event would have caused a failure.
In short, the claim states that since the software was merged, built, and deployed successfully it is therefore compliant.
In law this is referred to as argumentum ad ignorantiam, or “argument from ignorance.” A proposition that is considered true because it has not been proven false. An argument from ignorance holds little weight in the court of law because it depends on the absence of evidence rather than the existence of affirmative evidence.
Proof requires affirmative evidence that details the business process in question was measured and that it met or exceeded the policy requirements. This means that whether or not policy was enforced, you are only compliant if you can explain the what and why. What was measured and why did it pass.
Achieving the use of automation to enforce policy in your CI/CD pipeline is an extraordinary step in your journey to automated governance, but without proof it is just an argument from ignorance. In our next article we will talk about what differentiates good proof from bad proof, practical examples, and how you can extend your automation to prove your compliance beyond a reasonable doubt.
Schedule a demo today!
