{"id":3938,"date":"2024-05-04T11:23:13","date_gmt":"2024-05-04T15:23:13","guid":{"rendered":"https:\/\/dft.wiki\/?p=3938"},"modified":"2026-06-08T10:37:57","modified_gmt":"2026-06-08T14:37:57","slug":"how-to-transform-devops-into-devsecops","status":"publish","type":"post","link":"https:\/\/dft.wiki\/?p=3938","title":{"rendered":"How to Transform DevOps into DevSecOps"},"content":{"rendered":"<p>These are my notes about what DevSecOps effectively adds to DevOps.<\/p>\n<p>With DevSecOps, security checks are pushed to the beginning and are part of all development phases (Design \/ Plan \/ Develop \/ Build).<\/p>\n<p><strong>Key Points<\/strong><\/p>\n<ul>\n<li>Secure SDLC (aka SSDLC)\n<ul>\n<li>Source code quality checks<\/li>\n<li>Dependency management<\/li>\n<li>Artifact scanning<\/li>\n<li>Threat modeling<\/li>\n<li>Unit and integration tests<\/li>\n<li>And more.<\/li>\n<\/ul>\n<\/li>\n<li>Secure CI\/CD\n<ul>\n<li>Minimized permissions<\/li>\n<li>Defense against pipeline attacks or weaponization (no shared build agents)<\/li>\n<li>Strict segmentation for applications, data, systems, environments, etc.<\/li>\n<li>And more.<\/li>\n<\/ul>\n<\/li>\n<li>Infrastructure as Code (IaC)\n<ul>\n<li>Secret management<\/li>\n<li>GitOps best practices<\/li>\n<li>Monitoring and alerting<\/li>\n<li>And more.<\/li>\n<\/ul>\n<\/li>\n<li>Process\n<ul>\n<li>Add security requirements to story templates, review checklists, and refinement sessions.<\/li>\n<li>A risk assessment decision matrix for features, assets, and resources is required.<\/li>\n<li>Only build on top of approved base images.<\/li>\n<li>Staged and gradual rollout of new versions with fallback capabilities.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Challenges to Overcome<\/strong><\/p>\n<ul>\n<li>Security must not be siloed; it should be a company-wide discipline.<\/li>\n<li>Visibility (normalized logs, tailored alerts, behavioral monitoring, etc.) is paramount.<\/li>\n<li>Prioritizing security issues should be part of the company culture.<\/li>\n<li>Due diligence before using any tool or library, plus maintaining an up-to-date software bill of materials.<\/li>\n<li>Master and test in a sandbox before pushing to production.<\/li>\n<li>Continuously watch for developer bypasses before they leak into production.<\/li>\n<\/ul>\n<p><strong>Metrics<\/strong><\/p>\n<ul>\n<li>Mean Time to Production (MTTP)\n<ul>\n<li>The time it takes to get a brand new app deployed to production from its first commit.<\/li>\n<\/ul>\n<\/li>\n<li>Mean Time to Recover (MTTR)\n<ul>\n<li>The ability of the infrastructure to recover from a failed deployment (rollback) or a service interruption (self-healing).<\/li>\n<\/ul>\n<\/li>\n<li>Deployment Agility\n<ul>\n<li>A combination of the speed and frequency of deployments.<\/li>\n<\/ul>\n<\/li>\n<li>Failure Rate\n<ul>\n<li>The percentage of deployments that contained a bug relative to the total number of deployments. This reveals the scope of opportunity for improvement.<\/li>\n<\/ul>\n<\/li>\n<li>Communication of Risks\n<ul>\n<li>In DevOps, comparing these metrics over time shows the impact of changes during a given period. In DevSecOps, risk also means the likelihood of a vulnerability being exploited and its impact on systems.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Automated Tests<\/strong><\/p>\n<ul>\n<li>Unit Test\n<ul>\n<li>Tests whether the application works (is not broken) and responds as expected (across some or all endpoints of an API) when compared against a source of truth.<\/li>\n<\/ul>\n<\/li>\n<li>Integration Test\n<ul>\n<li>Focuses on testing the integration between applications (connectivity, authentication, etc.) and also tests for regression.<\/li>\n<\/ul>\n<\/li>\n<li>Security Test\n<ul>\n<li>SAST (Static) &#8211; Source code analysis.\n<ul>\n<li>Easy to integrate into code reviews at Pull Request (PR) \/ Merge Request (MR) time.<\/li>\n<\/ul>\n<\/li>\n<li>DAST (Dynamic) &#8211; Fuzzing and automated pentesting.\n<ul>\n<li>Can be triggered when promoting to staging, where intrusive tests can safely target the application in a controlled environment.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<hr \/>\n<p><strong>K8s<\/strong><\/p>\n<p><strong>DevSecOps Approach to Kubernetes<\/strong><\/p>\n<ul>\n<li>Integrates security at every stage of the container lifecycle: development, deployment, and operations.<\/li>\n<li>Applies security through the CI\/CD pipeline all the way into Kubernetes runtime, not as an afterthought.<\/li>\n<li>Scans container images for vulnerable dependencies; favors minimalistic base images, non-root users, non-privileged container execution whenever possible, and read-only filesystems.<\/li>\n<li>Policy as Code (PaC) \/ RBAC \/ ACL: restrict or limit inter-pod communication and access to other resources.<\/li>\n<li>Use Key Vaults for secrets and certificates, and use environment-specific variables provided at start time.<\/li>\n<li>Store logs in an external, centralized location for auditing and monitoring.<\/li>\n<\/ul>\n<div><b>PSA (Pod Security Admission) and PSS (Pod Security Standards)<\/b><\/div>\n<ul>\n<li><strong>PSS<\/strong> defines security policies at 3 levels (see below) at a namespace or cluster-wide level.\n<ul>\n<li><strong>Privileged:<\/strong> Near-unrestricted policy that allows privilege escalation.<\/li>\n<li><strong>Baseline:<\/strong> Minimally restricted policy that prevents privilege escalation but allows pods to be deployed with a default configuration.<\/li>\n<li><strong>Restricted:<\/strong> Heavily restricted policy that follows current pod hardening best practices.<\/li>\n<\/ul>\n<\/li>\n<li><strong>PSA<\/strong> enforces these <strong>PSS<\/strong> policies by intercepting API server requests and applying them accordingly.<\/li>\n<\/ul>\n<hr \/>\n<p><strong>TOOLS<\/strong><\/p>\n<ul>\n<li>Source code scanners:\n<ul>\n<li><strong>SonarQube Community Edition<\/strong> [<a href=\"https:\/\/github.com\/SonarSource\/sonarqube\">Link<\/a>]<\/li>\n<li><strong>Semgrep<\/strong> [<a href=\"https:\/\/github.com\/semgrep\/semgrep\">Link<\/a>]<\/li>\n<li><strong>Snyk<\/strong> [<strong>Link<\/strong>]<\/li>\n<\/ul>\n<\/li>\n<li>Credential scanning:\n<ul>\n<li><strong>GitLeaks<\/strong> [<a href=\"https:\/\/github.com\/gitleaks\/gitleaks\">Link<\/a>]<\/li>\n<li><strong>GittyLeaks<\/strong> (hack) [<a href=\"https:\/\/github.com\/kootenpv\/gittyleaks\">Link<\/a>]<\/li>\n<li><strong>TruffleHog<\/strong> [<a href=\"https:\/\/github.com\/trufflesecurity\/trufflehog\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<li>Dependency management and SBOM:\n<ul>\n<li><strong>OSV-Scanner<\/strong> [<a href=\"https:\/\/github.com\/google\/osv-scanner\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<li>Scan container and VM images for vulnerable packages, secrets, misconfigurations, and more:\n<ul>\n<li><strong>Docker Scout CVE<\/strong> [<a href=\"https:\/\/docs.docker.com\/reference\/cli\/docker\/scout\/cves\/\">Link<\/a>]<\/li>\n<li><strong>Trivy<\/strong> [<a href=\"https:\/\/github.com\/aquasecurity\/trivy\">Link<\/a>]<\/li>\n<li><strong>Grype<\/strong> [<a href=\"https:\/\/github.com\/anchore\/grype\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<li>Optimize container images:\n<ul>\n<li><strong>Slim<\/strong> [<a href=\"https:\/\/github.com\/slimtoolkit\/slim\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<li>Scan IaC manifests for errors and misconfigurations, and suggest best practices:\n<ul>\n<li><strong>Checkov<\/strong> [<a href=\"https:\/\/github.com\/bridgecrewio\/checkov\">Link<\/a>]<\/li>\n<li><strong>TFLint<\/strong> [<a href=\"https:\/\/github.com\/terraform-linters\/tflint\">Link<\/a>]<\/li>\n<li><strong>KICS<\/strong> [<a href=\"https:\/\/github.com\/Checkmarx\/kics\">Link<\/a>]<\/li>\n<li><strong>Conftest<\/strong> [<strong>Link<\/strong>]<\/li>\n<\/ul>\n<\/li>\n<li>Automate pentesting in the pipeline:\n<ul>\n<li><strong>OWASP ZAP<\/strong> [<a href=\"https:\/\/github.com\/zaproxy\/zaproxy\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<li>QA tests for web apps in the browser:\n<ul>\n<li><strong>Cypress<\/strong> [<strong>Link<\/strong>]<\/li>\n<\/ul>\n<\/li>\n<li>Real-time behavioral security analysis:\n<ul>\n<li><strong>Falco<\/strong> [<a href=\"https:\/\/github.com\/falcosecurity\/falco\">Link<\/a>]<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<hr \/>\n<p><strong>PROVISIONING INFRASTRUCTURE vs CONFIGURATION MANAGEMENT &amp; AUTOMATION<\/strong><\/p>\n<p>The overlapping capabilities of these tools often blur the distinction between concepts. While both provisioning and configuration management tools aim to automate IT processes, their core purposes differ. Understanding these differences helps in choosing the right tool for the right task.<\/p>\n<p>Overlap between both concepts:<\/p>\n<ul>\n<li><strong>Declarative approach<\/strong>\n<ul>\n<li>Define the desired state, and the tool ensures it is provisioned accordingly.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Idempotency<\/strong>\n<ul>\n<li>Applying the same configuration multiple times always produces the same result.<\/li>\n<\/ul>\n<\/li>\n<li><strong>State enforcement<\/strong>\n<ul>\n<li>Ensures that systems remain in the desired configuration.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Source control<\/strong>\n<ul>\n<li>Provides consistency, version tracking, and collaboration when managing infrastructure and configurations.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Distinct concepts:<\/p>\n<p><strong>Infrastructure as Code (IaC)<\/strong><\/p>\n<p>Infrastructure provisioning focuses on creating, modifying, and managing infrastructure resources such as virtual machines, networks, storage, and cloud services. The primary goal is to define cloud resources in a way that ensures deployment repeatability, version control, and scalability.<\/p>\n<ul>\n<li><strong>Terraform<\/strong>\n<ul>\n<li>Terraform (by HashiCorp) and OpenTofu (a community fork of Terraform) use state files (stateful) to track resources and their previous state.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Configuration Management &amp; Automation<\/strong><\/p>\n<p>Configuration management and automation tools focus on maintaining the desired state of system and software configurations on existing infrastructure. They are typically used to install and manage software, enforce security policies, and orchestrate system-wide tasks.<\/p>\n<ul>\n<li><strong>Ansible<\/strong>\n<ul>\n<li>Ansible takes a stateless approach, checking the current state of each component in scope at runtime using modules. It can also be used imperatively to execute commands (such as shell commands) on target systems.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<hr \/>\n<p><strong>BONUS<\/strong><\/p>\n<p><strong>GitOps<\/strong><\/p>\n<p>A methodology that uses Git as the single source of truth for declarative infrastructure and application management. It can be used in CI\/CD pipelines to automate infrastructure and application deployments, reducing both MTTP and MTTR to their minimum.<\/p>\n<ul>\n<li><strong>GitLab \/ GitHub<\/strong>\n<ul>\n<li>Regardless of the flavor, these services wrap around Git and add their own features on top. Each commit is a new layer identified by a checksum that cannot be tampered with or removed, making everything fully auditable down to exact changes, timestamps, and authors.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Jenkins<\/strong>\n<ul>\n<li>Responsible for identifying triggers and running the respective tasks to produce the desired output.\n<ul>\n<li>Example 1\n<ul>\n<li>A change to a Terraform module in Git could trigger the infrastructure change by applying it to the respective environment.<\/li>\n<\/ul>\n<\/li>\n<li>Example 2\n<ul>\n<li>A modification to an Ansible playbook could trigger a task that applies the change to a target or group of targets.<\/li>\n<\/ul>\n<\/li>\n<li>Example 3\n<ul>\n<li>A code change in an in-house application could trigger the necessary tasks to build it and deploy it to the desired environment (dev, stage, test, hotfix, prod, etc.)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>These are my notes about what DevSecOps effectively adds to DevOps. With DevSecOps, security checks [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-3938","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/3938","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3938"}],"version-history":[{"count":16,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/3938\/revisions"}],"predecessor-version":[{"id":5623,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/3938\/revisions\/5623"}],"wp:attachment":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3938"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3938"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3938"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}