BOSTON—June 26, 2024—Aqua Security, the pioneer in cloud native security, today revealed new research that shows how credentials, API tokens, and passkeys – collectively referred to as secrets – from organizations around the globe were exposed for years. By scanning the most popular 100 organizations on Github, which collectively includes more than 50,000 publicly accessible repositories, Aqua researchers found active secrets from open source organizations and enterprises such as Cisco and Mozilla providing access to sensitive data and software. The exposed secrets could lead to significant financial losses, reputational damage, and legal consequences.
Aqua Security’s research team, Aqua Nautilus, revealed that “phantom secrets” can persist in Git-based infrastructure used by most Source Code Management systems (SCMs), including GitHub, Gitlab, Bitbucket and others. This is due to the way in which even deleted or updated code commits are saved in those systems, such that even a one-time developer mistake can expose secrets to savvy threat actors over extended periods.
“Our findings are truly alarming, and it is crucial that everyone involved in software development grasps the seriousness of this issue,” says Yakir Kadkoda, Aqua Nautilus Lead Security Researcher. “For years, we’ve been educating developers not to hard-code secrets into their code. Now, it turns out that even doing this just once permanently exposes that secret – even when they thought it was deleted or overwritten. The impact of a sensitive data leak can lead to unauthorized access, compromised security controls and significant financial or reputational damage. This would be devastating.”
Among the exposed secrets found by scanning open Github repositories were API tokens of Cisco Meraki and the Mozilla project. The Cisco security team confirmed the findings: “We discovered privileged Meraki API tokens used by some Fortune 500 companies. These tokens could allow attackers to access network devices, Simple Network Management Protocol secrets, camera footage, and more, serving as an initial foothold for the exposed parties.”
The Mozilla project acknowledged “An API token for the Mozilla FuzzManager with read-write privileges” and that “an employee’s API token for sql.telemetry.mozilla.org was leaked”; both were assigned a “Critical” score. Not only does the FuzzManager allow access to many potential security vulnerabilities in Firefox and Tor, but the telemetry gave access to confidential information related to Mozilla products and business.
Additionally, Nautilus found an Azure service principal token belonging to a large healthcare company exposed in a Git commit. This token had high privilege and high access to obtain credentials to the internal Azure Container Registry, which could have led an attacker to perform a supply chain attack impacting the organization, and customers.
In all cases, the exposed secrets were immediately revoked.
Commit once, expose forever
While secure coding best practices already require that secrets should not be hard coded, many developers continue this practice. They rely on secrets scanning tools to ensure that such secrets are not pushed into production and often re-commit the updated code without those secrets.
Phantom secrets exist because of underlying processes within Git-based SCMs, which cause code that was overwritten or deleted in repositories to remain accessible within the underlying system. Most secrets scanners only look at repos accessible via the Git clone command, which overlooks almost 18% of secrets.
“The findings once again reinforce the best practice that secrets should never be put into code, not even for testing purposes, and security teams must be able to monitor this,” says Amir Jerbi, CTO and co-founder of Aqua Security. “The software supply chain is optimized for speed and convenience, but this cannot come at the expense of secure engineering practices.”
“IDC research underscores Aqua Nautilus’ findings, showing that organizations are overly confident in their posture related to the protection of application secrets,” says Katie Norton, Research Manager, DevSecOps & Software Supply Chain Security, IDC. “While organizations show high confidence in their ability to secure secrets, among DevSecOps tools the adoption of secrets management solutions is among the lowest.”
Available in August, Aqua customers using the Software Supply Chain Security module will be able to prevent developers from committing code with embedded secrets, and scan for phantom secrets hidden within their SCM file system.
The Phantom Secrets research details will be presented by Kadkoda today at CloudNativeSecurityCon in Seattle during his session: “Below The Radar: Identifying Hidden Threats Within The Development Ecosystem.” For a full technical explanation of how Phantom Secrets persist, and why they are often missed, read Aqua’s blog and the full research.
About Aqua Nautilus
Aqua Nautilus is a security research team whose mission is to analyze the evolving cloud native threat landscape, uncovering new threats targeting containers, Kubernetes, serverless, applications’ software supply chains and cloud infrastructure. The team aims to help Aqua customers and the community at large protect against the unknown, zero-day and emerging threats, turning insights from real-world attacks into powerful, intelligence-driven protection within the Aqua Platform.
About Aqua Security
Aqua Security sees and stops attacks across the entire cloud native application lifecycle in a single, integrated Cloud Native Application Protection Platform (CNAPP). From software supply chain security for developers to cloud security and runtime protection for security teams, Aqua helps customers reduce risk while building the future of their businesses. Founded in 2015, Aqua is headquartered in Boston, MA and Ramat Gan, IL protecting over 500 of the world’s largest enterprises. For more information, visit https://www.aquasec.com
