One does not commit or compile credentials
Context:
This meme was brought to you by the PyPI Director of Infrastructure who accidentally hardcoded credentials - which could have resulted in compromissing the entire core Python ecosystem.
If I had a dollar for every API key inside a config.json…
Here’s the thing, config.json should have been on the project’s .gitignore.
Not exactly because of credentials. But, how do you change it to test with different settings?
At my workplace, we use the string to designate code that shouldn’t be checked in. Usually in a comment:
// @nocommit temporary for testing
apiKey = 'blah';
// apiKey = getKeyFromKeychain();
but it can be anywhere in the file.
There’s a lint rule that looks for in all modified files. It shows a lint error in dev and in our code review / build system, and commits that contain anywhere are completely blocked from being merged.
(the code in the lint rule does something like "@no"+"commit" to avoid triggering itself)
This sounds like a really useful solution, how do you implement something like this? Especially with linter integration
Depending on which stack you’re using, you could use https://danger.systems to automatically fail PRs.
PRs? Isn’t the point of that something does not get committed, and therefore no credentials are stored in the git repository? Even if the PR does not get merged, the file is still stored as a hit object and can be restored.
At my workplace, we use the string @nocommit to designate code that shouldn’t be checked in
That approach seems useful but it wouldn’t have prevented the PyPI incident OP links to: the access token was temporarily entered in a .py python source file, but it was not committed to git. The leak was via .pyc compiled python files which made it into a published docker build.
Yeah, but a combination of this approach, and adding all compiled file types including .pyc to .gitignore would fix it.
Neat idea. This could be refined by adding a git hook that runs (rip)grep on the entire codebase and fails if anything is found upon commit may accomplish a similar result and stop the code from being committed entirely. Requires a bit more setup work on de developers end, though.
Would a git hook block you from committing it locally, or would it just run on the server side?
I’m not sure how our one at work is implemented, but we can actually commit files in our local repo, and push them into the code review system. We just can’t merge any changes that contain it.
It’s used for common workflows like creating new database entities. During development, the ORM system creates a dev database on a test DB cluster and automatically points the code for the new table to it, with a comment above it. When the code is approved, the new schema is pushed to prod and the code is updated to point to the real DB.
Also, the codebase is way too large for something like ripgrep to search the whole codebase in a reasonable time, which is why it only searches the commit diffs themselves.
I also personally ask myself how a PyPI Admin & Director of Infrastructure can miss out on so many basic coding and security relevant aspects:
- Hardcoding credentials and not using dedicated secret files, environment variable or other secret stores
- For any source that you compile you have to assume that - in one way or another - it ends up in the final artifact - Apparently this was not fully understood (“.pyc files containing the compiled bytecode weren’t considered”)
- Not using a isolated build process e.g. a CI with an isolated VM or a container - This will inevitable lead to “works on my machine” scenarios
- Needing the built artifact (containerimage) only locally but pushing it into a publicly available registry
- Using a access token that has full admin permissions for everything, despite only requiring it to bypass rate limits
- Apparently using a single access token for everything
- When you use Git locally and want to push to GitHub you need an access token. The fact that article says “the one and only GitHub access token related to my account” likely indicates that this token was at least also used for this
- One of the takeaways of the article says “set aggressive expiration dates for API tokens” - This won’t help much if you don’t understand how to handle them properly in the first place. An attacker can still use them before they expire or simply extract updated tokens from newer artifacts.
On the other hand what went well:
- When this was reported it was reacted upon within a few minutes
- Some of my above points of criticism now appear to be taken into account (“Takeaways”)
This will inevitable lead to “works on my machine” scenarios
Isn’t that what Python is all about?
Yes kids, the only stuff in ANY repo (public or otherwise) should be source code.
If it is compiled, built, or otherwise modified by any process outside of you the developer typing in your source code editor, it needs to be excluded/ignored from being committed. No excuses. None. Nope, not even that one.
No. 👏 Excuses. 👏
To err is to be human… right?
To be honest, this doesn’t instill me with much confidence, but who am I? If someone looked at my OpSec, probably they’d be horrified.
don’t commit credentials; split them up and place each part in a different place in the code and use code comments as a treasure map and make them work for it.
On the contrary, one can commit or compile credentials quite simply… Maybe Boromir isn’t the right person to ask.
