Idea: Make Woodpecker CI pipelines run as the user that triggered it instead of the woodpecker-agent user #181
Labels
No Label
bug
duplicate
enhancement
help wanted
invalid
question
security
wontfix
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Blocks
#175 Ideas for improving exozyme security
exozyme/exozyme
Reference: exozyme/exozyme#181
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Currently, our Woodpecker CI instance is using the local backend (that I wrote) which doesn't use containers. However, all pipelines are run by the
woodpecker-agent
user, which is terrible for security since any pipeline can take over other pipelines or even the Woodpecker agent itself. To mitigate this, we have currently disabled access to the CI for exozyme users unless you have been manually whitelisted.I have an idea for improving this. Instead of running pipelines using the
woodpecker-agent
user, the Woodpecker agent should first switch to the user who triggered it usingsudo
, and then run the pipeline. This also enables us to imitate Sourcehut CI's feature of being able to SSH into failed CI runs, since you can just SSH in andcd
to the pipeline directory. We'll also be able to open up the CI to all exozyme users.However, we still have to be careful about security, since this idea doesn't entirely fix the security problems with the local backend. I propose we use the following
sudoers
config to limit the security exposure of thewoodpecker-agent
:Other possible security issues include that pipelines will now have access to all your files, so someone could make a malicious pipeline, send your repo a PR, then if you merge it, you'll run the malicious pipeline with full access to your account 😱. Our current system of running everything as
woodpecker-agent
avoids this, but it isn't much better since pipelines can access each others' files.Another thing to worry about is if someone gains shell access to the
woodpecker-agent
after we implement this new idea. They will be able to become system users, but we might able able to mitigate this withrunas_check_shell
insudoers
.I think this idea should be simple to implement once we solve all the security issues and will probably be just a few lines of code. Should we contribute this upstream to Woodpecker CI, or is this too exozyme-specific?
Does anyone want to take a stab at implementing this? It should be pretty easy and only require changing a few lines in the local backend code. We could also do it during a future hackathon.
Bad news:
prints
root
if the binary hascap_setuid
sosetuid
won't work without creating a security hole and we should usesudo
instead.I finished this in today's hackathon. Here's the changeset:
We also need to rewrite the CI wiki article, but that's another issue. Enjoy the new CI and please report bugs!
Updated diff:
Does anyone think we should upstream this?
regarding your setuid question: https://www.oreilly.com/library/view/secure-programming-cookbook/0596003943/ch01s03.html (has example C code at the end)
The problem is that the
woodpecker-agent
process can't just drop its setuid privileges at some point. It needs it at all times so it can run pipelines as other users.We solved this instead using
sudo
, sincesudo
can be configured to allowwoodpecker-agent
to run pipelines as normal users but not root.Could we put the woodpecker CI agent into a separate mount namespace, and present it with a limited view of users $HOME's, so that it can access their binaries (e.g. Nix stuff or so), but configs and everything is at least guarded so that exploits injected into a pipeline of a user can just mangle the pipelines of that user, but can't e.g. inject malicious code into that user's bashrc or so. Another question would be if we can prevent other ways of e.g. keylogging using pipelines.
A way to reduce/avoid VM startup costs would be just letting the woodpecker CI run in a separate container, where users can log into, but the CI $HOME's would use this patch, but also be separate (the user should be able to provide a env init script, which initializes e.g.
$PATH
), so that malicious pipelines can't mess with anything except other pipelines, but the user can still log into the pipeline VM and e.g. test stuff. The "malicious pipeline" stuff with PRs applies anyways, but that problem is inherent to CIs (e.g. think about secrets to which CIs often have access), and happens to all of them (e.g. afaik happened already with some GitHub, GitLab projects, etc.).I'm sure we could use namespaces or containers to isolate pipelines so they can't do much harm, but on the other hand I don't really want any complicated or arbitrary restrictions on what pipelines can do. It kind of reminds me of https://xkcd.com/2044/
Also, I'm assuming that I trust my pipelines not to do malicious things since I'm writing my own pipelines, but yeah if someone sends me a PR to modify my project's Woodpecker pipeline, I would have to look at it in detail to ensure it's not malicious.
I just realized there's a possible vulnerability in our CI: if you push a commit impersonating another user, Woodpecker might run arbitrary code as that user!
OK, so I tried exploiting this, and fortunately that doesn't actually seem to happen. Gitea still knows the pusher of the commit and Woodpecker runs the pipeline as the pusher, not the user that supposedly created the commit.