Idea: Make Woodpecker CI pipelines run as the user that triggered it instead of the woodpecker-agent user
Closedopened 7 months ago by a · 9 comments
Reference in New Issue
There is no content yet.
Delete Branch '%!s(<nil>)'
Deleting a branch is permanent. It CANNOT be undone. Continue?
Currently, our Woodpecker CI instance is using the local backend (that I wrote) which doesn't use containers. However, all pipelines are run by the
woodpecker-agentuser, which is terrible for security since any pipeline can take over other pipelines or even the Woodpecker agent itself. To mitigate this, we have currently disabled access to the CI for exozyme users unless you have been manually whitelisted.
I have an idea for improving this. Instead of running pipelines using the
woodpecker-agentuser, the Woodpecker agent should first switch to the user who triggered it using
sudo, and then run the pipeline. This also enables us to imitate Sourcehut CI's feature of being able to SSH into failed CI runs, since you can just SSH in and
cdto the pipeline directory. We'll also be able to open up the CI to all exozyme users.
However, we still have to be careful about security, since this idea doesn't entirely fix the security problems with the local backend. I propose we use the following
sudoersconfig to limit the security exposure of the
Other possible security issues include that pipelines will now have access to all your files, so someone could make a malicious pipeline, send your repo a PR, then if you merge it, you'll run the malicious pipeline with full access to your account 😱. Our current system of running everything as
woodpecker-agentavoids this, but it isn't much better since pipelines can access each others' files.
Another thing to worry about is if someone gains shell access to the
woodpecker-agentafter we implement this new idea. They will be able to become system users, but we might able able to mitigate this with
I think this idea should be simple to implement once we solve all the security issues and will probably be just a few lines of code. Should we contribute this upstream to Woodpecker CI, or is this too exozyme-specific?
Does anyone want to take a stab at implementing this? It should be pretty easy and only require changing a few lines in the local backend code. We could also do it during a future hackathon.
rootif the binary has
setuidwon't work without creating a security hole and we should use
I finished this in today's hackathon. Here's the changeset:
We also need to rewrite the CI wiki article, but that's another issue. Enjoy the new CI and please report bugs!
Does anyone think we should upstream this?
regarding your setuid question: https://www.oreilly.com/library/view/secure-programming-cookbook/0596003943/ch01s03.html (has example C code at the end)
The problem is that the
woodpecker-agentprocess can't just drop its setuid privileges at some point. It needs it at all times so it can run pipelines as other users.
We solved this instead using
sudocan be configured to allow
woodpecker-agentto run pipelines as normal users but not root.
Could we put the woodpecker CI agent into a separate mount namespace, and present it with a limited view of users $HOME's, so that it can access their binaries (e.g. Nix stuff or so), but configs and everything is at least guarded so that exploits injected into a pipeline of a user can just mangle the pipelines of that user, but can't e.g. inject malicious code into that user's bashrc or so. Another question would be if we can prevent other ways of e.g. keylogging using pipelines.
A way to reduce/avoid VM startup costs would be just letting the woodpecker CI run in a separate container, where users can log into, but the CI $HOME's would use this patch, but also be separate (the user should be able to provide a env init script, which initializes e.g.
$PATH), so that malicious pipelines can't mess with anything except other pipelines, but the user can still log into the pipeline VM and e.g. test stuff. The "malicious pipeline" stuff with PRs applies anyways, but that problem is inherent to CIs (e.g. think about secrets to which CIs often have access), and happens to all of them (e.g. afaik happened already with some GitHub, GitLab projects, etc.).
I'm sure we could use namespaces or containers to isolate pipelines so they can't do much harm, but on the other hand I don't really want any complicated or arbitrary restrictions on what pipelines can do. It kind of reminds me of https://xkcd.com/2044/
Also, I'm assuming that I trust my pipelines not to do malicious things since I'm writing my own pipelines, but yeah if someone sends me a PR to modify my project's Woodpecker pipeline, I would have to look at it in detail to ensure it's not malicious.
I just realized there's a possible vulnerability in our CI: if you push a commit impersonating another user, Woodpecker might run arbitrary code as that user!
OK, so I tried exploiting this, and fortunately that doesn't actually seem to happen. Gitea still knows the pusher of the commit and Woodpecker runs the pipeline as the pusher, not the user that supposedly created the commit.