Distrobox doesn't work on exohub #170

Closed
opened 2022-07-29 19:43:24 +00:00 by a · 9 comments
Owner

This is the error when you try to create a Distrobox:

cannot clone: Operation not permitted
Error: cannot re-exec process

I think it's because user namespaces are disabled in the exohub systemd sandboxing.

This is the error when you try to create a Distrobox: ``` cannot clone: Operation not permitted Error: cannot re-exec process ``` I think it's because user namespaces are disabled in the exohub systemd sandboxing.
a added this to the v9.0 milestone 2022-07-29 19:43:24 +00:00
a added the
bug
security
labels 2022-07-29 19:43:24 +00:00
a self-assigned this 2022-07-29 19:43:24 +00:00
a added this to the (deleted) project 2022-07-29 19:43:24 +00:00
a started working 2022-07-29 19:43:27 +00:00
Author
Owner

This is similar to #129. Also, this is the default systemd sandboxing we use for JupyterHub:

# Apply some service hardening.
# The default LocalProcess spawner needs SETUID and SETGID to run the
# single-user servers.
CapabilityBoundingSet=CAP_SETUID CAP_SETGID
LockPersonality=true
NoNewPrivileges=true
PrivateTmp=true
PrivateDevices=true
ProtectClock=true
ProtectControlGroups=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=full
ReadWritePaths=/etc/jupyterhub
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
SystemCallErrorNumber=EPERM
SystemCallFilter=@system-service
This is similar to #129. Also, this is the default systemd sandboxing we use for JupyterHub: ``` # Apply some service hardening. # The default LocalProcess spawner needs SETUID and SETGID to run the # single-user servers. CapabilityBoundingSet=CAP_SETUID CAP_SETGID LockPersonality=true NoNewPrivileges=true PrivateTmp=true PrivateDevices=true ProtectClock=true ProtectControlGroups=true ProtectHostname=true ProtectKernelLogs=true ProtectKernelModules=true ProtectKernelTunables=true ProtectProc=invisible ProtectSystem=full ReadWritePaths=/etc/jupyterhub RestrictNamespaces=true RestrictRealtime=true RestrictSUIDSGID=true SystemCallArchitectures=native SystemCallErrorNumber=EPERM SystemCallFilter=@system-service ```
Author
Owner

I disabled all of the systemd sandboxing for exohub and now distrobox works! I'm considering now whether to try re-enabling some of the sandboxing.

I disabled all of the systemd sandboxing for exohub and now distrobox works! I'm considering now whether to try re-enabling some of the sandboxing.
Author
Owner

Alright, I re-enabled all the sandboxing options except for the following:

[Service]
NoNewPrivileges=false
RestrictNamespaces=false
SystemCallFilter=

Now Podman and Distrobox seem to work! Just reopen this issue if you still encounter any problems.

Alright, I re-enabled all the sandboxing options except for the following: ``` [Service] NoNewPrivileges=false RestrictNamespaces=false SystemCallFilter= ``` Now Podman and Distrobox seem to work! Just reopen this issue if you still encounter any problems.
a closed this issue 2022-07-29 20:15:30 +00:00
a stopped working 2022-07-29 20:15:30 +00:00
32 minutes 3 seconds
Author
Owner

It looks like Distrobox is still broken:

distrobox create --image docker.io/mageia
Image docker.io/mageia not found.
Do you want to pull the image now? [Y/n]: 
Trying to pull docker.io/library/mageia:latest...
Getting image source signatures
Copying blob 2b7a6260b5e1 done  
Error: writing blob: adding layer with blob "sha256:2b7a6260b5e1024ee3e3aaea14424ae322182becf6d1593b6542c7e711e2c6bc": Error processing tar file(exit status 1): open /usr/bin/chage: operation not permitted

An error occurred

An error occurred
It looks like Distrobox is still broken: ``` distrobox create --image docker.io/mageia Image docker.io/mageia not found. Do you want to pull the image now? [Y/n]: Trying to pull docker.io/library/mageia:latest... Getting image source signatures Copying blob 2b7a6260b5e1 done Error: writing blob: adding layer with blob "sha256:2b7a6260b5e1024ee3e3aaea14424ae322182becf6d1593b6542c7e711e2c6bc": Error processing tar file(exit status 1): open /usr/bin/chage: operation not permitted An error occurred An error occurred ```
a reopened this issue 2022-07-29 23:16:31 +00:00
Author
Owner

Setting RestrictSUIDSGID=false fixes the issue.

Setting `RestrictSUIDSGID=false` fixes the issue.
a closed this issue 2022-07-29 23:32:42 +00:00
Author
Owner

Oh oops, I'm getting a new error now:

Error: unable to start container "c36522b91f06eab693e8865a5cf02902a0208df983f425ab495cd0758e57c20f": crun: sethostname: Operation not permitted: OCI permission denied
Oh oops, I'm getting a new error now: ``` Error: unable to start container "c36522b91f06eab693e8865a5cf02902a0208df983f425ab495cd0758e57c20f": crun: sethostname: Operation not permitted: OCI permission denied ```
Author
Owner

Adding ProtectHostname=false fixes that issue.

Adding `ProtectHostname=false` fixes that issue.
a reopened this issue 2022-07-29 23:35:35 +00:00
a closed this issue 2022-07-29 23:35:37 +00:00
Author
Owner

OK, the issue still seems to be persistent. Here are some more errors I've encountered on exohub:

Created symlink /etc/systemd/system/graphical.target.wants/udisks2.service → /usr/lib/systemd/system/udisks2.service.
Failed to send reload request: Permission denied
Failed to write 'change' to '/sys/devices/LNXSYSTM:00/uevent': Permission denied
/usr/lib/tmpfiles.d/static-nodes-permissions.conf:17: Failed to resolve group 'kvm'.
fchownat() of /run/systemd/seats failed: Operation not permitted
fchownat() of /run/systemd/sessions failed: Operation not permitted
fchownat() of /run/systemd/users failed: Operation not permitted
fchownat() of /var/lib/systemd/coredump failed: Read-only file system
fchownat() of /tmp failed: Operation not permitted
fchownat() of /run/media failed: Operation not permitted
Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal failed: Read-only file system
Failed to re-open '/var/log/journal': Operation not permitted
fchownat() of /var/log/journal failed: Read-only file system
Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal/deccb8485cd444ab8cea39ee0572b4df failed: Read-only file system
Failed to re-open '/var/log/journal/deccb8485cd444ab8cea39ee0572b4df': Operation not permitted
fchownat() of /var/log/journal/deccb8485cd444ab8cea39ee0572b4df failed: Read-only file system
Failed to re-open '/var/log/journal/remote': Operation not permitted
Setting access ACL "u::rw-,g::r-x,g:adm:r--,g:wheel:r--,g:4294967295:r-x,g:4294967295:r-x,m::r--,o::---" on /var/log/journal/deccb8485cd444ab8cea39ee0572b4df/system.journal failed: Read-only file system
fchownat() of /var/log/journal/deccb8485cd444ab8cea39ee0572b4df/system.journal failed: Read-only file system

I think it might be a good idea at this point to just disable exohub sandboxing altogether, since it's causing more and more problems.

OK, the issue still seems to be persistent. Here are some more errors I've encountered on exohub: ``` Created symlink /etc/systemd/system/graphical.target.wants/udisks2.service → /usr/lib/systemd/system/udisks2.service. Failed to send reload request: Permission denied Failed to write 'change' to '/sys/devices/LNXSYSTM:00/uevent': Permission denied ``` ``` /usr/lib/tmpfiles.d/static-nodes-permissions.conf:17: Failed to resolve group 'kvm'. fchownat() of /run/systemd/seats failed: Operation not permitted fchownat() of /run/systemd/sessions failed: Operation not permitted fchownat() of /run/systemd/users failed: Operation not permitted fchownat() of /var/lib/systemd/coredump failed: Read-only file system fchownat() of /tmp failed: Operation not permitted fchownat() of /run/media failed: Operation not permitted Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal failed: Read-only file system Failed to re-open '/var/log/journal': Operation not permitted fchownat() of /var/log/journal failed: Read-only file system Setting access ACL "u::rwx,g::r-x,g:adm:r-x,g:wheel:r-x,g:4294967295:r-x,g:4294967295:r-x,m::r-x,o::r-x" on /var/log/journal/deccb8485cd444ab8cea39ee0572b4df failed: Read-only file system Failed to re-open '/var/log/journal/deccb8485cd444ab8cea39ee0572b4df': Operation not permitted fchownat() of /var/log/journal/deccb8485cd444ab8cea39ee0572b4df failed: Read-only file system Failed to re-open '/var/log/journal/remote': Operation not permitted Setting access ACL "u::rw-,g::r-x,g:adm:r--,g:wheel:r--,g:4294967295:r-x,g:4294967295:r-x,m::r--,o::---" on /var/log/journal/deccb8485cd444ab8cea39ee0572b4df/system.journal failed: Read-only file system fchownat() of /var/log/journal/deccb8485cd444ab8cea39ee0572b4df/system.journal failed: Read-only file system ``` I think it might be a good idea at this point to just disable exohub sandboxing altogether, since it's causing more and more problems.
a reopened this issue 2022-07-29 23:52:17 +00:00
Author
Owner

I disabled the following options:

[Service]
# Disable all hardening because it interferes with Podman containers
LockPersonality=false
NoNewPrivileges=false
PrivateTmp=false
PrivateDevices=false
ProtectClock=false
ProtectControlGroups=false
ProtectHostname=false
ProtectKernelLogs=false
ProtectKernelModules=false
ProtectKernelTunables=false
ProtectSystem=false
RestrictNamespaces=false
RestrictRealtime=false
RestrictSUIDSGID=false
SystemCallArchitectures=
SystemCallErrorNumber=
SystemCallFilter=

The only sandboxing I kept enabled is ProtectProc since it seems like it needs to be enabled, and ReadWritePaths.

TL;DR exohub isn't a great environment for Distrobox, and use SSH or remote desktop instead if you can.

I disabled the following options: ``` [Service] # Disable all hardening because it interferes with Podman containers LockPersonality=false NoNewPrivileges=false PrivateTmp=false PrivateDevices=false ProtectClock=false ProtectControlGroups=false ProtectHostname=false ProtectKernelLogs=false ProtectKernelModules=false ProtectKernelTunables=false ProtectSystem=false RestrictNamespaces=false RestrictRealtime=false RestrictSUIDSGID=false SystemCallArchitectures= SystemCallErrorNumber= SystemCallFilter= ``` The only sandboxing I kept enabled is `ProtectProc` since it seems like it needs to be enabled, and `ReadWritePaths`. TL;DR exohub isn't a great environment for Distrobox, and use SSH or remote desktop instead if you can.
a closed this issue 2022-07-30 00:51:40 +00:00
Sign in to join this conversation.
No Milestone
No project
No Assignees
1 Participants
Notifications
Total Time Spent: 32 minutes 3 seconds
a
32 minutes 3 seconds
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: exozyme/exozyme#170
No description provided.