Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.
If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!
Docker image attempts chown on NFS
-
When the pod starts, there's this error message
Initializing embedded database...
chown: changing ownership of '/var/proget/database': Operation not permittedWhen inside the pod, we can see that permissions are properly set, as also verified on the NAS and via the K8 config.
ls -la /var/proget total 0 drwxr-xr-x. 7 root root 82 Aug 22 02:52 . drwxr-xr-x. 1 root root 20 Aug 22 02:52 .. drwxrwxrwx. 1 1024 users 0 Aug 22 02:49 backups drwxrwxrwx. 1 1024 users 0 Aug 22 02:54 database drwxr-xr-x. 2 root root 6 Aug 22 02:52 extensions drwxrwxrwx. 1 1024 users 0 Aug 22 02:49 packages drwxr-xr-x. 2 root root 6 Aug 22 02:52 ssl
apiVersion: apps/v1 kind: Deployment metadata: name: proget namespace: proget spec: replicas: 1 selector: matchLabels: app: proget template: metadata: labels: app: proget spec: securityContext: runAsUser: 1024 runAsGroup: 100 runAsNonRoot: false containers: - name: proget image: proget.inedo.com/productimages/inedo/proget:latest ports: - containerPort: 8624 name: http volumeMounts: - name: proget-data mountPath: /var/proget/database - name: proget-package mountPath: /var/proget/packages - name: proget-backup mountPath: /var/proget/backups volumes: - name: proget-data persistentVolumeClaim: claimName: proget-data - name: proget-package persistentVolumeClaim: claimName: proget-package - name: proget-backup persistentVolumeClaim: claimName: proget-backup
This appears to be something in the setup of ProGet that is trying to perform a chown, which is not allowable on the NFS share.
-
Hi @tyler_5201
I'm not really familiar with all this topic, but why would you want to put the database file on an NFS? The PostgreSQL docs on clustering seem to warn against this.
That said, we added the
chown
command during early testing because PostgreSQL was failing with something like this:The files belonging to this database system will be owned by user "postgres". This user must also own the server process.
The only way to get to work was:
- Log into the pod via:
$kubectl exec -n -i -t -- bash -il
- Change the directory to:
$cd /var/opt
- Get the UID of postgres-User:
$id -u postgres
- Recursively change owner of the folder database:
chown -R 101:101 database/
Hence, why we added the
chown
.I'm not sure the best approach here -- we could try/catch, but I think the
postgres
user still needs to own the files?Cheers,
Alana
- Log into the pod via:
-
I chose NFS over iSCSI and SMB because it is more standard them then, but also easier to manage than iSCSI. The shares were originally setup without squashing, but then it became a mess to keep permissions on the NAS to match the pods. It's been far easier telling a few pods to use the UID/GID rather than to manage both the deployments & the NAS permissions.
You can't
chown
within the containers, without usingsudo
which is why this fails. I can runchown
on the NAS to get the permissions to match (I'll have to test tomorrow, DHCP is being a pain on the new servers being added).For your last question, I would have assumed the UID/GID configured would have taken over for the postgres user, and would make the
chown
irrelevant at that point. I'm actually running into a similar problem with the AWX helm chart because they forcefully usechmod 755
andchown 1000
without allowing any way to bypass it. The permissions are already set, so there's no need to change them.
-
Hi @tyler_5201
Thanks for the additional info; so it's not that this is an actual "network drive" just (i.e. far from the server), it just happens to be how the configuration works.
I don't think we can/should try to detect the owner; do you if
chown
will fail if the user is alreadypostgres
? If so, then we can wrap this with a try/catch and throw a more helpful error.Otherwise, we can try/catch and give a warning. But we want to make sure the user can see it when the database fails to load.
Thanks,
Alana
-
The
chown
fails no matter what because it needs to be run assudo
.I could set the permissions on the folder, but I'd have to create a new network share because I'm mapping all Kubernetes connections to
1024:100
to keep the pods consistent. For the most part, this usually fixes any access issues to the folders, if a pod has issues.securityContext: runAsUser: 1024 runAsGroup: 100 runAsNonRoot: false
-
Hi @tyler_5201 ,
We'll address this via PG-3098 in an upcoming maintenance release; basically we will add a permission check before chown on Docker/Linux for PostgreSQL. If the permissions are there, then chown won't be run.
Thanks,
Alana