Inedo Community Forums Forums
    • Recent
    • Tags
    • Popular
    • Login

    Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.

    If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!

    Proget docker update from 2026.14 to 2026.15 breaks Postgres DB

    Scheduled Pinned Locked Moved Support
    2 Posts 1 Posters 2 Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • felfertF Offline
      felfert
      last edited by

      Hi, After update from 2026.14 to 2026.15 the embedded postgres does not startup anymore.

      I am running proget on Rocky9 as a RHEL-quadlet and the usual upgrade results in postgres not running. Below is a systemd log of the startup attempt after pulling the latest 2026.15 image. This is on a temporary sandbox instance which was freshly installed (with 2026.14), migrated to postgres and finally attempted to upgrade using the following commands in bash:

      systemctl stop proget.service
      podman tag proget:current proget:previous
      podman pull proget.inedo.com/productimages/inedo/proget:latest
      podman tag proget.inedo.com/productimages/inedo/proget:latest proget:current
      systemctl start proget.service
      

      The log of starting the proget.service:

      Nov 15 11:47:49 fe-pgsandbox systemd[1]: Starting Inedo proget...
      Nov 15 11:47:49 fe-pgsandbox podman[4871]: 2026-11-15 11:47:49.67945833 +0100 CET m=+0.106530950 volume create 5302ee210ff0031da105f2981924f780fcffafbd023e28480eca79969af6ef7a
      Nov 15 11:47:49 fe-pgsandbox podman[4871]: 2026-11-15 11:47:49.689362147 +0100 CET m=+0.116434767 volume create a56cbe394584572b9bb748bb4a696a8e55d8b9518a5dee4b2758c6a54fb49b3e
      Nov 15 11:47:49 fe-pgsandbox podman[4871]: 2026-11-15 11:47:49.713049861 +0100 CET m=+0.140122498 container create 3664a42997ef1a2ee01c76b76dfd28f1d47ec2f2462c9293486a60dd89daf8cd (image=localhost/proget:current, name=systemd-proget, org.opencontainers.image.version=24.04, PODMAN_SYSTEMD_UNIT=proget.service)
      Nov 15 11:47:49 fe-pgsandbox podman[4871]: 2026-11-15 11:47:49.652473761 +0100 CET m=+0.079546468 image pull 65a036b666bc930c7d0d755f07140b814e55f22498cac35de35ad5df3b19cc30 localhost/proget:current
      Nov 15 11:47:50 fe-pgsandbox podman[4871]: 2026-11-15 11:47:50.221430187 +0100 CET m=+0.648502815 container init 3664a42997ef1a2ee01c76b76dfd28f1d47ec2f2462c9293486a60dd89daf8cd (image=localhost/proget:current, name=systemd-proget, org.opencontainers.image.version=24.04, PODMAN_SYSTEMD_UNIT=proget.service)
      Nov 15 11:47:50 fe-pgsandbox systemd[1]: Started Inedo proget.
      Nov 15 11:47:50 fe-pgsandbox podman[4871]: 2026-11-15 11:47:50.238660116 +0100 CET m=+0.665732727 container start 3664a42997ef1a2ee01c76b76dfd28f1d47ec2f2462c9293486a60dd89daf8cd (image=localhost/proget:current, name=systemd-proget, PODMAN_SYSTEMD_UNIT=proget.service, org.opencontainers.image.version=24.04)
      Nov 15 11:47:50 fe-pgsandbox systemd-proget[4948]: Updating certificates in /etc/ssl/certs...
      Nov 15 11:47:50 fe-pgsandbox proget[4871]: 3664a42997ef1a2ee01c76b76dfd28f1d47ec2f2462c9293486a60dd89daf8cd
      Nov 15 11:47:50 fe-pgsandbox systemd-proget[4948]: 0 added, 0 removed; done.
      Nov 15 11:47:50 fe-pgsandbox systemd-proget[4948]: Running hooks in /etc/ca-certificates/update.d...
      Nov 15 11:47:50 fe-pgsandbox systemd-proget[4948]: done.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Application started. Press Ctrl+C to shut down.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Hosting environment: Production
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Content root path: /usr/local/proget
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: ProGet version is 26.0.15.14.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: Current DB schema version is unknown.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: Updating database schema...
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Application started. Press Ctrl+C to shut down.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Hosting environment: Production
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Content root path: /usr/local/proget
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: Attempting to use shared configuration for ProGet.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: Shared configuration file not found at /etc/inedo/ProGet.config.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: No encryption key is configured. Credentials will be stored in plain text.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Inedo.Web.BackgroundTaskQueueService[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Background Task Queue is starting.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: warn: Microsoft.AspNetCore.Hosting.Diagnostics[15]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Overriding HTTP_PORTS '8080' and HTTPS_PORTS ''. Binding to values defined by URLS instead 'http://*:80'.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: warn: Microsoft.AspNetCore.Server.Kestrel[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Overriding address(es) 'http://*:80'. Binding to endpoints defined via IConfiguration and/or UseKestrel() instead.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[14]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Now listening on: http://[::]:80
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Application started. Press Ctrl+C to shut down.
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Hosting environment: Production
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]: info: Microsoft.Hosting.Lifetime[0]
      Nov 15 11:47:51 fe-pgsandbox systemd-proget[4948]:       Content root path: /usr/local/proget
      Nov 15 11:47:52 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      Nov 15 11:47:53 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      Nov 15 11:47:54 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      Nov 15 11:47:55 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      Nov 15 11:47:56 fe-pgsandbox systemd-proget[4948]: Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
      

      After rollback (Stopping the container and re-tagging the previous image, Starting container)
      everything was up and running again (with the old version of course).

      Thanks
      -Fritz

      felfertF 1 Reply Last reply Reply Quote
      • felfertF Offline
        felfert @felfert
        last edited by felfert

        Investigating myself, I stumbled over one significant change in the new container:

        The owner of the hierarchy in /var/proget/database has changed in the updated container:

        Old (2026.14):

        root@proget:/usr/local/proget# ls -la /var/proget/database/
        total 160
        drwx------. 20 postgres root      4096 Nov 15 09:11 .
        drwxr-xr-t.  8 root     root        94 Nov 15 09:11 ..
        -rw-r--r--.  1 root     root        98 Jun 21  2025 .pgsqlconn
        -rw-------.  1 postgres      104     3 Jun 21  2025 PG_VERSION
        drwx------.  7 postgres      104  4096 Aug 20  2025 base
        -rw-------.  1 postgres postgres    44 Nov 15 09:11 current_logfiles
        drwx------.  2 postgres      104 16384 Nov 15 09:11 global
        drwx------.  2 postgres postgres 12288 Nov 15 09:11 log
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_commit_ts
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_dynshmem
        -rw-------.  1 postgres      104   206 Aug 29  2025 pg_hba.conf
        -rw-------.  1 postgres      104  2640 Jun 21  2025 pg_ident.conf
        drwx------.  4 postgres      104  4096 Nov 15 12:46 pg_logical
        drwx------.  4 postgres      104  4096 Jun 21  2025 pg_multixact
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_notify
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_replslot
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_serial
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_snapshots
        drwx------.  2 postgres      104  4096 Nov 15 09:11 pg_stat
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_stat_tmp
        drwx------.  2 postgres      104  4096 Nov 15 00:50 pg_subtrans
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_tblspc
        drwx------.  2 postgres      104  4096 Jun 21  2025 pg_twophase
        drwx------.  4 postgres      104  4096 Nov 15 09:17 pg_wal
        drwx------.  2 postgres      104  4096 Oct  5 10:51 pg_xact
        -rw-------.  1 postgres      104    88 Jun 21  2025 postgresql.auto.conf
        -rw-------.  1 postgres      104 30875 Jul 31 14:28 postgresql.conf
        -rw-------.  1 postgres      104    64 Nov 15 09:11 postmaster.opts
        -rw-------.  1 postgres postgres   100 Nov 15 09:11 postmaster.pid
        

        New (2026.15):

        root@proget:/usr/local/proget# ls -la /var/proget/database/
        total 68
        drwx------. 20  101 root  4096 Nov 15 11:47 .
        drwxr-xr-t.  8 root root    94 Nov 15 11:47 ..
        -rw-r--r--.  1 root root    98 Nov 15 11:18 .pgsqlconn
        -rw-------.  1  101  103     3 Nov 15 11:18 PG_VERSION
        drwx------.  6  101  103    46 Nov 15 11:18 base
        -rw-------.  1  101  103    44 Nov 15 11:18 current_logfiles
        drwx------.  2  101  103  4096 Nov 15 11:22 global
        drwx------.  2  101  103    46 Nov 15 11:18 log
        drwx------.  2  101  103     6 Nov 15 11:18 pg_commit_ts
        drwx------.  2  101  103     6 Nov 15 11:18 pg_dynshmem
        -rw-------.  1  101  103   206 Nov 15 11:18 pg_hba.conf
        -rw-------.  1  101  103  2640 Nov 15 11:18 pg_ident.conf
        drwx------.  4  101  103    68 Nov 15 11:32 pg_logical
        drwx------.  4  101  103    36 Nov 15 11:18 pg_multixact
        drwx------.  2  101  103     6 Nov 15 11:18 pg_notify
        drwx------.  2  101  103     6 Nov 15 11:18 pg_replslot
        drwx------.  2  101  103     6 Nov 15 11:18 pg_serial
        drwx------.  2  101  103     6 Nov 15 11:18 pg_snapshots
        drwx------.  2  101  103    25 Nov 15 11:32 pg_stat
        drwx------.  2  101  103     6 Nov 15 11:18 pg_stat_tmp
        drwx------.  2  101  103    18 Nov 15 11:18 pg_subtrans
        drwx------.  2  101  103     6 Nov 15 11:18 pg_tblspc
        drwx------.  2  101  103     6 Nov 15 11:18 pg_twophase
        drwx------.  4  101  103   109 Nov 15 11:27 pg_wal
        drwx------.  2  101  103    18 Nov 15 11:18 pg_xact
        -rw-------.  1  101  103    88 Nov 15 11:18 postgresql.auto.conf
        -rw-------.  1  101  103 31163 Nov 15 11:18 postgresql.conf
        -rw-------.  1  101  103    64 Nov 15 11:18 postmaster.opts
        

        Note: the uid:gid of the postgres user has changed from 101:103 to 994:994

        Attempt to show this in a code snippet was denied for security reasons by the forum.
        Apparently just mentioning the files path in /etc/ that stores these values, triggers a security alert of the forum so I had to obfuscate it a little bit:

        Old:

        root@proget:/usr/local/proget# grep postgres /etc/p*wd
        postgres:x:101:103:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash
        

        New:

        grep postgres /etc/p*wd
        postgres:x:994:994:PostgreSQL administrator:/var/lib/postgresql:/bin/bash
        

        And success after manually updating the owner in the container.
        I just ran the following inside the container:

        chown -R postgres /var/proget/database
        

        After a container restart, it worked again.

        => I vaguely remember from the early releases with embedded postgres,
        that there was some permission problem too. So: problem found - fix pending (i hope 😊 )

        Cheers
        -Fritz

        1 Reply Last reply Reply Quote

        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

        With your input, this post could be even better 💗

        Register Login
        • 1 / 1
        • First post
          Last post
        Inedo Website Home • Support Home • Code of Conduct • Forums Guide • Documentation