Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.

If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!

ProGet Docker built in database fails after restart



  • Hello,

    I set up ProGet using Docker and nginx as a reverse proxy. When I start the container, everything works. But if I remove the container and start it again, it complains that it cannot connect to the database.

    This is a clean install using the latest version of the container image.

    # docker logs -f proget
    Updating certificates in /etc/ssl/certs...
    0 added, 0 removed; done.
    Running hooks in /etc/ca-certificates/update.d...
    done.
    info: Microsoft.Hosting.Lifetime[0]
          Application started. Press Ctrl+C to shut down.
    info: Microsoft.Hosting.Lifetime[0]
          Hosting environment: Production
    info: Microsoft.Hosting.Lifetime[0]
          Content root path: /usr/local/proget
    ProGet version is 25.0.5.16.
    Current DB schema version is unknown.
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    Attempting to use shared configuration for ProGet.
    Shared configuration file not found at /etc/inedo/ProGet.config.
    info: Inedo.Web.BackgroundTaskQueueService[0]
          Background Task Queue is starting.
    warn: Microsoft.AspNetCore.Hosting.Diagnostics[15]
          Overriding HTTP_PORTS '8080' and HTTPS_PORTS ''. Binding to values defined by URLS instead 'http://*:80'.
    warn: Microsoft.AspNetCore.Server.Kestrel[0]
          Overriding address(es) 'http://*:80'. Binding to endpoints defined via IConfiguration and/or UseKestrel() instead.
    info: Microsoft.Hosting.Lifetime[0]
          Application started. Press Ctrl+C to shut down.
    info: Microsoft.Hosting.Lifetime[0]
          Hosting environment: Production
    info: Microsoft.Hosting.Lifetime[0]
          Content root path: /usr/local/proget
    info: Microsoft.Hosting.Lifetime[14]
          Now listening on: http://[::]:80
    info: Microsoft.Hosting.Lifetime[0]
          Application started. Press Ctrl+C to shut down.
    info: Microsoft.Hosting.Lifetime[0]
          Hosting environment: Production
    info: Microsoft.Hosting.Lifetime[0]
          Content root path: /usr/local/proget
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    Cannot connect to database; will retry in 1 second... Full error: Failed to connect to 127.0.0.1:5728
    

    Here is my start command:

    #!/bin/bash
    
    /usr/bin/docker run -d --name=proget --restart=unless-stopped \
            -v /opt/proget/packages:/var/proget/packages \
            -v /opt/proget/database:/var/proget/database \
            -v /opt/proget/backups:/var/proget/backups \
            -v /opt/proget/encryption.key:/var/proget/encryption.key \
            -v /opt/proget/connection_string.txt:/var/proget/connection_string.txt \
            -p 8624:80 \
            -e TZ='Asia/Qatar' \
            -e PROGET_ENCRYPTION_KEY_FILE='/var/proget/encryption.key' \
            -e PROGET_POSTGRES_CONNECTION_STRING_FILE='/var/proget/connection_string.txt' \
            proget.inedo.com/productimages/inedo/proget:latest
    

    From the instructions it seems you only need the "PROGET_POSTGRES_CONNECTION_STRING_FILE" if you are using an external database, but the web interface complains about not having the connection string if I don't put it.

    Am I missing something?



  • I removed the connection string to try again. This time things are fine if I stop, remove, and run again. However on reboot, it goes back to not working again.

    The web interface shows the following:

    bae76a24-6247-467e-b7b8-5bd2d62b0ee8-image.png


  • inedo-engineer

    Hi @tames_0545 ,

    A connection string should not be specified.

    I haven't seen this behavior before; the container runs postgres on start to listen to port 5728 and I can't imagine how that would fail without an error message.

    Perhaps there's some kind of issue with /opt/proget/database and that's causing the postgres process to crash; I would start here.

    The only other thing I can imagine is some kind of security configuration that is interfering with the running container.

    I would keep it as simple as possible. Once you get it working, you can try adding in the encryption key. Unless you absolutely need to use a Docker secret (those are complex and don't really do anything), just specify the encryption key with an environment variable

    -e PROGET_ENCRYPTION_KEY='37D27A670394F7D82CE57F1F07D69747'
    

    Cheers,
    Alana



  • I have removed the encryption key and connection string. I removed the proget directories and started again.

    Things ran well until reboot. After reboot, the web was available but it started throwing the following message:

    info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
          Request starting HTTP/1.1 GET http://proget.example.com/feeds - - -
    fail: Microsoft.AspNetCore.Server.Kestrel[13]
          Connection id "0HNE94OV39U9F", Request id "0HNE94OV39U9F:00000001": An unhandled exception was thrown by the application.
          Inedo.ProGet.Web.Security.UserNotFoundException: Exception of type 'Inedo.ProGet.Web.Security.UserNotFoundException' was thrown.
             at Inedo.ProGet.WebApplication.ProGetHttpModule.AuthorizeRequestAsync(AhHttpApplication app) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E576295\Src\src\ProGet\WebApplication\ProGetHttpModule.cs:line 264
             at Inedo.Web.InedoHttpModule.Inedo.Web.IAhWebModule.AuthorizeRequestAsync(AhHttpApplication app)
             at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
             at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
             at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
             at Inedo.Web.AhWebHost.<>c.<<Configure>b__22_0>d.MoveNext()
          --- End of stack trace from previous location ---
             at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
    info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
          Request finished HTTP/1.1 GET http://proget.example.com/feeds - 500 0 - 3.7097ms
    

    Eventually the web server stopped responding.

    It seems completely unrelated to what I was seeing before. However, it shows something is not stable.


  • inedo-engineer

    Hi @tames_0545 ,

    Sorry, no idea -- there is definitely something wrong with your environment but aside from the advice I gave earlier, I don't have any other troubleshooting steps. You may want to try a different environment / Docker host.

    As I mentioned, it could something wrong with your database mount (/opt/proget/database) -- maybe that is getting deleted? Or it could be your security configuration that is interfering with the running container. We have seen some hyper-aggressive security tools that make things so secure nothing runs.

    Thanks,
    Alana


  • inedo-engineer

    @tames_0545 I just thought of something, perhaps try a different versions of ProGet, instead of latest -- like maybe 25.0.3, 25.0.4, etc, perhaps into ProGet 2024 as well



  • I figured it out. The container is changing the /var/proget/database directory ownership from root to the internal postgres user. I had created the directories using Puppet, which was putting them back to root as the owner.

    Thanks for the hints.


  • inedo-engineer

    @tames_0545 thanks for letting us know! We'll see if we can improve troubleshooting on this in the future as well!


Log in to reply
 

Inedo Website HomeSupport HomeCode of ConductForums GuideDocumentation