Error uploading asset via API - Access to the path '/var/proget/packages/.assets/dir' is denied.

[jeff.peirson_4344]

I'm using the asset content API endpoint to post a file to an asset directory. This has been working, but stopped working around 3 Jan 2023 for some reason. I'm not sure what changed around that time, but I suspect we upgraded our ProGet installation to the latest 2022 patch (would have been a patch upgrade only - we've been using v2022 for several months).

I'm using powershell to send the post request, which looks something like:

Invoke-WebRequest -ErrorAction Stop -Uri 'https://proget/endpoints/my-asset-feed/content/path/to' -Method POST -Headers @{'X-ApiKey' = 'XXX'} -InFile 'myfile.json' -ContentType 'application/json'

I get a 500 error response back with:

"Access to the path '/var/proget/packages/.assets/F19/path/to' is denied."

What's weird is that I can upload the file manually through the ProGet feed page from my browser, and can create and delete folders with no issues.

I'm running ProGet as a docker container, with the 2022.17 image from your registry. The packages are mounted as an external volume into /var/proget/packages. I can log into the docker container and verify the path exists. The directory attributes for the asset are drwxr-xr-x all the way up, the file is -rw-r--r-- and the file and directories are all owned by root:root. As far as I can tell, the ProGet service itself is running as root within the container.

Is this an known issue within ProGet? or could it be something with my environment?

[atripp]

Hi @jeff-peirson_4344,

I don't think there's been any changes to the Asset Directory in v2022 that would have caused anything like this.

Do you have a full stack trace of that message? It sounds like an operating system error, and if that's the case - it wouldn't make a lot of sense that the same action works from the Web UI, since it's the same code. So that might mean it's probably unrelated to ProGet, and some weird environmental thing.

It could also be an "error handling an error" that yields an incorrect message, but we'd want to see the full stack trace for that.

Cheers,
Alana

[jeff.peirson_4344]

So I have an nginx reverse proxy set up as part of my docker stack, which includes other software dev services besides ProGet. I did some more troubleshooting, and if I issue the same HTTP request via curl, I get the same error as with powershell, so powershell doesn't seem to be a factor. But if I issue the curl request from another container inside the docker stack so that the request isn't going through the reverse proxy, it works fine.

I haven't made any changes to the nginx container or configuration, and it's been running for over a month, well before I started having this issue, so I don't think any nginx config change is to blame. But the reverse proxy is a factor somehow.

My nginx config fragment for the proget container looks like:

server {
        server_name myserver.com;
        listen 8096 ssl http2 default_server;
        access_log /var/log/nginx/access.log vhost;
        ssl_session_timeout 5m;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_certificate /etc/nginx/certs/myserver.com.crt;
        ssl_certificate_key /etc/nginx/certs/myserver.com.key;
        add_header Strict-Transport-Security "max-age=31536000" always;

        location / {
                proxy_pass http://proget:8095/;
        }
}

Other env details: I'm accessing the nginx proxy over https on port 8096, and nginx forwards the request to the proget container on non-ssl port 8095. I read that there are issues with using any kind of URL rewrite with ProGet, so aside from the port number, there's no URL path changing taking place.

Other nginx config if it's relevant:

client_max_body_size 0;
chunked_transfer_encoding on;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
proxy_set_header X-Original-URI $request_uri;
proxy_set_header Proxy "";
proxy_buffering			on;
proxy_buffers			128 4m;
proxy_busy_buffers_size	256m;
proxy_buffer_size		16m;
proxy_max_temp_file_size 0;
client_body_buffer_size	4m;
proxy_connect_timeout	300;
proxy_read_timeout		3000;
proxy_send_timeout		300;
proxy_intercept_errors	off;

and if it's relevant I'm using jwilder/nginx-proxy:1.0 as the proxy image.

Does ProGet have more detailed logs somewhere? The error isn't being logged anywhere I can find (API key access logs, Diag Center log, ProGet Event Log, stdout/stderr from the running ProGet docker container). The "access denied" error I posted before is the literal body of the response, with no other info.

And does any of my nginx config stand out as having problems with ProGet?

[jeff.peirson_4344]

Bah - turned out to be user error. I was posting the file to https://server/endpoints/myfeed/content/dir instead of https://server/endpoints/myfeed/content/dir/. I suppose ProGet could better handle this error case with a non-500 error response, but in any case I've fixed my own problem.

[atripp]

Thanks for letting us know @jeff-peirson_4344 ; we'll see if we can better detect and improve this error.