upack push return (403) Forbidden without Feeds_OverwritePackage permission

philipp.jenni_7195

In release 2022.15 i become an error 403 forbidden when i will push a new universalpackage to the proget server.
The user has the follow permssions:

Feeds_AddPackage
Feeds_DownloadPackage
Feeds_PullPackage
Feeds_ViewFeed

The Version is not existing in the feed!

when i add the permission "Feeds_OverwritePackage" to the user, it works fine.

It has the permission requirements changed or is it a bug?

I use the Docker Container.

atripp

Hi @philipp-jenni_7195,

We have seen a few edge cases that will cause this behavior:

if the package version is "unlisted"
if the package file already exists on disk, but no database record exists

In this case, if you should be able to upload the package from the UI, then delete it, then it should work again.

Cheers,
Alana

philipp.jenni_7195

Hi Alana,

I have checked this and uploaded a new version where never exisits and it also not working.

atripp

Hi @philipp-jenni_7195 ,

I'm afraid I can't reproduce this, and we've reviewed this code already. Only packages the exist will require Feeds_OverwritePackage privilege.

If you can provide me with step-by-step guide using a New Feed and a package file to use, I will try your steps.

The steps should basically be

Create New Feed named XXX
Upload this file to the feed

Cheers,
Alana

philipp.jenni_7195

Hi,

I have tested the problem again and can reproduce it even with a new feed.

The upload via the UI works in any case. The upload via the UPack.exe works only as long as in the table [Feeds] in the field [FeedConfiguration_Xml] the property FeedUsage is set to value Mixed. If the value is PrivateOnly it does not work anymore. The value changes as soon as you save the type once in the feed configuration.

rhessinger

Hi @philipp-jenni_7195,

Can you please tell me what version of the upack CLI you are using? You can find that by running the command upack version.

I tested by creating a new Universal Packages feed with a feed usage type of PrivateOnly and the upack CLI 3.0.1.3 and I was able to push without issue. Can you also please answer a couple of other things for me?

What is the upack CLI command you are running to push your package?
Are you using an API key? If so, what permissions do you have configured on the API key?

Thanks,
Rich

philipp.jenni_7195

Version: upack 2.2.5.15

Command: upack puch c:\tmp\database-schema-7.60.0.upack https://nuget.exanic.local/upack/application --user=user:password

I doesn't use the API Key

I use Build-In Users. The User is an Group, and the Group has the Publish Package Permission for all Feeds

rhessinger

Hi @philipp-jenni_7195,

Can you please try upgrading your upack CLI to the latest version and see if that resolves the problem for you?

Thanks,
Rich

philipp.jenni_7195

I have tried with V.3.1 of upack. Same result. Upack returns: Inedo.UPack.Net.UniversalFeedException: 403 Forbidden: The Feeds_OverwritePackage task is required to perform this action.

atripp

Hi @philipp-jenni_7195 ,

This is still a mystery to us; since you say you can reproduce it with a new feed, can you provide us with a step-by-step guide using a New Feed and a package file to use, then we will try your steps.

The steps should basically be:

Create New Feed named XXX
Upload this file to the feed (please attach file)

If the server is publicly accessible (or you can create one that is), we can also log-in and attempt to reproduce it on your server.

Cheers,
Alana

philipp.jenni_7195

The server is not public, but i can send you a database backup and the docker config.

atripp

As a free user, we're a bit limited to how much we can investigate this, and it's unlikely we'd be able to see anything from the database or docker config. We've seen this message come-up from time-to-time, and it's been related to the two edge cases I mentioned:

package version is "unlisted"
package file already exists on disk, but no database record exists

We have yet to see this happen with a newly-created feed.

Perhaps you can set-up something on AWS LightSail, or another very inexpensive hosting platform? Using that, we could investigate much easier.

philipp.jenni_7195

I have tried it with another installation and have exactly the same problem there. No matter with which upack I tested.

With the following case the problem should be reproduced.

Proget: 2022.18 (Docker version)
UPack: 3.0.1

Create an API key which only has the permission "View/Download" and "Add/Package" on the application feed. The feed type is "Feed
Create UPack with new version
Push the package with the following command upack.exe push mypackage-3.0.0.upack http://myserver/upack/jnetwork-applicaton/ --user=api:XAPIKEYXXXXXXXXXXXXXXXXXXXXXX
Create UPack with new version
Push the package with the following command upack.exe push mypackage-3.0.1.upack http://myserver/upack/jnetwork-applicaton/ --user=api:XAPIKEYXXXXXXXXXXXXXXXXXXXXXX

The first upload works, but the second upload with an new version to an exisiting package will fail. UPack then gives me the following error:

Unhandled exception. Inedo.UPack.Net.UniversalFeedException: 403 Forbidden: The Feeds_OverwritePackage task is required to perform this action.

In one place it worked for me, but then I found out that the API key was mapped to a user who was in the administrators group.

atripp

Hi @philipp-jenni_7195 ,

Hmmm that's basically the same thing I tried a while back ....

Can email/send the package files in this example? mypackage-3.0.0.upack and mypackage-3.0.1.upack

If you email them to support at inedo dot com, with a subject of [QA-976], we will be able to find it. Please let me know when you email it, since we don't watch that box

Cheers,
Alana

philipp.jenni_7195

I have sent you screenshot of my config and 2 upacks.

rhessinger

Hi @philipp-jenni_7195,

I can confirm that we have received your email. Please give us a bit of time to review it and we will get back to you soon!

Thanks,
Rich

rhessinger

Hi @philipp-jenni_7195,

I took a look at your example and I was able to push the packages to my feed without issue. If I tried to push the package a second time, I then I would get that Feeds_OverwritePackage" error, but that is expected.

Are you getting this error when you try to push it initially? Are there other packages in the feed? If there are other packages in the feed, are there any with the same name but a different group name?

Thanks,
Rich

philipp.jenni_7195

Hi Rich,

I can reproduce this on 2 different systems with the same case over and over again.

create new feed
create API key for this feed
push package with the smaller version. This works.
create package with the bigger version. This gives an error

Both systems are based on a Docker container (Linux) with MSSQL.

Is there any way to generate a detailed log that I can send to you?

rhessinger

Hi @nselezneva_7646,

Am I correct that you had that exact issue with the example packages you sent over to us? Would you be able to send me just the upack,json files from the packages you are testing with?

Thanks,
Rich

philipp.jenni_7195

Yes. That is correct. I can it reproduce with any package.

rhessinger

Hi @philipp-jenni_7195,

I'm sorry, but I'm really struggling to recreate this.

The test I setup was:

Docker run ProGet 2022.18
- I also tested on ProGet 2022.15, but this one was on Windows. Functionally they do not differ in upack feed code.
Create a new upack feed
- No connectors or other packages
upack push database-schema-7.60.0.upack to the new feed
upack push database-schema-7.61.0.upack to the new feed

That all worked. I then proceeded to do different combinations like push 7.61, then 7.60. The only time I was able to get the "Feeds_OverwritePackage" error was when the package already existed in the feed and I tried to push it again (which works as designed).

Do you have anything sitting in front of your Docker image (like a reverse proxy or HTTP request forwarder) that may attempt to resend a request if it takes too long? The only thing that makes sense to me would be the push request is being duplicated and the first succeeds and the second errors because the package already exists.

Thanks,
Rich

philipp.jenni_7195

Here the steps how you can reproduce:

Create a docker-compose file:

services:
  proget:
	container_name: proget_proget
	deploy:
	  replicas: 1
	environment:
	  SQL_CONNECTION_STRING: "Data Source=sql-server-db; Initial Catalog=ProGet; User ID=sa; Password=my-p8ss53rd"
	expose:
	- 81
	image: proget.inedo.com/productimages/inedo/proget:latest
	labels:
	  generated: by 8gwifi.org
	ports:
	- '81:80'
  sql-server-db:
	container_name: sql-server-db
	image: mcr.microsoft.com/mssql/server:2019-latest
	ports:
	  - "1433:1433"
	environment:
	  SA_PASSWORD: "my-p8ss53rd"
	  ACCEPT_EULA: "Y"
version: '3'
```

Run "docker-compose up"
Connect to the SQL Server with SA and create a db "ProGet"
Run "docker-compose down"
Run "docker-compose up
Connect with browser to "http://localhost:81
Login with "admin/admin"
Enter a "Free" Lizence Key
Create a Universal Package feed "Test"
Create an API Key with Type "Feed" and Permission "View/Download" anad "Add/Repackage" for all Feeds
Download upack.exe from https://github.com/Inedo/upack/releases
Execute upack.exe push database-schema-7.60.0.upack http://localhost:81/upack/Test/ --user=api:<Your generated API>
Execute upack.exe push database-schema-7.61.0.upack http://localhost:81/upack/Test/ --user=api:<Your generated API>

On the second upack command you should receive the error.

I was just able to reproduce the error with this procedure on Windows 10 and Docker Desktop.

philipp.jenni_7195

On Docker output i see follow output:

Running Execution Dispatcher...
Running Execution Dispatcher...
Checking for executions to dispatch...
Checking for executions to dispatch...
Execution Dispatcher completed.
Execution Dispatcher completed.
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 PUT http://localhost:81/upack/Test/upload application/zip 2338404
A 403 error occurred in Test: The Feeds_OverwritePackage task is required to perform this action.
A 403 error occurred in Test: The Feeds_OverwritePackage task is required to perform this action.
An error occurred processing a PUT request to http://localhost:81/upack/Test/upload: The Feeds_OverwritePackage task is required to perform this action.
An error occurred processing a PUT request to http://localhost:81/upack/Test/upload: The Feeds_OverwritePackage task is required to perform this action.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 PUT http://localhost:81/upack/Test/upload application/zip 2338404 - 403 67 - 34.9552ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 POST http://localhost:81/0x44/ProGet.WebApplication/Inedo.ProGet.WebApplication.Controls.Layout.NotificationBar/GetNotifications - 0
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 POST http://localhost:81/0x44/ProGet.WebApplication/Inedo.ProGet.WebApplication.Controls.Layout.NotificationBar/GetNotifications - 0 - 200 30 - 2.3018ms
Running Feed Replication...
Running Feed Replication...
Feed Replication completed.
Feed Replication completed.
Running Execution Dispatcher...
Running Execution Dispatcher...
Checking for executions to dispatch...
Checking for executions to dispatch...
Execution Dispatcher completed.
Execution Dispatcher completed.

rhessinger

Hi @philipp-jenni_7195,

Thank you very much for the steps to recreate this issue. I was finally able to recreate it and it looks to be easier to recreate on Linux than on Windows. We have created a ticket, PG-2274, to track the fix. We should be able to get this into today's release!

Thanks,
Rich

philipp.jenni_7195

I have tested your latest release. Now i can't upload any file. You can reproduce it with the same steps, but now the upload of 7.60.0 will fail.

rhessinger

Hi @philipp-jenni_7195,

It looks like there was a regression applied to that fix. We have fixed it as part of PG-2288, due in a couple of weeks in ProGet v2022.22. If you would like, I can set up a CI release for you that you can install now using Inedo Hub.

Thanks,
Rich

philipp.jenni_7195

I wait for the next official release. i have a workaround with disable the permission