Artifacts getting removed from Database after FeedCleanup Scheduled task run

[Govind Gupta]

I had setup a new setup of Proget with datastore on S3 and it was set to replicate from another server (old setup) with datastore on disk (Windows local filesystem) using Feed Replication. I had 2 feeds to be replicated over from the old setup to the new setup which did get replicated and all the artifacts did got copied over for both the feeds after which they were in sync.

But a day after I discovered that a majority of my artifacts were not showing up on the new setup UI and the total count of packages was also 1/3rd of what it was day before. Also checked my database, the packages are also removed from it was well. But all the packages are present on the S3 bucket.
Turns out Schedule task FeedCleanup ran and identified most of my packages as orphaned and removed them from the database.

INFO: Checking for orphaned package index entries...
DEBUG: Found 133575 orphaned package entries.
DEBUG: Removing Id='abc',Version='1.0.168' from index...
DEBUG: Removing Id='abcd',Version='1.0.66' from index...

Running the FeedCleanup again was of no help and did not add the missing packages back.

I wish to understand

1. What does this FeedCleanup Schedule task does AFAIK is remove artifacts from the filesystem in case of missing metadata on DB.
2. Why were my artifacts removed in the first place from the database.
3. How do I go about restoring these artifacts since the artifacts are still present on the S3 bucket without essentially copying them again.
4. Can I disable this scheduled task to avoid such instances of package removal? Are there any side effects of disabling this task. ?

Let me know if any further details are needed from my end.
Thanks !!

Product: ProGet
Version: 4.5.3

[thoven]

Copying from associated support ticket:

It's not a bug per se, more of a design flaw with the introduction of external package stores. Basically, it checked to see if the package could be opened and if not, deleted it from the DB. However, there are many reasons a package might not be accessible (network down, invalid S3 credentials, disk inaccessible, S3 down, etc.), so this functionality was removed since then.