Hi @jolaka9284_9458,

Thanks for providing all these details. I was able to pinpoint the reason for the slow down and why it specifically happens on some crates and not others. This is related to some code that we have to determine when to point cargo to pull dependencies from ProGet vs crates.io. Specifically this comes from cargo's API specs for dependencies:

registry — cargo metadata uses a value of null to indicate that the dependency comes from crates.io. The index uses a value of null to indicate that the dependency comes from the same registry as the index. When creating an index entry, a registry other than crates.io should translate a value of null to be https://github.com/rust-lang/crates.io-index and translate a URL that matches the current index to be null.

As you can see, the value specified in the metadata is different than the value the index needs to return. In ProGet, we will return null if the package exists in the feed (including connectors) and https://github.com/rust-lang/crates.io-index if it does not. This is to support the case when ProGet is not used as a mirror and instead for only local crates. Unfortunately the use ProGet as a mirror option is stored only in the client config and is not sent to ProGet.

This is the reason why crates with a lot of dependencies take longer to generate the index than ones that don't and why you'll occasionally get timeouts, but after the retry it works. We have some caching on this to help with performance, but it's not a forever cache.

I'm going to work on some potential improvements for this and will let you know when I have a solution ready. Unfortunately, the only workaround we have for this currently is to use a package approval workflow (like our npm Package Approval blog article).

Thanks,
Rich

Hi @gravufo,

Would you be able to rerun the database scripts on your database? You will just need to run the Run inedosql to update the database step of our manual install guide. Can you see if this fixes your issue?

Thanks,
Rich

Hi @viceice,

That error is safe to ignore. It is currently a known bug and we are looking to fix that in an upcoming version of ProGet. The ticket tracking the fix for the log message is PG-1841.

Long story short, the ProGet service correctly detected that the product wasn't activated, and then logged that message. But it was doing it every time it accessed license information, which is on every connector health check, replication run, etc.
Activation happens automatically as soon as someone visits the Web application, and re-activation is required after upgrading certain versions.

Thanks,
Rich

Hi @mcascone ,

We have fixed the issue and the published date will now update when the package is overwritten. This will be released in ProGet 5.3.9 which is due out this Friday Augst 14, 2020.

Thanks,
Rich

Hi @viceice,

Thanks for the clarification on your environment! I see what is going on now. I have created a ticket, PG-1809, to track the fix for this. We expect this to be released in ProGet 5.3.11 which we are expecting to be released in September 11, 2020. Basically in that instance, we are not respecting the values within the X-Forwarded-* headers. I'll let you know if anything changes on the timeline.

Thanks,
Rich

Hi @markus4830,

I'm definitely sorry about this. The change was made to help to aide in improvements to other areas of the system related to NuGet. Unfortunately, it looks like it affected the NuGet API. Expect a more permanent solution in the near future.

Thanks,
Rich

Hi @nuno-guerreiro-rosa_9280,

We have finally been able to recreate this issue in our sandbox. We are currently looking into a fix, but we expect to have one in the next version of ProGet, 5.3.10. This looks to be an issue with the mono framework. We use mono runtime in our Docker images for ProGet. We are also going to be releasing a .Net Core based technical preview of ProGet Docker in version 5.3.10. This will be in addition to our standard mono based version. Our internal testing is going very well and it looks to have removed a lot of the gotchas that mono has.

Thanks,
Rich

Hi @harald-somnes-hanssen_2204,

The fix is scheduled for release in ProGet 5.3.16 which is due out on Friday. I'll reply back if anything changes.

Thanks,
Rich

Hi @tkolb_7784,

If you are hosting on a windows machine, the easiest solution right now is to migrate your server to use IIS and then add an SSL binding to your site. If you do not want to purchase a new certificate and the self-signed certificate is too much work, you can use Let's Encrypt and configure it via winacme.

If you do not want to use IIS, then you will need to use a reverse proxy to handle SSL connections. Any reverse proxy can be used and a pretty simple one to configure is stunnel. Most reverse proxies can also be used with Let's Encrypt.

If you are hosting via Linux (Docker), then you will need to use a reverse proxy to handle SSL connections. We have a documentation page for different Linux-based reverse proxies including an example for setting up NGINX. These reverse proxies also support Let's Encrypt also.

Please let me know if you have any questions.

Thanks,
Rich

Hi @gravufo,

Great! Glad to hear it! Please post back if you find anything else.

I also recommend that you switch to the Inedo Hub in the future. We are in the process of deprecating our traditional installer. The Inedo Hub has the ability to update an installation previously installed with the traditional installer and the Inedo Hub now supports offline installations as well, if you need that functionality.

Thanks,
Rich

Hi @kquinn_2909,

I'm working to recreate this issue, but I'm struggling to recreate it. Would you be able to send us the results of the following API?

app/rest/projects/WebProjects_Replicator?fields=buildTypes(buildType)

The thought I currently have is that the Build Type list is returning something different that is filtering out those builds (https://github.com/Inedo/inedox-teamcity/blob/7d447a5f4f3c3e38c98012c65e9db40afec224b6/TeamCity/InedoExtension/TeamCityClient.cs#L84)

Thanks,
Rich

Hi @koksime-yap_5909,

After looking into this further, although we can fix the import to handle the images without a media type, it will still be broken when you attempt to pull those images using the Docker client. This is because ProGet will only send the manifest as it is stored in ProGet, where Nexus will manipulate it to match the accept headers of the client. Since it is only returning the manifest as it is stored, the media type will be null which will cause the Docker client to fail with a missing or empty Content-Type header.

Since we cannot assume what the mediaType should be, I think it would be better to resolve those prior to importing into ProGet. The nice thing is you can use the import logs to identify what images are missing the media type. You can run the import multiple times and it will only import the missing images and layers.

Thanks,
Rich

Hi @koksime-yap_5909,

I apologize for the delay. This is next on my list to review and I will have an update for you on Monday.

Thanks,
Rich

Hi @udi-moshe_0021,

Would you be able to provide the Installation logs from a failed install? You can get them by rerunning the Offline installer and then clicking on the Logs tab.

The other ting I would try doing is try running ProGet from the command line and then sharing the output with us. That may tells us any startup issues the application has. To do that, open a command prompt, navigate to the install directory (cd "C:\Program Files\ProGet\Service"), and then run proget.exe run. Could you share the output from that?

Thanks,
Rich

Hi @henderkes,

No problem! Both these issues are expected to be fixed in this Friday's release of ProGet 2025.18.

Thanks,
Rich

Hi @henderkes,

So we have identified the issue. When it comes to your packages, there where two issues that were preventing them from being updated and installed.

1. The packages you provided to me are unsigned. According to the APK documentation, that is not supported. In reality, that is a case that they support and have supported for some time. So we have a ticket, PG-3192, to correct the checksum generation for unsigned packages.
2. When a checksum is not generated, ProGet was inserting the hash of the package instead, which would cause the index to fail.
   • In older versions of APK tools V2, this would fail the install of that package, but show you the all packages in the index.
   • In newer versions of APK Tools V2, it will stop processing once a package with a bad checksum is shown. Which is why it looked like only one package existed in the feed. Installing these packages will most likely fail with a checksum error.
   • In APK Tools v3, it errors on apk update and these packages cannot be installed at all.

We are going to be adding the checksum verification to the Feed integrity checks and a way to regenerate the checksums in ticket PG-3193. That should resolve all these issues for you. That also explains why my test cases worked fine, because most public index packages are signed packages.

Thanks,
Rich

Hi @udi-moshe_0021,

Our Windows test servers are a vanilla install of Server Standard (we test against a few different versions) with nothing but the defaults configured and the latest Microsoft updates applied. It is possible that specifying a different user could cause it. Just make sure that the user you specified has read, write, and modify permissions to the Config File Directory (typically C:\ProgramData\Inedo). Upon first start of the ProGet Service, it will create and setup the embedded database and update the ProGet.config file. Based on error, that looks to be where it is failing. I wonder if you are missing the modify permission on that directory and it is failing to update the ProGet.config file.

Thanks,
Rich

Thanks,
Rich

Hi @henderkes,

Sorry for the delay on this. Right now, there looks to be a couple issues I'm working through with this. I should have an update for you on Friday. I believe I have identified the issues, but I'm trying to confirm some things based on your APK packages. Just to confirm, the packages you shared are APK Tools V2 packages correct?

Thanks,
Rich

Hi @koksime-yap_5909,

Thanks for sharing the steps. I'm going to attempt to reproduce this and I'll let you know what i find!

Thanks,
Rich

Hi @udi-moshe_0021,

I just ran some tests with both 2025.16 and 2025.17 using the embedded database on a clean install of Windows Server 2022 and everything installed and ran as expected. The only other thing I could think of could be a permissions issue. What account did you have ProGet service running as?

Thanks,
Rich