Posts made by atripp

Hi @jan_ko ,

Thanks for sharing this; the .nuspec file looks okay to me, so it doesn't seem like a problem with the package.

After building the agent pushed these nupkg's to the Azure DevOps Artifacts. This feed is connected to a ProGet connector.

Well, I'm thinking the issue is in the Azure DevOps Artifacts API. We found many bugs in their feeds already (i.e. not implementing the API as specified). It happens to work in Visual Studio, or Visual Studio already worked around the bug.

We can spot it very easily with a debugger; if you can share to use a feed url and credentials (I think it's just n access token?), then we can connect to the feed and see what's causing that error. You can send us the details via the ticket, so then it's not public.

Cheers,
Alana

@forcenet_4316 very sorry on the delayed response -- didn't notice that you had sent something to the box (it gets filled with spam very quickly) I've reopened this issue, and we will investigate ASAP

@mcascone unfortunately this is where it can get complex; there's nothing you can change in ProGet, so it's going to be either a server or proxy change

The most likely case is that the server doesn't trust the certificate presented by chocolatey.org; it's also very possible that your proxy server is stripping/replacing the certificate, and your server doesn't trust the proxy server's certificate.

It could also be a cipher issue. One user reported a very strange custom Windows setting caused this error; "a cipher not present on the server, due to a custom cipher list policy declared in the registry key (HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Policies\Microsoft\Cryptography\Configuration\SSL\00010002).

Since this is happening in the service, you should be able to log-in to server using the Service Account that ProGet is running under, and use the web browser to visit that URL. That will make it easier for your IT team to fix the certificate issues, in theory.

Thanks @philippe-camelio_3885 - with this, we can clearly see what the issue is; basically it's a UI bug on this page where we must have forgot to save the data as a UTC time. We'll get this fixed ASAP - stay tuned.

Hello; sorry on the slow response, but I'm still trying to reproduce this. I even downloaded antlr4-runtime and can't seem to do it.

[1] Are you able to provide us a copy of the MavenArtifactFiles in your database? Basically just SELECT * FROM MavenArtifactFiles, and send results to support at inedo dot com, with [QA-674] in the subject? That will let us spot some things right away. I'm struggling with seeing how this error is even possible, given the way that stored procedure works.

[2] What version of ProGet are you using? When did you start using this maven feed.

[3] Have you used replication for the feed?

Thanks,
Alana

@mcascone do you happen to remember where you found that link? We are trying to be conservative about adding a ton of redirect rules...

We are now running automated scans, so HOPEFULLY it's picking up bad links within the docs.

Hi @forcenet_4316

While the message isn't great, it's saying that the artifact (antlr4-runtime.jar-v4.5.2-1) is already in the target feed.

Can you confirm if that's the case? We should improve the message of course, but i want to check it

Thanks,
Alana

Hi @Stephen-Schaff ,

Yes, this can be a little bothersome. Actually this is something that we're considering for v6, to mostly replace the "impersonation" feature:

• Allow on Feeds:
  • [feed1, feed2, «group1»]
• Feed Permission:
  • View/Download
  • Publish/Push
  • Delete/Overwrite

The "username" would still be there, but mostly for "personal api keys".

Thanks,
Alana

Hello;

I'm not all that familiar with Ruby Gems, but I'll try to help. At first, I think the problem is that you're using host instead of source.

gem push <gem file> --source http://<redacted>/rubygems/RubyGem

Hope that does the trick :)

FYI; we recently updated our RubyGems Documentation to help others get setup.

Hello;

Thanks for all the detailed information, we'll definitely investigate what could be causing a segfault. This definitely shouldn't be the case...

is anything logged in Admin > Diagnostic Center?

Thanks.
Alana

Thanks @antony-booth_1029 !

This should get fixed via BM-3740 in the next maintenance release of BuildMaster

@grant-sparks_1282 I'm surprised there's no index, but I guess that's a property popular repository, so we can make ProGet not error in this case.

i logged PG-2007 to handle this :)

@nkr said in ProGet shows seemingly random versions as "latest" when using a NuGet connector:

quite an old build of the NuGet Gallery.

Oooh that might cause a lot of headaches/problems

Older versions of the NuGet Gallery have some notoriously strange/quirky behaviors -- so much so that we had a special feed type just handle them ("Quirks Feeds"). We finally removed that feed type last year (ProGet 5.3) after the bugs were mostly fixed.

The "missing dependency info" sounds very familiar, and the same thing happened with packages from nuget.org. It was such a long time ago, and the nuget team fixed it with new code and a data update.

Anyway.. I would recommend just migrating your packages using a bulk import.

Hi @grant-sparks_1282

Maven Connectors work by downloading a file called /.index/nexus-maven-repository-index.gz from the repository. It's weird, but I guess Google's repository doesn't have such an index?

https://dl.google.com/dl/android/maven2/.index/nexus-maven-repository-index.gz

I didn't know that repositories didn't have those... but you can confirm this is indeed the error under Admin > Scheduled Jobs > FullMavenConnectorIndex > History --- Can you confirm, and see it?

I guess it's not a problem, if you can still download artifacts. But without an index, you can't search then...

Thanks,
Alana

Hi @brett-polivka ,

Hmm.. I tried to look at a way we can get more diagnostic info, but I don't think we can find much more. It sounds like is an authentication problem with your PAT?

Did this happen after an upgrade of ProGet? We haven't made any changes to connectors in a while, but if you can tell the from->to versions, we can see if it might be related.

Otherwise, I'm thinking it's on the end of github, and maybe a relation to the PAT? It's possible your token expired, doesn't have right permission, etc. They may have also change the API too.

Thanks,
Alana

Hi @grant-sparks_1282,

Thank you for the very detailed information in sharing the problem and reproduction instructions. Based on this, I'm convinced you've run into this issue: PG-2005 (POM files downloaded from Maven feeds may be saved with empty contents).

ProGet 5.3.37 (which has that fix) is scheduled to be fixed on Friday, but we may release it sooner given the impact. It's holiday in US, otherwise we likely would have done it today already

Anyways, you have two choices:

1. [Recommended] Upgrade to ProGet 5.3.37-rc.2; for the Inedo Hub installation, you just need to switch to the Prerelease Feed

2. Downgrade to ProGet 5.3.34

The only difference between 5.3.36 and 5.3.37-rc.2 is that single issue (PG-2005).

Cheers,
Alana

@arozanski_1087 that's a slightly different caching setting

If you navigate Feeds > Connectors >your conenctor, it will be here:

Clicking on 'status" will show you all the cached queries.

@antony-booth_1029 thanks for the update!

Anecdotally, I've heard of similar issues with AWS RDS. I understand that, behind the scenes, it's some large/shared SQL Server database installation, so I wonder if other traffic impacts it? We've certainly confirmed that was the case in some organizations with their own large SQL Server cluster.

You may find using SQL Server Express + Self Backup is okay enough, too.

Hi @arozanski_1087 ,

Once a package is added to ProGet (either via drop folder, API, web upload, etc.), it should immediately show in the web UI. And it sounds like it does... because you can see the new package in the UI.

However, if the package isn't available through NuGet (which uses the NuGet API), then it means something is using a cached API response.

It could be ProGet. Connectors in ProGet have a feature called Metadata caching, so if you're accessing this feed through a connector, and this feature is enabled on that connector, it would behave exactly how you describe.

It could also be NuGet / VisualStudio, etc. The simplest thing to do would be to just run Fiddler, and capture the communication between NuGet/VS and ProGet. You will see cached responses if this is the case.

It might be a proxy/third-party server. Someone would have had to configure this in your network, and maybe they did to reduce traffic at one point? This should also show up in the Fiddler trace, hopefully, as a cached response.

Thanks,
Alana

Hi @grant-sparks_1282 ,

That's odd... real simple set-up, and I just tested on my end to be double-sure. Looking at the codes, I can't see how he would throw that error either...

What version of ProGet are you using (please see bottom right corner), and how is it installed (LInux/Docker or Windows)?

Thanks,
Alana

Hey @mjc_4927

Since you're already using Docker, you can install a new version of the container...

proget.inedo.com/productimages/inedo/buildmaster:7.0.9-ci.2

Cheers.
Alana

Thanks for sending those over @RobIII

At first I want to confirm, are you using the "V3" API endpoint in ProGet, for your VS2019 configuration? If not, please do that, since VS treats those APIs differently.

Otherwise, there's nothing off about the index files... the most likely scenario is that there is a problematic "translation" between VS2019's API call to ProGet, and ProGet's API call to GitLab. There are more than just those index files that are used.

It's most likely a bug on GitLab's end, in that they're doing something against the spec that just "happens to work" in VS2019. We struggled with a very similar bug with GitHub, but their timeline to investigate/fix it was crazy, so we just did a workaround (see PG-1932) and have it mostly working.

Ultimately, the customer decided not to use GitHub in this manner, because the workflow (outside of the API) was painful and GitHub is buggy they said. So there's that. They just publish packages to ProGet directly instead. You may find this workflow better as well.

Anyways... if you're already using the v3 api endpoint in ProGet, we will need to get this reproduced so we can attach a debugger and find the problem.

I understand it's not feasible to provide access... maybe it won't be so bad to set-up a reproduction case with a public repository? We don't know how to use GitLab's packages features, so if you can help set that up, we can attach a debugger and get it tested.

Thanks

Hi @kenneth-garza_2882 ,

Based on your use cases, an extension wouldn't be a very good fit... and Webhooks are the way to go.

For ProGet (unlike BuildMaster/Otter), extensions are really specialized, and it's pretty rare for folks to make their own, and we're keen on helping anyways.

For example, depending on what your vulnerability scanning software is, we'd love to help and even adopt the integratino, so other users can use it.

The main three things we see are:

• Cloud Storage;
  • https://github.com/Inedo/inedox-aws
  • https://github.com/Inedo/inedox-whitesource
• Vulnerability Scanning / Package Blocking
  • https://github.com/Inedo/inedox-sonatype
  • https://github.com/Inedo/inedox-azure
• User Directory
  • https://github.com/Inedo/inedox-inedocore

And the best way to do those, are mostly to copy the examples/code we already have.

Let us know more details, and we can do ourbest to help!

Thanks,
Alana

Hi @paul-reeves_6112 , sorry on the slow reply, we talked about this in our engineering meeting, but then I forgot to reply to it.

We've seen this come up a few times, but it's not limited to offline installations. We are thinking the problem is on the Windows Service Control Manager side of things (it's not sending the right signal), but we can't seem to pin it down. Our general plan is to add a timeout or something, and hope that it addresses the problem.

Not sure what's going on with the threads though; it might be related to the in-process database we use (Sqllite), doing some background things? I don't know...

Can you let us know if you it keeps happening?

Hi @tomg_5321

Glad you can get some results using the non-filtered search. If there is more demand, we may implement it... but you're the first person mention it.

However, based on your requirements, I don't think you should use search..

my actual underlying requirement is to test if a certain version (which may not be the most recent) of a certain package (by exact id) exists on the server, in an efficient way, for use over relatively low-performance network links (ie, not over local LAN)

In this case, I recommend you just do a HEAD request on this URL: /nuget/<feed-name>/v3/catalog/<package-name>/<version>.json

For example in PowerShell, Invoke-WebRequest -Method Head https://proget.inedo.com/nuget/NuGetLibraries/v3/catalog/inedo.buildmaster.sdk/2.0.0.json

A 200 means package exist, a 404 means it doesn't exist.

Cheers,
Alana

Hi @daniel-cooper_8892 ,

At first, please note you'll need to upgrade to v5.2 first, and then you can move to v5.3: please see Upgrading from ProGet v4.

Of course, you'll need access to your database. Please check out: Connect to SQL Server when system administrators are locked out

Once you do that, you can connect from SQL Management Studio, and then add yourself (and ideally the local Administrators group) as the sysadmin of sql server.

Cheers,
Alana

Hi @tomg_5321 ,

Looking at the code real quick, it seems that when performing searches against a local package store, ProGet does not parse the query in the same way nuget.org does (i.e. using filters).

Instead, if the search string (ID:mycompany in your case) exists within the Id, Title, or Tags, it's returned. I don't think that string exists, but mycompany must.

Can you try doing nuget search mycompany ... instead?

Thanks,
Alana

Hi @RobIII ,

Thanks for sending that over; we've reviewed it, but nothing is jumping out as a problem. I'm thinking it's something else...

Can we do this?

At first, let's move this to a new topic. I'm going to lock this thread, can you reply as a new topic? The reason for this is so that we can track much easier.

Is it possible to share access to your GitLab repository? That's the easiest thing of course, but you say "on-prem" so maybe not so easy.

As an alternative, can you create a simple reproduction case using a public GitLab repository? This way, we can plug ProGet into it, attach a debugger, and figure out what's going on.

The most likely scenario is that GitLab doing something wrong/weird against the spec, but it just happens to work in Visual Studio. That was the sitaution of GitHub.

Thanks,
Alana

@rbenfield_1885 hello just as an update, we make it so that automated scanners don't complain about this version in upcoming versions of ProGet (no ETA)

Hello,

Can you provide some specific reproduction instructions?

For example, I create a request like yours (but using Inedo and our ProGet server), and it produces results:

https://proget.inedo.com/nuget/NuGetLibraries/v3/search?q=ID:Inedo&skip=0&take=20&prerelease=true&semVerLevel=2.0.0

Thanks,
Alana

@maxim_mazurok this has been implemented for a couple years now -- however it is still considered an "Experimental feature" because the npm audit API is undocumented and npm, Inc. does not support third-party implementations.

Instead, ProGet will attempt to forward requests to the audit endpoint to npmjs.org or a connector. This may stop working if npmjs.org changes the API or blocks ProGet's requests. They've only changed APIs once, and we've promptly fixed it. In any case, you can configure the proxy URL in ProGet 5.3 by navigating to the Manage Feed page on your npm feed.

@Stephen-Schaff you got it! We'll try to get PG-1996 added in the next maintenance release. Should just be a pretty straight-forward loop :)

Hi @Stephen-Schaff,

I'm not all that familiar with Helm charts, but on some feeds, we have a "Delete all versions of this package" on the Delete Package page. I can see that checkbox is not on the Delete Helm Package page.

I guess, that would have the same effect? A checkbox that said "Delete all versions of this chart (i.e. delete entire repository)"?

What do you think? It seems relatively easy to do, and then we can fix the redirect thing as well.

Cheers,
Alana

Hi @paul-reeves_6112 ,

Of course, it should totally work in the usecase you described, I just wanted to explain how it got missed during testing :)

In general, we recommend not including configuration files that contain environment-specific configuration in artifacts to begin with, or at least using include/exclude filters on the Deploy Artifact operation. Then, deploying using the configuration files feature. But that's just general advice of course.

However, your use case sounds a little different, in that you want the configuration file deployed, but only the first time? I don't have a great solution to that, but the main downside I see is that engineers who manage/edit the configuration file (add values, etc), may be surprised when it's "not deploying" like they expect it to.

Config files can be tricky to balance as you probably already know :)

Cheers,
Alana

Thanks for investigating that; unfortunately I don't know the details to that.

We did a bit update to MyInedo recently, and tested it with an old install of BuildMaster v 3.5 and ProGet v2 - both requested a new key and activated it fine.

Maybe, try requesting a new license key at MyInedo that might help?

Hello; what version of ProGet are you using?

@internalit_7155 although v3.7 is really old, it should still work. The box on MyInedo is currently really teeny, so make sure that you expand the box and copy/paste the entire activation code. It's pretty long, and it should end with = or ==. (it's a base64 encoded string)

Hi @jndornbach_8182 ,

Thanks so much for sending that over, super helpful!

Here's what I did:

• created a new feed named maven-feed
• opened the .saz file
• reissued the .jar PUT request (changed URL) ... jar uploaded
• reissued the .pom PUT request... pom uploaded
• reissued the maven-metadata.xml GET request... results were identical to yours
• reissued the maven-metadata.xml PUT request.... no error

There are no ProGet software changes; from the headers, I can see you've since upgraded to ProGet 5.3.32, and I don't see anything that would remotely yield this difference - https://my.inedo.com/downloads/issues?Product=ProGet&FromVersion=5.3.33

The only thing I can think of is .NET5/Linux (you're running on Docker) vs .NET452/Windows (I was using IIS). We keep finding a few oddities like this, so I logged this as PG-1992 and escalated it. We should et it fixed for next maintenance release (July 30), but a few folks are on vacation now, so it'll take a couple days to get thru review.

Hi @paul-reeves_6112 ,

Good catches! These will be hidden in BM-3720. Originally this was intended only to hide it from the navigation, but it should also be displayed here.

FYI; you can download this and the other fixes in 7.0.6-rc.3

Alana

Thanks @jeff-peirson_4344 ! That's great. I'm going to assign this to a senior engineer to review (looks good to me), and then we'll merge/publish a new version next week.

And of course if you have any other suggestions on how to improve, please let us know. This feature came directly from a user collaboration - https://blog.inedo.com/feature-not-a-bug

Hi @paul-reeves_6112 ,

This was a strange bug that came from a v7 compiler change and missed test case; if you do ReadOnly=false (the typical usage), it works fine.

Fortunately it was a very simple fix, and if you download InedoCore 1.12.2-CI.1, it'll be included in that. We will ship 1.12.2 with the next maintenance release as well.

You can set your extensions feed url (Admin > Advanced Settings) to be https://proget.inedo.com/feeds/PrereleaseExtensions, which will download prerelease versions from the extensions page

Thanks,
Alana

Hello;

This will be fixed via BM-3722; it indeed was quite similar to that PowerShell

Alana

Hi @paul-reeves_6112 ,

@apxltd requested that we implement this after all

So it will be handled via BM-3719 in an upcoming maintenance release, and the icons will be proxied through buildmaster server.

Alana

Hi @adrian-leeming_9656 ,

The Inedo Hub uses the system's local package registry (`C:\ProgramData\upack\installedPackages.json) to determine which packages/programs are installed. The easiest thing to do is just delete that directory.

Here are the full troubleshooting instructions for fixing a broken install:
https://docs.inedo.com/docs/desktophub-troubleshooting

cheers,
Alaa

Hello,

The NuGet Client only uses the the -ApiKey you on certain requests to the -Source, specifically the PUT of the package file. Before doing that, it will query the package source using a GET, and in this case, the API Key isn't sent.

The easiest solution is to allow anonymous view access to your feed. That simplifies configuration for all users of your feed.

Alternatively, you will need to add credentials to the PackageSourceCredentials in the NuGet.config file . Note this file exists in several places, please see Config file locations and uses

Cheers,
Alana

Hi @paul-reeves_6112 , looks like this was a v7 regression due to some UI/library changes.

hopefully we can create a new/better configuration file editor at some point soon :)

This is fixed in 7.0.6-ci.2; there haven't been many code changes since 7.0.5, so it's quite safe (even though it's pre-release). Here is how to install it:

https://docs.inedo.com/docs/desktophub-overview#prerelease-product-versions

Cheers,
Alana

@kichikawa_2913 glad it's working now, you seem to have found the issue.

FYI, you can also configure that value from Advanced Settings page under Admin.

Hi @benjamin-soddy_9591 ,

ProGet does not have support for package deprecations at this time, and we haven't had anyone ask about them until now :)

It could be very tricky to implement, because once a package version is pulled or cached to ProGet, there is no "synchronization" of "server-only" metadata. The reason is, packages versions are supposed to be immutable, and metadata self-contained.

The "unlisted" flag is similar. If a NuGet pulled/cached package becomes unlisted, then ProGet won't "know" about that, and therefore can't inform the client.

There doesn't seem to be any value in depreciating a first-party package (i.e. a library you create), as even the largest organizations should have communication channels for library authors and framework teams. Plus, they can monitor usage, and reach out directly to teams.

However, but I can see why you'd want to know which third-party packages are deprecated... but so far as we saw, very few package authors deprecate their packages. There seem to be better tools (vulnerability scans) to see if there are problem libraries.

So, if you could share some more specifics about what you're doing, why deprecation is helpful (with specific packages), etc., metadata would really help us understand the value in such a feature.

Hi @jharbison_7839 ,

We definitely need to update our release template documentation, but in the meantime...

The "Allow all variables to be added" option allows you to freely add/edit/remove variable from the Build/Release pages, regardless of restrictions set-up by the Release Template. When it's not checked, only templated variables may be configured.

This is not checked by default on new release templates, and prior to v7, there was no ability to edit variables using the template.

So, checked = just like it was in v6.2 and earlier, and not checked is temlate-based editing.

Alana

Hi @paul-reeves_6112 ,

I ended up fixing this in the extension after all - can you try 1.10.5?

We will still want to make the SDK change, but it was more involved than I thought. So the team wants to wait to do that later... especially since we could quickly fix it in the extension.

Cheers,
Alana