Posts made by atripp

Hi @grant-sparks_1282,

Thank you for the very detailed information in sharing the problem and reproduction instructions. Based on this, I'm convinced you've run into this issue: PG-2005 (POM files downloaded from Maven feeds may be saved with empty contents).

ProGet 5.3.37 (which has that fix) is scheduled to be fixed on Friday, but we may release it sooner given the impact. It's holiday in US, otherwise we likely would have done it today already

Anyways, you have two choices:

1. [Recommended] Upgrade to ProGet 5.3.37-rc.2; for the Inedo Hub installation, you just need to switch to the Prerelease Feed

2. Downgrade to ProGet 5.3.34

The only difference between 5.3.36 and 5.3.37-rc.2 is that single issue (PG-2005).

Cheers,
Alana

@arozanski_1087 that's a slightly different caching setting

If you navigate Feeds > Connectors >your conenctor, it will be here:

Clicking on 'status" will show you all the cached queries.

@antony-booth_1029 thanks for the update!

Anecdotally, I've heard of similar issues with AWS RDS. I understand that, behind the scenes, it's some large/shared SQL Server database installation, so I wonder if other traffic impacts it? We've certainly confirmed that was the case in some organizations with their own large SQL Server cluster.

You may find using SQL Server Express + Self Backup is okay enough, too.

Hi @arozanski_1087 ,

Once a package is added to ProGet (either via drop folder, API, web upload, etc.), it should immediately show in the web UI. And it sounds like it does... because you can see the new package in the UI.

However, if the package isn't available through NuGet (which uses the NuGet API), then it means something is using a cached API response.

It could be ProGet. Connectors in ProGet have a feature called Metadata caching, so if you're accessing this feed through a connector, and this feature is enabled on that connector, it would behave exactly how you describe.

It could also be NuGet / VisualStudio, etc. The simplest thing to do would be to just run Fiddler, and capture the communication between NuGet/VS and ProGet. You will see cached responses if this is the case.

It might be a proxy/third-party server. Someone would have had to configure this in your network, and maybe they did to reduce traffic at one point? This should also show up in the Fiddler trace, hopefully, as a cached response.

Thanks,
Alana

Hi @grant-sparks_1282 ,

That's odd... real simple set-up, and I just tested on my end to be double-sure. Looking at the codes, I can't see how he would throw that error either...

What version of ProGet are you using (please see bottom right corner), and how is it installed (LInux/Docker or Windows)?

Thanks,
Alana

Hey @mjc_4927

Since you're already using Docker, you can install a new version of the container...

proget.inedo.com/productimages/inedo/buildmaster:7.0.9-ci.2

Cheers.
Alana

Thanks for sending those over @RobIII

At first I want to confirm, are you using the "V3" API endpoint in ProGet, for your VS2019 configuration? If not, please do that, since VS treats those APIs differently.

Otherwise, there's nothing off about the index files... the most likely scenario is that there is a problematic "translation" between VS2019's API call to ProGet, and ProGet's API call to GitLab. There are more than just those index files that are used.

It's most likely a bug on GitLab's end, in that they're doing something against the spec that just "happens to work" in VS2019. We struggled with a very similar bug with GitHub, but their timeline to investigate/fix it was crazy, so we just did a workaround (see PG-1932) and have it mostly working.

Ultimately, the customer decided not to use GitHub in this manner, because the workflow (outside of the API) was painful and GitHub is buggy they said. So there's that. They just publish packages to ProGet directly instead. You may find this workflow better as well.

Anyways... if you're already using the v3 api endpoint in ProGet, we will need to get this reproduced so we can attach a debugger and find the problem.

I understand it's not feasible to provide access... maybe it won't be so bad to set-up a reproduction case with a public repository? We don't know how to use GitLab's packages features, so if you can help set that up, we can attach a debugger and get it tested.

Thanks

Hi @kenneth-garza_2882 ,

Based on your use cases, an extension wouldn't be a very good fit... and Webhooks are the way to go.

For ProGet (unlike BuildMaster/Otter), extensions are really specialized, and it's pretty rare for folks to make their own, and we're keen on helping anyways.

For example, depending on what your vulnerability scanning software is, we'd love to help and even adopt the integratino, so other users can use it.

The main three things we see are:

• Cloud Storage;
  • https://github.com/Inedo/inedox-aws
  • https://github.com/Inedo/inedox-whitesource
• Vulnerability Scanning / Package Blocking
  • https://github.com/Inedo/inedox-sonatype
  • https://github.com/Inedo/inedox-azure
• User Directory
  • https://github.com/Inedo/inedox-inedocore

And the best way to do those, are mostly to copy the examples/code we already have.

Let us know more details, and we can do ourbest to help!

Thanks,
Alana

Hi @paul-reeves_6112 , sorry on the slow reply, we talked about this in our engineering meeting, but then I forgot to reply to it.

We've seen this come up a few times, but it's not limited to offline installations. We are thinking the problem is on the Windows Service Control Manager side of things (it's not sending the right signal), but we can't seem to pin it down. Our general plan is to add a timeout or something, and hope that it addresses the problem.

Not sure what's going on with the threads though; it might be related to the in-process database we use (Sqllite), doing some background things? I don't know...

Can you let us know if you it keeps happening?

Hi @tomg_5321

Glad you can get some results using the non-filtered search. If there is more demand, we may implement it... but you're the first person mention it.

However, based on your requirements, I don't think you should use search..

my actual underlying requirement is to test if a certain version (which may not be the most recent) of a certain package (by exact id) exists on the server, in an efficient way, for use over relatively low-performance network links (ie, not over local LAN)

In this case, I recommend you just do a HEAD request on this URL: /nuget/<feed-name>/v3/catalog/<package-name>/<version>.json

For example in PowerShell, Invoke-WebRequest -Method Head https://proget.inedo.com/nuget/NuGetLibraries/v3/catalog/inedo.buildmaster.sdk/2.0.0.json

A 200 means package exist, a 404 means it doesn't exist.

Cheers,
Alana

Hi @daniel-cooper_8892 ,

At first, please note you'll need to upgrade to v5.2 first, and then you can move to v5.3: please see Upgrading from ProGet v4.

Of course, you'll need access to your database. Please check out: Connect to SQL Server when system administrators are locked out

Once you do that, you can connect from SQL Management Studio, and then add yourself (and ideally the local Administrators group) as the sysadmin of sql server.

Cheers,
Alana

Hi @tomg_5321 ,

Looking at the code real quick, it seems that when performing searches against a local package store, ProGet does not parse the query in the same way nuget.org does (i.e. using filters).

Instead, if the search string (ID:mycompany in your case) exists within the Id, Title, or Tags, it's returned. I don't think that string exists, but mycompany must.

Can you try doing nuget search mycompany ... instead?

Thanks,
Alana

Hi @RobIII ,

Thanks for sending that over; we've reviewed it, but nothing is jumping out as a problem. I'm thinking it's something else...

Can we do this?

At first, let's move this to a new topic. I'm going to lock this thread, can you reply as a new topic? The reason for this is so that we can track much easier.

Is it possible to share access to your GitLab repository? That's the easiest thing of course, but you say "on-prem" so maybe not so easy.

As an alternative, can you create a simple reproduction case using a public GitLab repository? This way, we can plug ProGet into it, attach a debugger, and figure out what's going on.

The most likely scenario is that GitLab doing something wrong/weird against the spec, but it just happens to work in Visual Studio. That was the sitaution of GitHub.

Thanks,
Alana

@rbenfield_1885 hello just as an update, we make it so that automated scanners don't complain about this version in upcoming versions of ProGet (no ETA)

Hello,

Can you provide some specific reproduction instructions?

For example, I create a request like yours (but using Inedo and our ProGet server), and it produces results:

https://proget.inedo.com/nuget/NuGetLibraries/v3/search?q=ID:Inedo&skip=0&take=20&prerelease=true&semVerLevel=2.0.0

Thanks,
Alana

@maxim_mazurok this has been implemented for a couple years now -- however it is still considered an "Experimental feature" because the npm audit API is undocumented and npm, Inc. does not support third-party implementations.

Instead, ProGet will attempt to forward requests to the audit endpoint to npmjs.org or a connector. This may stop working if npmjs.org changes the API or blocks ProGet's requests. They've only changed APIs once, and we've promptly fixed it. In any case, you can configure the proxy URL in ProGet 5.3 by navigating to the Manage Feed page on your npm feed.

@Stephen-Schaff you got it! We'll try to get PG-1996 added in the next maintenance release. Should just be a pretty straight-forward loop :)

Hi @Stephen-Schaff,

I'm not all that familiar with Helm charts, but on some feeds, we have a "Delete all versions of this package" on the Delete Package page. I can see that checkbox is not on the Delete Helm Package page.

I guess, that would have the same effect? A checkbox that said "Delete all versions of this chart (i.e. delete entire repository)"?

What do you think? It seems relatively easy to do, and then we can fix the redirect thing as well.

Cheers,
Alana

Hi @paul-reeves_6112 ,

Of course, it should totally work in the usecase you described, I just wanted to explain how it got missed during testing :)

In general, we recommend not including configuration files that contain environment-specific configuration in artifacts to begin with, or at least using include/exclude filters on the Deploy Artifact operation. Then, deploying using the configuration files feature. But that's just general advice of course.

However, your use case sounds a little different, in that you want the configuration file deployed, but only the first time? I don't have a great solution to that, but the main downside I see is that engineers who manage/edit the configuration file (add values, etc), may be surprised when it's "not deploying" like they expect it to.

Config files can be tricky to balance as you probably already know :)

Cheers,
Alana

Thanks for investigating that; unfortunately I don't know the details to that.

We did a bit update to MyInedo recently, and tested it with an old install of BuildMaster v 3.5 and ProGet v2 - both requested a new key and activated it fine.

Maybe, try requesting a new license key at MyInedo that might help?

Hello; what version of ProGet are you using?

@internalit_7155 although v3.7 is really old, it should still work. The box on MyInedo is currently really teeny, so make sure that you expand the box and copy/paste the entire activation code. It's pretty long, and it should end with = or ==. (it's a base64 encoded string)

Hi @jndornbach_8182 ,

Thanks so much for sending that over, super helpful!

Here's what I did:

• created a new feed named maven-feed
• opened the .saz file
• reissued the .jar PUT request (changed URL) ... jar uploaded
• reissued the .pom PUT request... pom uploaded
• reissued the maven-metadata.xml GET request... results were identical to yours
• reissued the maven-metadata.xml PUT request.... no error

There are no ProGet software changes; from the headers, I can see you've since upgraded to ProGet 5.3.32, and I don't see anything that would remotely yield this difference - https://my.inedo.com/downloads/issues?Product=ProGet&FromVersion=5.3.33

The only thing I can think of is .NET5/Linux (you're running on Docker) vs .NET452/Windows (I was using IIS). We keep finding a few oddities like this, so I logged this as PG-1992 and escalated it. We should et it fixed for next maintenance release (July 30), but a few folks are on vacation now, so it'll take a couple days to get thru review.

Hi @paul-reeves_6112 ,

Good catches! These will be hidden in BM-3720. Originally this was intended only to hide it from the navigation, but it should also be displayed here.

FYI; you can download this and the other fixes in 7.0.6-rc.3

Alana

Thanks @jeff-peirson_4344 ! That's great. I'm going to assign this to a senior engineer to review (looks good to me), and then we'll merge/publish a new version next week.

And of course if you have any other suggestions on how to improve, please let us know. This feature came directly from a user collaboration - https://blog.inedo.com/feature-not-a-bug

Hi @paul-reeves_6112 ,

This was a strange bug that came from a v7 compiler change and missed test case; if you do ReadOnly=false (the typical usage), it works fine.

Fortunately it was a very simple fix, and if you download InedoCore 1.12.2-CI.1, it'll be included in that. We will ship 1.12.2 with the next maintenance release as well.

You can set your extensions feed url (Admin > Advanced Settings) to be https://proget.inedo.com/feeds/PrereleaseExtensions, which will download prerelease versions from the extensions page

Thanks,
Alana

Hello;

This will be fixed via BM-3722; it indeed was quite similar to that PowerShell

Alana

Hi @paul-reeves_6112 ,

@apxltd requested that we implement this after all

So it will be handled via BM-3719 in an upcoming maintenance release, and the icons will be proxied through buildmaster server.

Alana

Hi @adrian-leeming_9656 ,

The Inedo Hub uses the system's local package registry (`C:\ProgramData\upack\installedPackages.json) to determine which packages/programs are installed. The easiest thing to do is just delete that directory.

Here are the full troubleshooting instructions for fixing a broken install:
https://docs.inedo.com/docs/desktophub-troubleshooting

cheers,
Alaa

Hello,

The NuGet Client only uses the the -ApiKey you on certain requests to the -Source, specifically the PUT of the package file. Before doing that, it will query the package source using a GET, and in this case, the API Key isn't sent.

The easiest solution is to allow anonymous view access to your feed. That simplifies configuration for all users of your feed.

Alternatively, you will need to add credentials to the PackageSourceCredentials in the NuGet.config file . Note this file exists in several places, please see Config file locations and uses

Cheers,
Alana

Hi @paul-reeves_6112 , looks like this was a v7 regression due to some UI/library changes.

hopefully we can create a new/better configuration file editor at some point soon :)

This is fixed in 7.0.6-ci.2; there haven't been many code changes since 7.0.5, so it's quite safe (even though it's pre-release). Here is how to install it:

https://docs.inedo.com/docs/desktophub-overview#prerelease-product-versions

Cheers,
Alana

@kichikawa_2913 glad it's working now, you seem to have found the issue.

FYI, you can also configure that value from Advanced Settings page under Admin.

Hi @benjamin-soddy_9591 ,

ProGet does not have support for package deprecations at this time, and we haven't had anyone ask about them until now :)

It could be very tricky to implement, because once a package version is pulled or cached to ProGet, there is no "synchronization" of "server-only" metadata. The reason is, packages versions are supposed to be immutable, and metadata self-contained.

The "unlisted" flag is similar. If a NuGet pulled/cached package becomes unlisted, then ProGet won't "know" about that, and therefore can't inform the client.

There doesn't seem to be any value in depreciating a first-party package (i.e. a library you create), as even the largest organizations should have communication channels for library authors and framework teams. Plus, they can monitor usage, and reach out directly to teams.

However, but I can see why you'd want to know which third-party packages are deprecated... but so far as we saw, very few package authors deprecate their packages. There seem to be better tools (vulnerability scans) to see if there are problem libraries.

So, if you could share some more specifics about what you're doing, why deprecation is helpful (with specific packages), etc., metadata would really help us understand the value in such a feature.

Hi @jharbison_7839 ,

We definitely need to update our release template documentation, but in the meantime...

The "Allow all variables to be added" option allows you to freely add/edit/remove variable from the Build/Release pages, regardless of restrictions set-up by the Release Template. When it's not checked, only templated variables may be configured.

This is not checked by default on new release templates, and prior to v7, there was no ability to edit variables using the template.

So, checked = just like it was in v6.2 and earlier, and not checked is temlate-based editing.

Alana

Hi @paul-reeves_6112 ,

I ended up fixing this in the extension after all - can you try 1.10.5?

We will still want to make the SDK change, but it was more involved than I thought. So the team wants to wait to do that later... especially since we could quickly fix it in the extension.

Cheers,
Alana

Hi @sbolisetty_3792 ,

Cron format is a bit tricky, but it "can do anything"... so it seemed like an "decent" compromise to use that for advanced/custom intervals.

But to be honest, even after reading the docs, I can never figure it out, and always use a generator like this:

http://www.cronmaker.com/

https://freeformatter.com/cron-expression-generator-quartz.html

What cron expression do you use, and what time of day did you check the "next run time".

Cheers,
Alana

Hi @paul-reeves_6112 ,

Thanks for all the detailed information; it was a bit of a rush due to US holiday yesterday, and I didn't look so carefully. That patch to Scripting (1.10.4) seems to fix text mode, but the editor is clearly still broken unless you prepend the application name (which you figured out).

The application name should not be required like this, and I've logged a product change to fix this (BM-3707 and SDK-74). This will get fixed in the next maintenance release of BuildMaster, which is scheduled for Friday.

We may ship it sooner, because this regression is quite annoying.

Cheers,
Alana

@paul-reeves_6112 I'm thinking I didn't fix the regression after all

Can you go to Admin > Diagnostic Center and share the stack trace that resulted from the message? It should be logged in the case of an editor crash like that. I suspect it's related, but I only used a very simple/quick script test.

Hi @paul-reeves_6112 ,

I was able to track down this error; it's a regression introduced into v7, and only impacts the OtterScript editor.

If you upgrade the Scripting extension (Admin > Extensions) to 1.10.4 then it should work :)

Cheers,
Alana

@mcascone thanks!! I added a redirect

Great, thanks @hwittenborn -- where's pull requests for our docs when we need it ;)

I'm not familiar enough with Linux or Debian to understand or make the second changes you suggested (I'll leave it to @rhessinger), but I replaced the wget command with yours, which avoids creating that file.

Cheers!

Thanks for letting us know @hwittenborn! That wasn't very clear that it was two separate commands....

I updated the docs to hopefully be clearer, and break out these into three steps.

@Stephen-Schaff said in API to apply an Alternate Tag to Docker Container Image:

Here is the PowerShell function that does both the promotion and the tagging (incase anyone ever needs something like this). Kind of a "do it yourself" repackaging. (Might be nice to have the Repackaging API support Docker container images someday).

Thanks for sharing this, I've added it to our Semantic Versioning for Containers docs page, I hope that's okay :)

And yes I agree, it woudl be nice to make this an easier API call

Thanks for the update @Stephen-Schaff -- And just to add to this, you should see "Note that you can use the following alternative tags to refer to this image:..." on the browse image page.

@jndornbach_8182 said in ProGet shows "(500) Server Error: Value cannot be null. (Parameter 'version')" when opening "Dependencies" tab of Maven artifact:

Is there a way that ProGet can cope with "parent-managed" dependency versions in future or, at least, does not throw this error?

For sure! It does look like this is supported in most places, but not on this page in the UI.

Easy fix... I logged this as PG-1968, and it will ship in the next maintenance release of ProGet, which is scheduled for next Friday. Or I can show you how to download a pre-release if you'd prefer sooner :)

Hi @jndornbach_8182,

Thanks for posting all the information, it's really helpful!

With that stack trace information, we can see where the error is occurring... and as the error says, the PUT body doesn't contain XML as the code expects.

if (context.Request.HttpMethod == "PUT")
{
    if (info.IsMetadataRequest)
    {
        if (info.HashAlgorithm == null)
        {
            var xdoc = XDocument.Load(context.Request.InputStream);
            var metadataElement = xdoc.Element("metadata");
            if (metadataElement != null)
            {
                var versioningElement = metadataElement.Element("versioning");
                if (versioningElement != null)
                {
                    var releaseVersion = (string)versioningElement.Element("release");
                    if (!string.IsNullOrWhiteSpace(releaseVersion))
                        await new DB.Context(false).MavenArtifacts_SetReleaseVersionAsync(feed.FeedId, info.GroupId, info.ArtifactId, releaseVersion.Trim());
                }
            }
        }
        else
        {
            // Don't need to actually save the hash since it's computed on demand
        }
    
        context.Response.StatusCode = 201;
    }
}

If I'm being totally honest, I don't understand why the code is doing what it's doing

But that doesn't mean I can't help fix it! Could I trouble you to attach a tool like Fiddler to capture the requests, and then send us the session file? We can then inspect that PUT request, and see what's going on.

You can send to support at inedo dot com with a subject of [QA-597]-- but please let me know when you do, so I can dig in the box and find it.

Thanks,
Alana

Hi @Ashley_8010

In nearly all of our external retail websites, there are tonnes of connections to third party websites which inevitably drive up the loading times for our customers. Some loading times can be upwards of 10+ seconds (depending on huge sneaker drops or major releases)

It sounds like what might need is a sort of web bundler/packager (perhaps like webpack)?

And then a Content Delivery Network on top of that, that can cache and serve static content faster than your server? We use Cloudflare ourselves.

ProGet does allow you to access individual files within a package (see the files tab on InedoLib for example), and download those files with a URL like this.

However, this feature was not designed to be a front-end web server, and I wouldn't recommend using it as such.

Cheers,
Alana

Hi @joel_6345 ,

Can I trouble you to read the latest version of the documentation?

https://docs.inedo.com/docs/proget-feeds-nuget-symbol-and-source-server

Actually I just rewrote it, so I'm really hoping that this can clarify your question .. and that it would have made it easier in the first place :)

thank you

Hi @Stephen-Schaff ,

I'm not sure about the API, but ProGet implements the Chart Repository API, and from a look at that, you should be able to just access index.yaml at the API Endpoint URL

Here is some information about the format of that file: https://helm.sh/docs/topics/chart_repository/#the-index-file

Can you give that a try and let us know what you find out? Cheers :)

Cheers,
Alana

Hi @coskun_0070 ,

Did you try setting an api key with setApiKey? Perhaps there's another way to suppress this messages?

While it's relatively easy to add privileges and features, we've learned the hard way that it creates a lot more work in the long-run from a support standpoint and user confusion. It's best to keep things simple.

I think this is something addressable via nuget client configuration.
Cheers,
Alana