Posts made by atripp

@shaun-d-scott_2657 Inedo products do not use log4j, so it's not an issue for ProGet :)

See more information here: https://blog.inedo.com/log4shell-high-severity-vulnerabilities

@darkbasics_6739 thanks, of course that's not the problem then

We're still investigating this, and it's quite odd why the extension isn't getting transfered. The workaround you're doing is fine to at least evaluate/test, but we'll most certainly get this fixed ASAP

@v-makkenze_6348 thanks for the additional information

ProGet does not cache npm package lists/indexes. It's always generated from the information in the database, so if you pushed it - then it's in the database. You should be able to see it in the ProGet UI right after publishing.

It's very possible that the npm client is doing some sort of caching as well, but isn't doing that cache with authenticated requests. I'm not familiar enough, but that's my guess.

You could use a tool like Fiddler to verify / see what requests npm is actually making.

Hi @mike_4027 ,

I would definitely recommend upgrading; 5.1 is a couple major versions behind.

The issue is most certainly related to the connector; when you disable it, do you get a near-immediate 404? Under Admin > Diagnostic Center, you may see connector errors/warnings as well.

Once you confirm it's the connector, then the next best thing to do would be to monitor the traffic between ProGet and the internet. This involves setting up a Proxy (Fiddler works nicely), and then having ProGet connect to that proxy (Admin > Proxy). You should be able to identify corresponding requests, and maybe we'll see something there!

Let us know what you find,

Alana

Hi @darkbasics_6739 ,

That's really weird, but I can see how that error might happen now. It's been addressed as OT-446, and will be fixed in the next maintenance release.

We're still not sure why the SCripting extension isn't coming over. We're thinking, it might be due to a time out of sorts, since the file is now pretty large. Obviously that should throw a different error.... is there a lot of bandwidth between servers?

Cheers,
Alana

@v-makkenze_6348 it looks like you've enabled caching, which means that ProGet's responses won't be newly generated. Please disable this :)

@sbindra_9387 you can enter the activation code in ProGet, when you click the activate button

Here is the instruction for manual activation:
https://docs.inedo.com/docs/myinedo-activating-a-license-key

Hi @patrick-groess_2616 ,

The /symbols/<feed-name> URL is only for Visual Studio's Symbol Location setting, and is used to download symbols. Do not try to use it with nuget.exe, it will give that error.

You need to use /nuget/<feed-name> to push symbol or nuget.exe packages.

Thanks,
Alana

@entro_4370 we are definitely seeing "package" interest from "data science" teams inside of large organizations, but it seems they're more Python users instead of R users ‍️

@jblaine_9526 sounds good, I've forwarded this to our presales / customer success team internally - so they'll probably reach-out in the coming days about that process to see how they can help

Hi @entro_4370,

I'm afraid not... seems this thread has been a little quiet.

We haven't had many enterprise customers or sales leads requesting it as a feature, and we didn't see a good way to market the feature. It seems very few people search for CRAN-related topics or wanting private repository

Best,
Alana

@lukel_3991 thanks.

So this works totally fine if you're using localhost instead of computername.domain.com then And then, computername.domain.com is a Windows server?

If that's the case, it means the agent (which is connected via PowerShell Remoting) isn't fully loading; basically, the Scripting extension isn't loading, or had an error.

Can you try using a different Temp path? Maybe there's a relation....

Thanks,
Alana

Hi @falk-winkler_2111,

ProGet free edition only supports self-connectors when they connect directly to ProGet. If you bypass the proxy, that should resolve the license violations for you.

To do this, just use http://localhost or http://<PROGET_IP_ADDRESS> in your connector without issue. Please give that a try and let me know if you run into any issues.

Cheers,
Alana

@jblaine_9526 good question; it's not exactly spec'd out, but that would make sense. If SAML is enabled (as one of the directories), then users should be able to bypass SAML behavior. Maybe two Login Buttons, or something.

Since SAML is a ProGet Enterprise feature, it's very possible we could implement this specific requirement much sooner as part of a different program (i.e. "customer success" vs "product roadmapping"), but neither is really my department

I can get you in touch with my colleague who's on that team, but he's not on the forums -- let me know if it's okay to share your email to him (I can see it from my end)

Hi @jblaine_9526 , there are!

I don't have a schedule just yet, but we are planning to simplify the "hybrid" user directory by just making it so that you can set an "active/inactive" flag on each user directory, and then ProGet will run through the active ones in a prioirty order.

Cheers,
Alana

Hi @mcascone,

It looks like this isn't being displayed in ProGet's UI, so we'll fix this via PG-2038

cheers,
Alana

Hello,

This is unfortunately due to bugs in the PowerShellGet & PowerShell Gallery API that aren't feasible to workaround. Basically, the API is reporting different versions than the package file, which makes working with local/cached packages and remote packages painful like this. However, it seems like it's going to be addressed by Microsoft, finally, in PowerShellget v3.

In any case, given your architecture (i.e. "Machines on our network cannot connect to PSGallery direct due to the firewall and are only allowed to retrieve via this one feed."), we recommend using a Package Approval Workflow instead.

• create an 'unapproved-powershell' and approved-powershell feed
• create a connector on 'unapproved-powershell' to PSGallery.org
• Promote packages from unapproved-powershell to approved-powershell

This will totally eliminate the "bad API" problem as well.

Cheers,
Alana

Generally speaking, if yo ucan try to identify the pattern -- like which pages are slow -- that will help.

Some different pages to investigate:

• / is somewhat resource intensive, and involves database calls
• /health uses minimal system resources
• /resources/images/layout/logo.svg is a static image, is unauthenticated, and is cached by the browser

Hi @dac_4595 ,

If ProGet is hosted in IIS, then one thing that might be happening is that Application Pool shuts down if it's not accessed in a set amount of time. That 5-10 seconds might be the initial start-up time. You can control the timeouts in IIS, and even set it to be AlwaysOn.

Otherwise, there's nothing in the browser's cache that would cause this. If anything, clearing the cache would cause things to go slower, as pages need to re-load.

Cheers,
Alana

@forcenet_4316 great, sounds good! Actually we didn't end up shipping 6.0.0 yet (we will soon), so I just changed PG-2011 to target 5.3.40, which is scheduled for release on Friday. This way it will be in both.

Thanks @drew-stinnett_4680 !

I've updated the documentation with a link to this discussion.

@forcenet_4316 thanks to the data you sent, I was finally able to figure this out!

The problem is with antlr4-runtime-4.5.2-1-sources.jar. When ProGet replicates or promotes this package (logic is the same), the artifact's fileType is assumed to be .jar instead of -sources.jar; this is what's causing the PK constraint. This seems to only impact a very small number of artifacts, since I think I mostly saw .sources.jar in other cases (which would work).

I've fixed this via PG-2011, which is scheduled for 6.0.1 (shipping later this week).

Are you planning to upgrade to v6 soon? If not, I should be able to backport the fix to v5.3.

It seems that PowerShell client doesn't provide much details about errors, so glad you could figure out what the issue was for powershellgallery.com. That message probably could mean a lot of things, and is probably unrelated to TLS/SSL..

So the first thing that's jumping out to me is this:

WARNING: Unable to resolve package source 'http://proget.domain.org/nuget/ps'

There is no https connection, so there'd be no TLS/SSL issues. But assuming you may have changed the url when posting this... here is a registry key you can use to force that setting (it makes that powershell command unnecessary)

https://inedo.com/support/kb/1161/tls-v12-configuration-and-connection-errors

If you're on HTTPS the other thing to look at would be certificates.

Otherwise, you may need to use something like Fiddler to see the actual request/responses that PowerShell client is issuing/reciveing.

hi @paul_6112 - just curiosu if this continues to happen? We had another report about this, but are still struggling to find out why / and a resolution. We're going to try to take another stab at figuring this out again.

Hi @jan_ko ,

Thanks for sharing this; the .nuspec file looks okay to me, so it doesn't seem like a problem with the package.

After building the agent pushed these nupkg's to the Azure DevOps Artifacts. This feed is connected to a ProGet connector.

Well, I'm thinking the issue is in the Azure DevOps Artifacts API. We found many bugs in their feeds already (i.e. not implementing the API as specified). It happens to work in Visual Studio, or Visual Studio already worked around the bug.

We can spot it very easily with a debugger; if you can share to use a feed url and credentials (I think it's just n access token?), then we can connect to the feed and see what's causing that error. You can send us the details via the ticket, so then it's not public.

Cheers,
Alana

@forcenet_4316 very sorry on the delayed response -- didn't notice that you had sent something to the box (it gets filled with spam very quickly) I've reopened this issue, and we will investigate ASAP

@mcascone unfortunately this is where it can get complex; there's nothing you can change in ProGet, so it's going to be either a server or proxy change

The most likely case is that the server doesn't trust the certificate presented by chocolatey.org; it's also very possible that your proxy server is stripping/replacing the certificate, and your server doesn't trust the proxy server's certificate.

It could also be a cipher issue. One user reported a very strange custom Windows setting caused this error; "a cipher not present on the server, due to a custom cipher list policy declared in the registry key (HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Policies\Microsoft\Cryptography\Configuration\SSL\00010002).

Since this is happening in the service, you should be able to log-in to server using the Service Account that ProGet is running under, and use the web browser to visit that URL. That will make it easier for your IT team to fix the certificate issues, in theory.

Thanks @philippe-camelio_3885 - with this, we can clearly see what the issue is; basically it's a UI bug on this page where we must have forgot to save the data as a UTC time. We'll get this fixed ASAP - stay tuned.

Hello; sorry on the slow response, but I'm still trying to reproduce this. I even downloaded antlr4-runtime and can't seem to do it.

[1] Are you able to provide us a copy of the MavenArtifactFiles in your database? Basically just SELECT * FROM MavenArtifactFiles, and send results to support at inedo dot com, with [QA-674] in the subject? That will let us spot some things right away. I'm struggling with seeing how this error is even possible, given the way that stored procedure works.

[2] What version of ProGet are you using? When did you start using this maven feed.

[3] Have you used replication for the feed?

Thanks,
Alana

@mcascone do you happen to remember where you found that link? We are trying to be conservative about adding a ton of redirect rules...

We are now running automated scans, so HOPEFULLY it's picking up bad links within the docs.

Hi @forcenet_4316

While the message isn't great, it's saying that the artifact (antlr4-runtime.jar-v4.5.2-1) is already in the target feed.

Can you confirm if that's the case? We should improve the message of course, but i want to check it

Thanks,
Alana

Hi @Stephen-Schaff ,

Yes, this can be a little bothersome. Actually this is something that we're considering for v6, to mostly replace the "impersonation" feature:

• Allow on Feeds:
  • [feed1, feed2, «group1»]
• Feed Permission:
  • View/Download
  • Publish/Push
  • Delete/Overwrite

The "username" would still be there, but mostly for "personal api keys".

Thanks,
Alana

Hello;

I'm not all that familiar with Ruby Gems, but I'll try to help. At first, I think the problem is that you're using host instead of source.

gem push <gem file> --source http://<redacted>/rubygems/RubyGem

Hope that does the trick :)

FYI; we recently updated our RubyGems Documentation to help others get setup.

Hello;

Thanks for all the detailed information, we'll definitely investigate what could be causing a segfault. This definitely shouldn't be the case...

is anything logged in Admin > Diagnostic Center?

Thanks.
Alana

Thanks @antony-booth_1029 !

This should get fixed via BM-3740 in the next maintenance release of BuildMaster

@grant-sparks_1282 I'm surprised there's no index, but I guess that's a property popular repository, so we can make ProGet not error in this case.

i logged PG-2007 to handle this :)

@nkr said in ProGet shows seemingly random versions as "latest" when using a NuGet connector:

quite an old build of the NuGet Gallery.

Oooh that might cause a lot of headaches/problems

Older versions of the NuGet Gallery have some notoriously strange/quirky behaviors -- so much so that we had a special feed type just handle them ("Quirks Feeds"). We finally removed that feed type last year (ProGet 5.3) after the bugs were mostly fixed.

The "missing dependency info" sounds very familiar, and the same thing happened with packages from nuget.org. It was such a long time ago, and the nuget team fixed it with new code and a data update.

Anyway.. I would recommend just migrating your packages using a bulk import.

Hi @grant-sparks_1282

Maven Connectors work by downloading a file called /.index/nexus-maven-repository-index.gz from the repository. It's weird, but I guess Google's repository doesn't have such an index?

https://dl.google.com/dl/android/maven2/.index/nexus-maven-repository-index.gz

I didn't know that repositories didn't have those... but you can confirm this is indeed the error under Admin > Scheduled Jobs > FullMavenConnectorIndex > History --- Can you confirm, and see it?

I guess it's not a problem, if you can still download artifacts. But without an index, you can't search then...

Thanks,
Alana

Hi @brett-polivka ,

Hmm.. I tried to look at a way we can get more diagnostic info, but I don't think we can find much more. It sounds like is an authentication problem with your PAT?

Did this happen after an upgrade of ProGet? We haven't made any changes to connectors in a while, but if you can tell the from->to versions, we can see if it might be related.

Otherwise, I'm thinking it's on the end of github, and maybe a relation to the PAT? It's possible your token expired, doesn't have right permission, etc. They may have also change the API too.

Thanks,
Alana

Hi @grant-sparks_1282,

Thank you for the very detailed information in sharing the problem and reproduction instructions. Based on this, I'm convinced you've run into this issue: PG-2005 (POM files downloaded from Maven feeds may be saved with empty contents).

ProGet 5.3.37 (which has that fix) is scheduled to be fixed on Friday, but we may release it sooner given the impact. It's holiday in US, otherwise we likely would have done it today already

Anyways, you have two choices:

1. [Recommended] Upgrade to ProGet 5.3.37-rc.2; for the Inedo Hub installation, you just need to switch to the Prerelease Feed

2. Downgrade to ProGet 5.3.34

The only difference between 5.3.36 and 5.3.37-rc.2 is that single issue (PG-2005).

Cheers,
Alana

@arozanski_1087 that's a slightly different caching setting

If you navigate Feeds > Connectors >your conenctor, it will be here:

Clicking on 'status" will show you all the cached queries.

@antony-booth_1029 thanks for the update!

Anecdotally, I've heard of similar issues with AWS RDS. I understand that, behind the scenes, it's some large/shared SQL Server database installation, so I wonder if other traffic impacts it? We've certainly confirmed that was the case in some organizations with their own large SQL Server cluster.

You may find using SQL Server Express + Self Backup is okay enough, too.

Hi @arozanski_1087 ,

Once a package is added to ProGet (either via drop folder, API, web upload, etc.), it should immediately show in the web UI. And it sounds like it does... because you can see the new package in the UI.

However, if the package isn't available through NuGet (which uses the NuGet API), then it means something is using a cached API response.

It could be ProGet. Connectors in ProGet have a feature called Metadata caching, so if you're accessing this feed through a connector, and this feature is enabled on that connector, it would behave exactly how you describe.

It could also be NuGet / VisualStudio, etc. The simplest thing to do would be to just run Fiddler, and capture the communication between NuGet/VS and ProGet. You will see cached responses if this is the case.

It might be a proxy/third-party server. Someone would have had to configure this in your network, and maybe they did to reduce traffic at one point? This should also show up in the Fiddler trace, hopefully, as a cached response.

Thanks,
Alana

Hi @grant-sparks_1282 ,

That's odd... real simple set-up, and I just tested on my end to be double-sure. Looking at the codes, I can't see how he would throw that error either...

What version of ProGet are you using (please see bottom right corner), and how is it installed (LInux/Docker or Windows)?

Thanks,
Alana

Hey @mjc_4927

Since you're already using Docker, you can install a new version of the container...

proget.inedo.com/productimages/inedo/buildmaster:7.0.9-ci.2

Cheers.
Alana

Thanks for sending those over @RobIII

At first I want to confirm, are you using the "V3" API endpoint in ProGet, for your VS2019 configuration? If not, please do that, since VS treats those APIs differently.

Otherwise, there's nothing off about the index files... the most likely scenario is that there is a problematic "translation" between VS2019's API call to ProGet, and ProGet's API call to GitLab. There are more than just those index files that are used.

It's most likely a bug on GitLab's end, in that they're doing something against the spec that just "happens to work" in VS2019. We struggled with a very similar bug with GitHub, but their timeline to investigate/fix it was crazy, so we just did a workaround (see PG-1932) and have it mostly working.

Ultimately, the customer decided not to use GitHub in this manner, because the workflow (outside of the API) was painful and GitHub is buggy they said. So there's that. They just publish packages to ProGet directly instead. You may find this workflow better as well.

Anyways... if you're already using the v3 api endpoint in ProGet, we will need to get this reproduced so we can attach a debugger and find the problem.

I understand it's not feasible to provide access... maybe it won't be so bad to set-up a reproduction case with a public repository? We don't know how to use GitLab's packages features, so if you can help set that up, we can attach a debugger and get it tested.

Thanks

Hi @kenneth-garza_2882 ,

Based on your use cases, an extension wouldn't be a very good fit... and Webhooks are the way to go.

For ProGet (unlike BuildMaster/Otter), extensions are really specialized, and it's pretty rare for folks to make their own, and we're keen on helping anyways.

For example, depending on what your vulnerability scanning software is, we'd love to help and even adopt the integratino, so other users can use it.

The main three things we see are:

• Cloud Storage;
  • https://github.com/Inedo/inedox-aws
  • https://github.com/Inedo/inedox-whitesource
• Vulnerability Scanning / Package Blocking
  • https://github.com/Inedo/inedox-sonatype
  • https://github.com/Inedo/inedox-azure
• User Directory
  • https://github.com/Inedo/inedox-inedocore

And the best way to do those, are mostly to copy the examples/code we already have.

Let us know more details, and we can do ourbest to help!

Thanks,
Alana

Hi @paul-reeves_6112 , sorry on the slow reply, we talked about this in our engineering meeting, but then I forgot to reply to it.

We've seen this come up a few times, but it's not limited to offline installations. We are thinking the problem is on the Windows Service Control Manager side of things (it's not sending the right signal), but we can't seem to pin it down. Our general plan is to add a timeout or something, and hope that it addresses the problem.

Not sure what's going on with the threads though; it might be related to the in-process database we use (Sqllite), doing some background things? I don't know...

Can you let us know if you it keeps happening?

Hi @tomg_5321

Glad you can get some results using the non-filtered search. If there is more demand, we may implement it... but you're the first person mention it.

However, based on your requirements, I don't think you should use search..

my actual underlying requirement is to test if a certain version (which may not be the most recent) of a certain package (by exact id) exists on the server, in an efficient way, for use over relatively low-performance network links (ie, not over local LAN)

In this case, I recommend you just do a HEAD request on this URL: /nuget/<feed-name>/v3/catalog/<package-name>/<version>.json

For example in PowerShell, Invoke-WebRequest -Method Head https://proget.inedo.com/nuget/NuGetLibraries/v3/catalog/inedo.buildmaster.sdk/2.0.0.json

A 200 means package exist, a 404 means it doesn't exist.

Cheers,
Alana

Hi @daniel-cooper_8892 ,

At first, please note you'll need to upgrade to v5.2 first, and then you can move to v5.3: please see Upgrading from ProGet v4.

Of course, you'll need access to your database. Please check out: Connect to SQL Server when system administrators are locked out

Once you do that, you can connect from SQL Management Studio, and then add yourself (and ideally the local Administrators group) as the sysadmin of sql server.

Cheers,
Alana