Posts made by atripp

In this case, please ensure that the ProGet service is running and has permission to read and delete from that folder (Admin > Service). You can manually trigger the drop path monitor from that page, and see what he's doing behind the scenes.

Great!! I've logged this as PG-1798, and it's planned for 5.3.10 (Aug 28), but may get delayed depending on other priories.

hi Simon, thanks for the suggestion.

I'm not so familiar with the ping command, and i didn't find much on searching "docker ping command". Do you have any information on it, is it a kind of standard?

Or is this the usual ping that you use to like, ping google.com or something?

Cheers, Alana

Hello;

ProGet does support SNAPSHOT versioning, and when we test it, there's no problem. So I'm guessing it might be a problem with your POM file, repository configuration, or naming conventions?

They are a bit complex to get right, and the conventions have to be perfect; https://maven.apache.org/guides/getting-started/index.html#What_is_a_SNAPSHOT_version

Hi @scroak_6473

Shouldn't there be an option to "ignore or skip certificate verification"

If it's possible at the library level, it's something that could be added, I think; we recently added the LDAPS support. I'm not an expert on security/certification verification, or if this is even a good option to have, however.

The VerifyServerCertificateCallback may allow it, and it seems like it could be an easy checkbox to add to ADUserDirectory.cs, maybe.

It's definitely possible to fork InedoCore, modify, then manually install. Or we can very easily shipped branched-builds to our CI-extensions feed, if this is something you'd be interested in collaborating/testing. This is how we got LDAPS working in the first place, as it's quite complex to reproduce an environment for that.

Cheers,
Alana

Oh no! Well, I can do this -- I can see the email of your new account ( @JonathanEngstrom), and I think I can just delete your new account, and set the email of your old account @Jonathan-Engstrom?

Let me know if it's okay, and I'll try that.

Can you try it again?

It was a setting on the Forums admin side, it was disabled for change by regular users.

I see; that makes sense, the re-indexing would delete it.

Hmm, okay, I just thought about another work-around instead. Please download v12 package, edit the .nuspec file, rename it to be cased like the others, and then upload it? This ought to solve it, as well.

It's definitely a bug we can look to fix, but thus far it's impacted just a single user on a single package in several years so far. It's a high-risk change, and we have a lot of things on the backlog that add more impact/value to the users.

@scroak_6473 thanks Simon!

I added PG-1790 to our system; one blocker I have... how should we document this?

We'd love to even give an example of more advanced configuration, like how you have, of the Docker Compose file. Here's the current docs we have.

https://github.com/Inedo/inedo-docs/blob/master/ProGet/installation/installation-guide/linux-docker.md

If you can docs suggest or do a Pull Request that would help us get this done ASAP :)

Thanks, got it. An annoying work-around for sure, but it somewhat makes sense, because the package name must be decided based on the first package in the list? I don't know... but the right-answer is to store everything as lower case and have a fallback for mixed-case (which is what we did in the cloud storage).

If no cache is set for the connector, none of this should technically matter, correct?

The problem is occurring when the package is being loaded some disk, so it would impact Cached and Local packages, but not Remote packages.

I think dot net publish, but remember these are really meant to work with nuget.org. This might be an issue to take up with the NuGet team; it's very possible their tool doesn't support this use case and is buggy.

It's just an HTTP PUT, so you could do it in a line of PowerShell as well using the Invoke-RestMethod CmdLet: Invoke-RestMethod -Uri $uri -Method Post -InFile $uploadPath

Here is some more information from NuGet's docs: https://docs.microsoft.com/en-us/nuget/api/package-publish-resource#push-a-package

@christian_panten as I mentioned, we recommend making a single symbol package to avoid this kind of headache (it changes between versions of nuget CLI ), and I don't use nuget.exe to push packages in this manner so I can't say it

If you were to just upload or PUT the package to ProGet, then it would work fine. So we are just trying to figure out nuget.exe's quirks, and why he refuses to push to a custom symbol source...

The screenshot you showed seems to be fine. I also read the nuget.exe push documentation.

I don't think you're supposed to specify the snupkg file. I believe he will search for a .snupkg and then push it? It's hard for a guess.

I also remember this was broken for a while in nuget.exe (not using of SymbolApi at all), but it works in dotnet push.

Thanks; so just so I understand, if you made a properly-cased version of the v12 package file, it worked, for both versions?

It's not ideal, I know. But this is situation must be extremely rare (first report in many years), and fixing it is costly/risky, so we need to weigh cost/benefit (especially when there are a lot more valuable things we can could improve in the software) against a workaround.

@scroak_6473 great suggestions, thanks!! The mockups will really help me to present a case :)

I see that the priority is on finding the unassessed vulnerabilities, which make sense. I don't know about a "mouseover" (we don't have this UI construct in our products like this to make this easily doable), but I can envision a modal window (popup) or a regular page that allows for quickly asessing those vulnerabilities

This isn't trivial, but it's not terribly complicated either. I'm going to try to get this submitted internally next week (I'll share what I write up), and from there we might be able to get this in the following or near-term maintence release

@wsah_6160 @pluskal_4199 assuming this is related to 5.3, then there should be a fix coming soon!

hi Simon, thanks for the suggestion!

I'll be honest, I'm really not that familiar with Docker Swarm or Secrets, but I wonder, from the "code inside ProGet perspective", does this seem as simple as like,

// fake code, just an example
if (EnvironmentVariables["proget_connection_string"] != null)
   return EnvironmentVariables["proget_connection_string"];

else if (EnvironmentVariables["proget_connection_string_file"])
   return File.ReadAllText(EnvironmentVariables["proget_connection_string_file"]);

else 
  return ReadFromNormalConfigFile();

First off, if you want to have separate library and symbols packages (which we don't recommend), then you'll need to make a separate symbols feed. ProGet supports a "combined package" and will strip out symbols/source unless explicitly requested, so you don't have to bother.

But if you want to separate, please note that nuget is a "little" funny. If the file extension ends in .snupkg then it ignores the source argument, and attempts to push the file to Microsoft's server (symbols.nuget.org).

This is because symbol packages and regular packages must be pushed to a different feed. If you want to specify a custom symbols feed, then please use the --symbol-source when using a .snupkg file.

@christian_panten I have a favor.... can you suggest (or do a pull request) on how we can update our documentation to make this more clear? Thanks

ProGet has a really cool feature called Semantic Versioning for Docker Container Images, and it works pretty much how you described, but it also enforces that containers have a valid semantic version tag. For example, latest will always refer to the highest, stable-version of a container, and 4 will be the latest stable of 4.x.y

As for "generic archives", check out Universal Packages - they're really powerful, and are like "nuget/maven" but for your own applications and components, and extensible.

Hi all, just an update! We will be shipping a potential fix in PG-1783, which adds a new checkbox in advanced settings (unchecked by default):

Close Database Connections Early
EXPERIMENTAL (for debugging/testing only) - As of ProGet 5.3, database connections are left open during the lifecycle of a NuGet API request as a means to reduce overhead; however, this may be causing ProGet to run out of available connections to SQL Server. Set this value true to open/close database connections as needed on NuGet feeds.

We'll update when this is shipped --- but if we can get some folks to verify that this works better (we can't repro, at all ), then we will likely make it the default. Hopefully this will do it. Seems better than raising connection pool limits

If so, then the savings in connection open/close overhead don't seem to make it particularly worthwhile. This "keep open" technique made a ton of differences elsewhere in our software, but since a NuGet API requests may yield a ton of other network requests (via connectors) and block, the pool may be getting drained too quickly...

just a theory, as I mentioned. Anyway hope this helps, stay tuned!!

Thanks!

I see; it seems the problem is the casing change from v12 to v17 of the package.

I assume you are't using both at the moment? As a work-around, can you try deleting v12 of the package, then renaming folder to WPF instead of Wpf?

Will that work?

Thanks, you have have a pretty good point here. Finding where the vulnerabilities live is kind of difficult, but let's make it easier

First, bc9ab73e5b14 is a layer that's in one or more container images that has zero or more tags in a repository (in a registry/feed).

What's actually useful information is registry (feed), then repository+tag (containername:version).

If all this added up to a single tag in a single registry + repository + tag combination, we could display that instead. But there are going to be a lot of container images using that layer...

Maybe clicking that page opens up a page that is like, "tags that use this layer" or something, and it displays Registry (Feed) and REpository+Tag in a simple list view?

The only way to get to this page would be clicking on a image hash like that, so perhaps it could be a modal-popup window instead?

Just brainstorming... what do you think?

At first, I wouldn't worry about connector timeout errors to nuget.org unless it's frequent. They happen due to service or network outages (your end or theirs), nothing to worry about too much.

But otherwise it's hard to say; are you totally sure that the error is coming from that connector? Maybe you have multiple connectors to nuget.org?

Otherwise, nuget,.org's JSON-LD API (v3) sometimes points to the nuget.org ODATA-based URLS (v2), though you'd have to follow the URLs returned by the v3 API to see that.

It's definitely possible; we've seen some situations in Cloud Storage where a package's casing changes between versions, and this is the result. But it only impacted replication to our knowledge.

The cloud storage package stores were modified to fix that, but I suppose it could happen on Linux-based as well, but this is the first report to my knowledge.

Can you confirm this is happening? It's a very risky change (unlike the cloud storage, which is easy to rollback becuase it's in an extension) that fixes a very small edge case, so we need to be extremely careful in fixing it. It's not clear what's being impacted here, aside from the package verification job...

Hi Philipp;

Glad you were able to find the feature of Repackaging, it's quite helpful!

Now, when the WPF Application is going to be tested, we'd like to generate a whole tree of all those dependencies labeled as "-prerelease".
And after testing we'd like to move the whole tree into the production state where all packages do not have any prelease tag at all.

This sounds good to me! I think this process makes a lot of sense to do.

I'm not sure if you're asking about the ability to tell ProGet "repackage the WPFApp package and all it's dependencies", using one API call or something? In this case it's not possible.

The reason is, it's not feasible for ProGet to handle the dependency resolution. Actually it's quite complicated (especially with version ranges possible), and NuGet's algorithm is among the most complex in the industry. Because of this, it's not really possible to do dependency resolution from ProGet, since you'd need to specifying additional context including what framework version you're targeting and what version of Visual Studio you're using, etc.

But anyways, the idea you have to promote all packages in one stage sounds nice to me!

What you're describing is a "drop folder", which ProGet has for some feed types -- they're not supported for asset directories, however.

Do note that there is an Import Directory Endpoint in the Asset Directory API where you can publish a ZIP file that gets automatically extracted.

In this case, you can just install on top of the old installation -- the database or package store won't be overwritten.

But do note it might impact the IIS/application pool customizations you made.

Hello; you might be able to use an Asset Directory for that, but note that they don't support connectors at this time.

Hello,

Please see the guide over here - https://docs.inedo.com/docs/proget/feeds/nuget/symbol-and-source-server

If you wish to use a separate symbols package (snupkg), then you will need to create a separate feed, and push the symbols package to that feed.

@scroak_6473 thanks, I just updated that too, hopefully this will help :)

Thanks for the report , very simple code typo and now it'll be fixed in BM-3611 in the next maintence release

Hello, what version did you upgrade from?

Can you also check the version of the Windows extension you had loaded?

Hello;

There are a few things that could cause re-activation of ProGet, or any of our products:

• The software was moved to a different system
• Changing or adding a license key
• The software was installed on virtual machine that changes CPU IDs or network interfaces when it restarts (to prevent this, configure the network adapter of the virtual machine hosting the software to be a static value)
• A new network interface was added
• Upgrading to a minor or major version of the software

The License Key Activation Documentation explains in more detail as well.

Hope that helps,

Alana

Thanks!! I fixed it.

Hello; due to limitations of the DockerHub, we no longer distribute the software on their registry. Instead you can get it from our own instance of ProGet :)

However it's available here: proget.inedo.com/productimages/inedo/proget:<version>

https://docs.inedo.com/docs/proget/installation/installation-guide/linux-docker

If you can suggest where to better notificy users of this change, please do :)

Hello; thank you pointing this out, this is a documentation bug, and that page wasn't updated. But I just fixed it now.

Configuration files now use the OtterScript/ExecutionEngine to handle replacements, so please use the grave apostrophe (`) to escape, so in the case, it's \$myvar `

  <variable name="fileNamePrefix" value="`${var:logRoot}/`${shortdate}" />  

Cheers!

Hello, great question.

.indeox deployment is no longer supported, so I updated the documentation as follows

Direct Deployment {#zip-file}

An extension can be directly deployed to an Inedo product's extension root as a simple universal package.

To do for an extension named MyExample, first create a manifest file called upack.json with the following contents:

{
  "name": "MyExample",
  "version": "1.0.1"
}

Then, create a zip file called MyExample.zip with the compiled output in a package folder, and the upack.json at the root, like this:

  /package/
     /MyExample.dll
     /SomeLibrary.dll
  
  /upack.json

Rename the the zip file to MyExample.upack, and copy the .upack file into the Product extensions directory. By default, this will usually be in C:\ProgramData\«product-name»\Extensions, but you can verify the exact location by going to the Admin->All Settings page and looking for the ExtensionsPath value.

Finally, restart the product services (and application pool if hosting in IIS).

hi Simon, not too familiar with usage of tzdata or TZ: variables; do you have examples of how these get used or added in containers, especially when we rely on system-provided time. I can make a suggestion to the product team, if it's easy.

hi Simon,

This records are maintained via Package Usage Scanning, which is something that routinely scans external resources for packages, and updates the status when a package is or isn't installed.

It's a little different from deployment records, in that deployment records are historic data and package usage is current data.

While you could write your own PackageContainerScanner by using the Inedo.SDK, Otter can collect the packages that are actually installed on various servers; the Otter-usage scanner (buitl in) simply queries to Otter about the status of those scans.

A server cluster can have up to 5 nodes; so in the case of a HA cluster, then it'd be the $10K for servers (But you could have up to 5).

Hello;

The pricing for ProGet is available on the ProGet Pricing Page.

ProGet Enterprise can be installed as a server cluster, but you'd really be best requesting a quote, because there are a lot of configurations, and we'd want to better understand your use case before we share to you our recommend configuration.

Alana

Hello;

The InedoHub just edits that ProGet.Config file; but in any case, those bindings are only used for the integrated web server, not IIS. We should definitely improve that experience so it's clearer. Are you able to access your instance via http://local.proget.net/ in a browser? If so, then, it seems to have worked.

I don't know npm very well, but I looked at the docs of [npm publish](https://docs.npmjs.com/cli/publish), and it seems that --registry isn't an option on that page. But there's also this page: https://docs.npmjs.com/misc/registry

Hope that helps

Great, thanks!

What specific information is missing from the "landing" page? Even the "old" package page has many tabs, so not everything is displayed.

I'm not sure where to troubleshoot either... ProGet doesn't have any control over upload/download speed, it just uses as fast of connection as possible.

I haven't seen it before. It must be network related, but I'm not so experienced on networking. I think if you search for like, "slow upload/download from docker container" maybe you'll find a lot of advice? Please share what you find

Hello; thanks for the feedback.

In all of the "new view" testing, we received positive feedback from users; and since release, the feedback has almost entirely positive. Our goal was be load faster, break up the relevant information more logically, and simplify the code (thereby reducing bugs in the future and enabling much quicker improvements). However, changes are changes, and they take some "training" to get used to, even for us.

On the "List Packages" page, there are now two links: List Package Versions and Package Package Version:

We may add an enhancement, like "Latest Pre-release" version or something on this list as well, so you can see, there's a lot of opportunity for adding even more relevant info in the future.

I'm guessing, you're simply trained to click on the bigger link (as I am), and aren't clicking on the smaller (version). Do note, on the List Package Versions (the bigger link), we will add more relevant information, such as information about the latest version and pre-release version, vulnerabilities, disk size, etc.

As for the "All Versions", that is now in the bread crumb:

It takes "training" as well. This is where we're hoping the "List Versions Page" will provide a view that will be more informative.

Please continue to share suggestions about improvements, and we'll do our best to make them happen quickly!

As for the "old view", it will be removed at some point... the reason both "old" and "new" are available are so that we provide functionality, and work to improve "new" one.

The JSON-LD (v3) API is available in ProGet 5.3 and later, so please upgrade. We'll update the documentation to clarify that!

Hello; this issue is known now, I've logged an issue and we'll get it fixed in the next release; PG-1727

Release is scheduled for Friday, but we may ship it sooner.

Ah, I can see that.

So, in this case, you'd want to:

• Grant access to Feed API
• Specify a Username to impersonate with the appropriate access to feeds/tasks

In this case, you'll need to remove the Feeds_OverwritePackage privilege from the API Key or User that's doing the push. The behavior you want/expect is not possible otherwise.

nuget.exe does not send any information to ProGet about whether packages should or shouldn't be overwritten; the --skip-duplication option merely treats certain server errors (i.e. a 409 error) as a warning instead of a failure.

Thanks for the very detailed analysis.

The NetBIOS mapping is only used for Windows Integrated Authentication, and to convert names like SOMTHNG\username to username@something.local.

Unfortunately, as you noticed, the "Search recursively" can be painfully slow in some configurations. We're not sure why, it doesn't seem to necessarily be related to the number of groups, or the nesting levels.

The SearchGroupsRecursively option (see code) does not attempt to extract all groups, it simply crawls up the group tree; this is the only way that we could find to get users' nested groups, and we even asked to Microsoft who confirmed this was the way to do it.

HOWEVER, when I say "slow", I mean on the order of "30 seconds", like your PowerShell script. 9 minutes is crazy! Even 5 seconds to load a page, without that option, seems very slow, so there might be some underlying networking/configuration issues?

In any case we can make a few optimizations to recursive searching, but you shouldn't bee seing 5-second page load times in any case.

Sounds neat, we'd love to learn how you have it set-up! This is the first request for this, and like all feature requests, we want to make sure we can document it well.

So I wonder, can you think of a way to improve our documentation with an example of your Stack File, and some mention how you're suing docker swarm to monitor things?

Maybe as a subpage? Here's what we have now:
https://docs.inedo.com/docs/proget/installation/installation-guide/linux-docker

In any case, we are definitely open to collaborating with ways to improve not only our API, but our docs. Seems this might be a great way to work together?

We honestly don't anything but basic knowledge on swarm / Stack Files, so if you could write how you set it up / monitor it, we will be happy to fix the code to work with the docs. I suspect changing the API to work as you requested would be infinitely easier for us than figuring out Docker swarm :)

Also --- please note that we are now using proget.inedo.com instead DockerHub, it seems a lot of other vendors are doing the same, and it's easier for us to maintain our own build process (not possible on Dockerhub).