Posts made by rhessinger

Hi @philippe-camelio_3885,

I have just built a pre-release version (3.0.5-ci.3) of Otter you can install to test out this feature. Please let me know if you have any issues.

Thanks,
Rich

Hi @philippe-camelio_3885,

Agent 49 will work with both Otter 2.2 and Otter 3.0. I'm not sure I understand your second question, but you should be able to use the same agent install on more than one intance (Otter 3 and Otter 2.2) at the same time without issue. Basically, you can add that same agent to both Otter instances you have.

Thanks,
Rich

Hi @Stephen-Schaff,

After looking into this further, the License Detection & Blocking feature is primarily intended for third-party developer packages; Helm charts are generally first-party, and even the third-party charts don't have a consistent license format. Due to the inconsistency in their license format, we are not able to leverage this in Helm charts. We have added a ticket, PG-1937, to remove the display of licenses from Helm charts.

Thanks,
Rich

Hi @Stephen-Schaff,

I was able to recreate this issue. It looks like ProGet is currently not parsing the LICENSE file. Let me discuss this with the products team and I will let you know how we plan to proceed.

Thanks,
Rich

Hi @nicolas-morissette_6285,

Always glad to help! Thanks for giving us an update and letting us know that this fixed your issue. Unfortunately, the package count is a side effect of ADO not implementing the search API. It does not give us the ability to get a package count.

Thanks,
Rich

Hi @nicolas-morissette_6285,

We have released this change in ProGet 5.3.26. I apologize for not replying sooner. If you upgrade your ProGet instance, then you can update your NPM connecter and select the Exact Match Only option on the connector configuration. That will then allow you to see all your local packages and still allow you to search by the exact name of your NPM packages on Azure DevOps Connector.

Thanks,
Rich

Hi @JonathanEngstrom,

You can find a list of the available arguments in the InedoHub CLI documentation.

Hope this helps!

Thanks,
Rich

Hi @nicolas-morissette_6285,

I'm still waiting for feedback from the products team, but my idea is to basically add a setting that allows you to disable listing/searching the packages on that connector and only allow exact name matches in the UI. We do a similar thing for Docker connectors because not all registries support listing or searching. The product change ticket tracking this fix is PG-1916. I will reply as soon as we have confirmed the fix and scheduled it for release. I'm hoping we can get it in for ProGet 5.3.26.

Thanks,
Rich

Hi @joshuagilman_1054,

I was able to reproduce this error and fixed it in ticket PG-1917. We will be making a ProGet release, 5.3.25, later today and this fix will be included.

Thanks,
Rich

Hi @nicolas-morissette_6285,

I took a look into this and what I have found is that Azure DevOps does not support the NPM search API which is why this errors. I created a connector to the Azure DevOps npm feed and then used a username of token, set the password to the PAT (not Base64Encoded), and then selected Basic authentication. The list of all packages will not work, but if you navigate directly to the package or pull through npm, it will access the package. In my case, I was able to navigate to http://PROGET_SERVER/feeds/NPM_FEED_NAME/PACKAGE_NAME/versions and it would load the package's page.

I'm currently working with the products team on a solution. I'll let you know when I have more information for you.

Thanks,
Rich

Hi @jerome-jolidon_1453,

Thanks for the extra information, we will make sure to get the UI fixed to properly show that the package is blocked and we will also get the documentation updated for this as well.

Thanks,
Rich

Hi @jerome-jolidon_1453 ,

If you navigate to the connector's overview page, you should see a link called configure filtering to the right of the Package Filters heading. That will show you your filtering level. I agree with your comments on our filtering documentation. I have also contacted our products team to do a rewrite of this documentation.

Thanks,
Rich

Hi @jerome-jolidon_1453,

Thanks for all of the information, this is very helpful. I will make sure we can get these addressed. Can you tell me what your filtering level is on your connector?

Thanks,
Rich

Hi @jerome-jolidon_1453,

Can you share what npm connector filters you are using? I can verify that @microsoft/* works on NPM filters. I tested using a block of * and allow @microsoft/*. That allowed only @microsoft packages to be downloaded. I will add that if an npm package has already been cached, the download is still allowed. These filters only work if the package is remote and has not been cached.

I will note that I have found a UI bug that will not show the download button blocked for NPM packages, but if you attempt to download the package you will get a 404 error.

Thanks,
Rich

Hi @Stephen-Schaff,

This should not affect anything. If you are using a single domain (did not use multiple domains in the domains to search input), the domain does not need to be appended to the username. So your existing command-line tools should not be affected.

If you also need to be able to prepend your domain (ex: domain\user), I can send you some documentation on how to configure it. The previous login formats of username and username@domain.local will still work, but it adds the ability for domain\username. A small disclaimer though, the domain\user functionality is still in testing.

Thanks,
Rich

Hi @brett-polivka,

It looks like there was a mix-up in one of our branches. I just pushed an updated image with the ParallelEnumerable removed. Can you please give the proget.inedo.com/productimages/inedo/proget:5.3.23-ci.5 image a try?

Thanks,
Rich

Hi @Stephen-Schaff,

Thanks for confirming the extension version for me.

As for the user directory question, if you navigate to Administration -> Change User Directory Did you configure and select Active Directory (LDAP)? We previously had a configuration for a user directory called LDAP or Single Domain Active Directory (now called LDAP or Single Domain Active Directory (Legacy)). I wanted to verify you are not using an old LDAP implementation.

Thanks,
Rich

Hi @Stephen-Schaff,

The Publish Packages task should actually trump View & Download. When I test your steps, it shows me that the user has no effective permissions on the Base feed. To accomplish what you would like to do you should do the following:

1. Create a new Custom Task (in the Customize Tasks). Call it Publish Only and give it the Add Package attribute.
2. Add the user to publish on all feeds
3. Restrict Publish only on the Base feed

That should give you the publish restriction you are looking for. Can you please give that a try and let me know if that works?

Thanks,
Rich

Hi @saurulau,

It looks like your colleague submitted a support ticket with us. Is it OK to continue with that ticket, or would you like me also to work with you through this forum's post?

Thanks,
Rich

Hi @Stephen-Schaff,

I'm having a bit of trouble recreating this issue. Can you answer a few questions for me?

• What version of ProGet are you running?
• Is this installed on Windows or running in Docker?
• Can you please attach a screenshot of the attribute of the Publish Packages task?
  • You can find this by navigating to _Administration -> Users & Tasks -> Tasks -> Customize Tasksand clike on thePublish Packages` task.

Thanks,
Rich

Hi @Stephen-Schaff,

@atripp is correct. There was an issue previously that would ignore the specified type of a principal and favor a group, but that has been fixed for a bit now.

One thing that can still happen is accidentally selecting a group versus a user when adding a permission or restriction to a task. Currently, we just present the list of users and groups in the order they are returned from the domain. If a user and a group are using the same name, it may be difficult to decipher which one you are selecting.

Can you please tell us what version of the InedoCore extensions is installed? Also, can you please verify that you are using the Active Directory (LDAP) user directory?

Thanks,
Rich

Hi @scroak_6473,

ProGet automatically handles creating a temporary API key for use with clair. When ProGet integrates with clair, it generates a temporary key to use to check layers for vulnerabilities then deletes it when the process completes. So there is nothing you have to do to authenticate clair to ProGet. If the key is still there after running the scan, my guess is an error occurred while trying to clean it up.

As for the other error you are seeing, are you able to pull an image with that layer attached to it? I can probably create a SQL query for you to link a layer to the image if needed.

Thanks,
Rich

Hi Joshua,

It looks like the exception is missing the actual error message. I have added some code to improve that error. Could you install Get 1.10.2-CI.1 and see if we can get a better error message relating to what issue you are seeing?

You can manually install this extension by:

1. Download the new version
2. Rename it to Git.upack
3. Copy it to your Otter Extensions folder (You can find the location by checking the Extensions.ExtensionsPath in Administration -> Advanced Settings)
4. Restart the Otter web application (restart the application pool in IIS or restart the integrated web server service) and the Otter Service

Thanks,
Rich

Hi @Joshua_1353,

That all sounds correct. Can you please send a screenshot of your root folders in your Git branch? I'm thinking that maybe there is a folder conflict.

Thanks,
Rich

Hi @Joshua_1353,

I'm having a bit of trouble recreating this issue. I have a few questions for you to help get me started.

• What provider are you hosting your Git repository through?
• Are you using Secure Credentials to connect to this? Or are you using the username and password fields on the Git raft?
• Do you see any other errors in the diagnostics center?
• Did your Git repository have anything in it already or was it a blank repository?

This should help me try to recreate the error.

Thanks,
Rich

Hi @Jeff-West_5638,

Do you see the error in your Diagnostics Center under Administration? Could you please post that error stack trace here?

Thanks,
Rich

Hi Jeff,

The anonymous user has been moved to a built-in account in ProGet now, so it will not show up under the Users tab now. But, you should still be able to add Anonymous to tasks on the Tasks tab. I would recommend removing the anonymous permission you had set up on any task previously and re-add them. When you start to type Anonymous, you should see it pop up as an available user to select. Can you try that?

Thanks,
Rich

Hi Jeff,

Could you please provide a screenshot of what you are referring to with the Anonymous user no longer shows up under Users? Anonymous should be a built-in account and you should be able to add it to any task. Although being a free user, you do not have the ability to have feed level support. In that case, if you set anonymous globally, then all feeds will have anonymous access.

What version were you using in your lab that worked previously?

Thanks,
Rich

Hi @sonismile10_4270,

BuildMaster only supports Visual Studio Test (formerly MSTest), nUnit, and jUnit out of the box. I'm not very familiar with SOAP UI, but I think your best bet would be to use Soap UI's jUnit integration and then run the jUnit tests to track them in BuildMaster. If not, you would probably need to build a custom BuildMaster extension to support this. Please see our unit test documentation for more information.

Thanks,
Rich

Hi @markus4830,

I definitely recommend changing your NuGet.org connector to the V3 endpoint simply for the fact that NuGet.org is deprecating portions of their V2 API. The V3 endpoint on NuGet.org is also much faster than the V2 endpoint and the V2 endpoint on NuGet.org currently throttles package connections.

On the same note, I recommend the same thing for connecting to ProGet as well. Your error message above shows that your build server is connecting to the V2 endpoint for your ProGet NuGet feed. The V3 API is much more efficient in both database connections and overall SQL execution.

If your NuGet feed in ProGet is still showing a V2 endpoint address, you will need to go to your feed's Manage Feed page by clicking the Manage Feed button and change the supported API to include JSON-LD (v3) . Your feed will still support V2 connections, but you should migrate all your builds to access it over the V3 endpoint.

I feel the combination of those two pieces should help bring down the number of concurrent connections to SQL server.

Thanks,
Rich

Hi @markus4830,

Thanks for sending that over. The Close Database Connections Early value was removed because we enabled that for everyone which helped this issue initially. Can you remind me, are you using the Docker version of ProGet or do you have it installed on Windows?

For the (big) build, is it multiple builds triggering at the same time, or is it just a single massive build with a lot of packages associated with it? I'm also noticing that you are using the NuGet v2 endpoint. Is it possible to switch your build to use the NuGet v3 version of your endpoint?

Thanks,
Rich

Hi @markus4830,

I think the first thing to check is if you are actually running out of connections or not. Can you please run the following SQL next time you see ProGet experiencing this problem? Can you also run it under normal load when everything is running normally?

SELECT DB_NAME(dbid) as DBName, COUNT(dbid) as NumberOfConnections --, loginame as LoginName
FROM sys.sysprocesses
WHERE dbid > 0 AND DB_NAME(dbid) = 'ProGet'
GROUP BY dbid --, loginame

I commented out the loginName, but you can uncomment that if you want to verify the connections are coming from ProGet versus other users connected to SQL server.

What is the number of connections you are seeing when you run that?

Thanks,
Rich

Hi @jn_7742,

This fix will be released in ProGet 5.3.21 which is due out January 22, 2021. When you upgrade and test this, can you please reply back and let us know if this worked for you?

Thanks,
Rich

Hi @viceice,

I took a look at this and it looks like the issue is that the GitHub repository is not returning the published date. I have put a fix in, PG-1871, which will release in ProGet 5.3.21.

Thanks,
Rich

Hi @viceice,

This typically happens because the NuGet package metadata has an invalid format. Does this error happen for all packages or just one?

We run into similar issues when connecting to Azure DevOps registries because they do not always follow NuGet's API spec. Are you able to send over the package registration metadata for a package with the issue? The index.json may also help with this. If you do not feel comfortable submitting this via the forums, you can email it to support@inedo.com. Just please let me know when you send it so I know to look in that mailbox for it.

Thanks,
Rich

Hi @Crimrose,

Thanks for posting a follow-up solution for this! I'll keep this in our notes incase this comes up again in the future!

Thanks,
Rich

Hi @nicolas-morissette_6285,

Thanks for sending that over. This confirmed my thoughts. It looks like they only include the SearchQueryService/3.0.0-beta, not the SearchQueryService, which is why the v3 connector does not work. I'm going to talk this over with the internal team and see how we want to approach it. ProGet currently does not implement the 3.0.0-beta API endpoint because it is still likely to change, which increases the likelihood of it breaking in ProGet.

I'll discuss this internally and figure out the best way to proceed.

Thanks for sending this over!

Thanks,
Rich

Hi @nicolas-morissette_6285,

Would you be able to post the JSON from your v3 API? It looks like AzureDevOps is missing the URL for the SearchQueryService. If you navigate to https://proget.inedo.com/nuget/NuGetLibraries/v3/index.json You'll see the very first service API URL is for SearchQueryService. I'm wondering if Azure DevOps is implementing a weird version of it, if at all. If you don't feel comfortable posting it here, you can email it to support@inedo.com, let me know, and I can pull it from there.

I'm sorry for the confusion. Since you originally posted the values from the registrations-gz, I made the assumption you were using the v3 API. The fix I added was for the V3 API JSON. I'll have to see what I can do with the V2 version because it parsed a little differently than the v3 version.

Thanks,
Rich

Hi @nicolas-morissette_6285,

Thanks for the updated information. This is most likely not a configuration issue with the connector. The only thing that could be affected by the connector setup is if you are using a v2 API NuGet feed instead of a v3 on from Azure DevOps, but I believe you already confirmed above that it was the v3 API from Azure DevOps.

Let me look into this a bit further. It looks that I must have changed the code that the UI uses, but not what the nuget.exe uses. I'm not sure how that is possible, but let me take another look.

Thanks,
Rich

Hi Simon,

Unfortunately hitting the URL in the browser will not show the underlying error because the Docker client is a relatively chatty client. It will hit a number of different API endpoints to perform the login. You would need to use a tool like Wireshark or Fiddler to proxy requests to the server to see which URL has the problem.

Are you using nginx as a reverse-proxy to ProGet? Is it possible to bypass it to try to use docker login directly on the ProGet server to rule out any header manipulation issues?

Thanks,
Rich

Hi @nicolas-morissette_6285,

You should not need to do anything to the connector. Can you try and open the package in the ProGet UI? Does that package/version load? Also, can you download it within the ProGet UI?

Thanks,
Rich

Hi @scroak_6473,

We have fixed the issue with creating new AD User Directories and vulnerability sources in .NET 5. It is set to release in ProGet 5.3.19 which is due out December 18, 2020.

Thanks,
Rich

Hi @nicolas-morissette_6285,

No problem. Thanks for moving the comment over here! I will delete it from the other topic.

I can confirm the fix I made was released and is currently in the code. Do you see any errors in the Diagnostic Center in ProGet?

Thanks,
Rich

Hi @scroak_6473,

For the add new User Directory and add new Vulnerability source, we have identified the issue and we are currently investigating the cause of it. It has to do with changes in .NET 5 for parsing types from strings. I will update this when we have a bit more information on it.

As for the AD stuff. That is definitely the issue, the LDAPS certificate is not being viewed as a valid certificate. I think we will need to investigate this further on how to bypass certificate validation. I would prefer you to open a new thread to discuss this issue with LDAPS in Docker. To answer your question, our code for connecting to Active Directory exists on GitHub in the inedox-InedoCore extension. It also includes a windows application for testing AD connections, including LDAPS. The library we are using on .NET 5 is Novell.Directory.Ldap.NETStandard. For the .NET 4.5.2 target, we use the typical System.DirectoryServices. We always encourage Pull Requests with fixes for this also!

Hope this helps!

Thanks,
Rich

Glad to hear the workaround works. We are still currently working on the issue to fix the Add New functionality in .NET 5. To the best of my knowledge, LDAPS is currently working and we have users that are using it currently. Can you please tell me what issue you are having with LDAPS?

Thanks,
Rich

Hi @scroak_6473,

I was able to recreate your issue. I think this may be an issue with the .NET 5 version of ProGet and creating a new Active Directory (LDAP) user directory. If one is already created, I do not have any issues editing it.

As a workaround, I would suggest changing over to the progetmono image and create the new user directory, then change back to the proget image to configure it.

I will reply back when I have an update on a proper fix for this.

Thanks,
Rich

Hi @shijiyong_6709,

I just wanted to confirm that you are using docker login -u admin https://proget.company.com and not docker login -u admin. Is that correct?

The first thing we need to figure out is if the problem exists in your ProGet configuration or in your nginx configuration. Can you start with temporarily allowing anonymous access to your Docker registry in ProGet, does Docker pull work then?

Thanks,
Rich

Hi @arozanski_1087,

That's great! I'm sorry it took this long to get this working. Thanks for working with me to figure this out! I'm going to make a change to the documentation right now to include the requirement of naming the extension file. I will also get a full release of this extension out today so it will lock in that version with the ProGet release tomorrow.

Thanks,
Rich