Posts made by atripp

Hi @paul_6112 ,

How can I justify to the security teams that I am running unsupported software version

Well.... technically none of your software is supported since you are a free user

But otherwise you can either upgrade the agents or link them to this post where an Inedo engineer advised it's fine in your scenario. Or if you can suggest a way to clarify the docs... we recently reformatted that download page and split into "supported" and "unsupported".

So I can actually test that the automatic upgrade does work, which version of the Agent should be installed to prove it would take place ?

Without digging in the code I'm not sure, but I'm pretty sure this functionality was disabled in BuildMaster 7 - the sentence mentioning it in the docs was obviously not removed. Instead, you will likely get some kind of warning on older versions.

The failure rate was relatively low (~0.1% or so), but it's high enough that it's not worth risk for customers.

Thanks,
Alana

Hi @paul_6112 , we could definitely be more consistent here.

The listing is generally called "Deployments & Executions" , which we shorten to "Deploys".

I'll make a note in our roadmap to review this!

Hi @paul_6112 ,

It sounds like there is some kind of proxy issue. We will investigate this via the other ticket.

We don't plan on removing Inedo Samples... just the "Application Templates" feature, which basically allowed users to maintain a gallery of your own templates or samples.

Cheers,
Alana

Hi @paul_6112,

There are no issues w/ using v49 or v50 in newer versions of BuildMaster; the bug is just with older (but still supported) versions of BuildMaster.

The "critical bug" is that an improperly-formatted encryption key in the agent's configuration file will cause the agent to not crash, but instead fallback to unencrypted mode. Newer versions of BuildMaster will detect this configuration problem. v46 does not have this bug.

Cheers,
Alana

Hi @paul_6112 thanks for the report -- this is a known issue/regression in our internal UI library with focus- it only impacts a very few number of pages, but it's on our list to eventually address

Hi @paul_6112,

Thanks for the report; looks like this is a regression that can happen after upgrade; to work around this

1. Go to Admin > Resources
2. Click Add Secure Resource, then "Show All"
3. Select "[Deprecated] Universal Package Feed"
4. Name of Templates and API Endpoint Url of https://proget.inedo.com/upack/BuildMasterTemplates/

You can use a different URL if you have a ProGet universal feed of course.

This will be fixed in next maintenance release via BM-3908.

This should solve the issue. And as an FYI, application templates are now deprecated in favor of "cloning" a "template" application.

Hi @paul_6112 ,

I just updated the documentation to clarify our Inedo Agent Upgrade guidance:

You shouldn't upgrade unless you are directed by a version of your Inedo tool (Otter, BuildMaster, etc) or an Inedo support engineer.

In this case, you wouldn't be prompted to upgrade if you're using v46 or v49, so we really don't suggest upgrading. There's zero benefit and a nonzero cost (time to upgrade, risk of problems, etc).

Keep in mind that the Inedo Agent is really just a lightweight "agent host", and the "actual agent" is upgraded transparently all the time.

Cheers,
Alana

Hi @paul_6112

In that case, you may want to try restarting the service? It's possible the proxy change didn't get reflected...

Thanks,
Alana

Hi @paul_6112,

It definitely looks like the proxy settings need to be configured/changed. You can configure and test the Proxy settings under Admin > Proxy. That page also has a "Test" button that you can try different URLs.

With a Proxy configured and allowing;
my.indeo.com
proget.indeo.com

Now this is probably a typo on the forums, but just in case not... inedo.com not indeo.com

Hope that helps,
Alana

Hi @neo ,

If you haven't seen it already, here is our guide for migrating a ProGet instance:
https://docs.inedo.com/docs/migrate-a-proget-installation-to-a-new-server

That generalizes our advice as much as possible. And since you're using such an an older version of ProGet, you should also upgrade this as well.

Note that our professional services team can usually handle this for you for a very low cost; the service isn't listed on the page, but we offer it via the support channel.

Product Upgrade/Migration Service

Flat/fixed cost of $995. Eligible to ProGet users through Dec 31, 2023. Our professional services team will help every step of the way. Often, a migration/upgrade involve a lot of steps and back-and-forth, including:

• Meeting with your team to plan the migration/upgrade
• Review existing configuration and report on issues, like permissions, missing backups, security concerns, etc.
• Help resolve issues before migrating/upgrading
• Create a testing and rollback plan
• Answering questions/concerns before
• Execute the upgrade/migration plan (Assuming we have the access, otherwise we will direct where to click.)
• Follow and addressing issues after the fact
• Documenting issues and concerns for next time

If that's of interest, please fill out the form on this page (select "Other services not listed"):
https://inedo.com/professional-services

Cheers,
Alana

@Srinidhi-Patwari_0272 here is some information about HTTPS support on Windows:
https://docs.inedo.com/docs/installation-windows-https-support

However.... as Dean mentioned, this is something you'll need to work with your network team (a domain administrator) on, as the cert needs to be trusted, etc.

Hi @Srinidhi-Patwari_0272 ,

I'm not totally sure I understand, but when you do a silent install you need to specify a connection string.

So you would just specify the connection string you need like this I guess

hub.exe install ProGet:5.2.3 --ConnectionString="Data Source=externals,qlserver.corp.local; Databse=ProGet;Integrated Security=True;"

Hi @nicholas-boltralik_3634 ,

One thing I'm thinking is that this is a UI-bug, in that an option is set on the rule that cannot be edited in the UI; can you try deleting that particular rule, then re-adding it?

Otherwise There's no easy way to delete these;

• You could tag them, and then browse in the UI, and delete.
• Or you could do a DELETE request using the API, but that's not really that easy to do due to how Docker API auth works.

But we should be able to fix this, just need to figure what's causing it.

FYI -- The bug fix you mentioned earlier ( PG-2477) had to do with tag matching. The specific case was, if you had the rule "Delete Images (not requested for 10 days, matching 0.0.0-*) from the feed" set-up, then an image with BOTH 0.5.0-abc4.1.1-posse7.4.2.2 and 0.0.0-issue-PC3-1507 would not be deleted. I don't think it's related.

Thanks,
Alana

Hi @cshipley_6136 ,

When you specify dbcache=false, then ProGet will immediately invalidate the cache used for database connection-testing. This is otherwise cached for up to 5 minutes. Otherwise, there isn't anything that ProGet (as in the code we write) that would cache or hold any connection.

So, this is ultimately related to ADO.NET's internal DNS caching / connection pooling /etc., and it would likely resolve within a few minutes on its own. But it's hard to say exactly what's going on. ADO.NET is the "driver" that's used by all .NET-based applications to connect to SQL Server, so I suspect this is solvable with configuration of some kind.

Unfortunately, I'm not sure on your configuration nor advanced SQL Clustering scenarios. On our end, we just "let the driver" do the work and follow ADO.NET best practices with regards to driver usage.

I did some initial searching, and I wonder if you just need additional configuration on your server connection string, like MultiSubnetFailover =true. They discuss a lot of information on it: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/sqlclient-support-for-high-availability-disaster-recovery

Happy to help parse some of it, but please keep in mind this is a "driver" issue and we're learning about the internals for the first time, along with you. But the MultiSubnetFailover may help for a faster cut-over??

NOTE: if / failed but /health didn't, then it's most certainly related to this internal connection pooling, and it wouldn't be consistant at all. I would most definitely not hit the / page for a health check, as that will introduce a lot of load on your instance and cause reliability issues.

@nicholas-boltralik_3634 thanks for sharing the other details; nothing is jumping out as off to me in the code (and i can't reproduce this with a trivial case), so please give us some time to do a little more investigation - it's most certain we won't get this in the upcoming maintenance release (this Friday), but we have one on Nov 17 that hopefully we'll be able to figure this out in

Hi @A-Schoder_7166 ,

It sounds like your installation is somehow corrupt?? I can't imagine what would cause that error but it's basically saying that the upack.json file within the extension file (xxx.upack) is corrupt.

Maybe some antivirus tool went and "quarantined" zip files on disk that contained executable code? Really hard to guess. The file sare loaded , typically in c:\Program Files\ProGet\Extensions, and they are part of the installation.

I would uninstall and reinstall, and hopefully that will fix the issue.

Thanks,
Alana

Hi @nicholas-boltralik_3634 ,

Do you have anything else defined on the other tabs? That could have an impact if so...

Otherwise, best way to troubleshoot this would be to look at the retention logs. That should give an idea of what's happening in the policy. And then if you can find a specific example of an untagged image, that would definitely help to track it down...

Cheers,
Alana

Hi @mharen,

You can try to enable Windows Integrated Authentication in ProGet, and then use two sites in IIS (once with WIA, one without), but that will only work for NuGet packages. Other client (npm, docker) do not support WIA.

Otherwise, you can take a "DMZ" approach, and configure two instances of ProGet - one for Devs, one for CI systems - then use a connector. The two-instance approach also has the benefit of reducing load, though not quite as much as a load balancing configurating.

Cheers,
Alana

Hi @jw ,

Thanks for reporting these!

We'll get the fixed in hopefully the next upcoming maintenance release via PG-2520 and PG-2521

Cheers,
Alana

Hi @lucas-almeida_8120 ,

This error implies that there's some kind of database corruption or other error that happened while updating the database a while ago. But it's hard to say; this is the first time I've seen this error.

Unfortunately, unless I were to analyze a backup of your database and try to reproduce the installation/upgrade, I won't be able to offer any guidance or scripts that could repair the state of things.

From here, I would recommend rolling back to whatever version was working, then starting a new ProGet server and importing your packages.

Best,
Alana

Hi @Srinidhi-Patwari_0272,

To accomplish this, I would recommend writing a script that does the following:

1. Installs SQL Express with the configuration you desire
2. Download latest version of Inedo Hub
3. Run hub.exe to perform silent installation

Installing SQL Server in an automated manner is not trivial, and you will need to research the best way to do that in your environment. Here is an article I found when searching for that: SQL Server Unattended Installation with PowerShell.

AS for steps #2 and #3, here is an example script from our Silent/Automated Installation Guide that you can adapt as you need:

# create working directories
mkdir C:\InedoHub
cd C:\InedoHub

# download and extract file to working directory
Invoke-WebRequest "https://proget.inedo.com/upack/Products/download/InedoReleases/DesktopHub?contentOnly=zip&latest" -OutFile C:\InedoHub\InedoHub.zip
Expand-Archive -Path InedoHub.zip -DestinationPath C:\InedoHub

# perform silent installation
hub.exe install ProGet:5.2.3 --ConnectionString="Data Source=localhost; Integrated Security=True;"

Best,
Alana

Hi @jw ,

With the way things are "wired up" today, there are some edge cases when this will not show up right away. This is primarily for performance reasons, and it's something we absolutely plan to address in ProGet 2024.

The "trick" is that the PackageAnalyzer job needs to be run to do some back-end linking in the database; this is typically done on a nightly basis (there is a scheduled job for this), and in practice it's rarely something you'll spot outside of testing.

For example, after that job runs... if you were to delete then recreate TestProject 1.0.0 by pushing an SBOM, it should show the vulnerability.

Thanks,
Alana

Hi @rob-leadbeater_2457 ,

Good timing! This is something we plan to ship in the next or following maintenance release.

Actually, we had to completely reimplement Debian/APT feeds from scratch, so there will be a "Debian" and a "Debian (Legacy)" feed type, and a way to migrate from the legacy to the standard feed types.

I added a note to update this post once we have it ready to ship.

Cheers,
Alana

Hi @haralambop_0645 ,

A 403 error means that the user does not have the required permission to perform the action. So you'll need to make sure that feeduser is authorized to push packages to the feed, which you can do under Admin > Tasks.

Cheers,
Alana

Hi @gunmaden_7628 ,

The Common Packages API does not work with Maven feeds, since they are not a package-based feed. Otherwise... it's hard to say what the issue is with Python or Debian, but most likely it's missing a parameter that identifies which file to delete?

If you can share more details about the calls you are making and the packages we may be able to help.

Cheers,
Alana

@haralambop_0645 oh see - in this case, you'll need to add a source with the username api and password of your key using dotnet nuget add source

dotnet nuget add source https://myProGetServer.local/nuget/myFeed/v3/index.json -n myFeed -u api -p myApiKey --store-password-in-clear-text

This is because NuGet does not send an api key during list, it uses the name/password in the source file.

Hi @haralambop_0645 ,

Can you share more information about the usage scenario? Are you trying to use Docker images, for example?

Thanks,
Alana

You may have also seen this message but...

What Windows server version is Otter installed on? That might help us narrow this down.

We have seen that error if RSAT is not enabled:
https://learn.microsoft.com/en-us/troubleshoot/windows-server/system-management-components/remote-server-administration-tools

Cheers,
Alana

Thanks for the update @mness_8576; based on this, it sounds like there's definitely a Gateway problem, and that the Gateway is misconfigured / misreporting some error condition from ProGet due to a quirky package request. That's a pretty common thing we've seen as well.

Hi @itops_6398 ,

On a remote package (i.e. the ones with the radio icon), the download count is whatever is presented by the remote feed. On python packages, some have negative numbers.

Once you pull or cache (i.e. download) the package , the count will start at 0 and ProGet will keep track of the download count.

Cheers,
Alana

Hi @itops_6398 ,

It sounds like you're on the right track!

From here, you should create a group on your AD domain forest called "ProGet Administrators" (or whatever). Basically what you described in [2].

Then, on the "Tasks" tab, just add the appropriate permission (i.e. admin) for that group. You should see the "ProGet Administrators" appear in the dialog when you type a few characters.

After that, it should work as you'd like.

Behind the scenes, ProGet will attempt to query the user directories you've configured to discover the groups that the user belongs to. On the Testing tool (in the drop down on the security page), you can try to list a particular user's groups if it's not working.

Hope that helps!

Cheers,
Alana

@itops_6398 glad that worked! I noticed same quirk in the UI and fixed it :)

Probably why we/no one noticed before... editing the connector just happened to fix it once the value was saved.

Hi @itops_6398 ,

It doesn't work for me either, and when I looked closer I realized what the issue was

PyPi.org doesn't support searching, and the "New Feed Wizard" doesn't check of the "Use Exact Match" on the connector by default. So to work-around this issue, simply edit the connector, make sure box is checked (on the "Advanced" tab), then hit save.

After you do that, you can find numpy. But you won't be able to "search" packages unfortunately due to API limitations.

I forgot about that. We'll fix the Wizard via PG-2512 in the next maintenance release, but the work-around will do the trick.

Cheers,
Alana

Hi @avoisin_5738 ,

Thanks; so in this case, it sounds like there is something wrong with the zip file that our compression library does not like. We of course do not write our own zip library, but we can investigate. We have never had this problem in over 10 years... so it must be a something very strange. But we will look at it.

From here, can you create a test .npukg that we can use to reproduce? And also, can you send us the script/code that you use to create the custom package?

If it's a small file, then please email it to support at inedo dot com, with [QA-1267] in the subject (so that we can find it). If it's a huge file, then let's find a nother way to transmit it to us.

Thanks,
Alana

Hi @admin_9486 ,

We did not design the internal storage to be used/managed outside of ProGet... so it's not a scenario we really considered. Does re-indexing add the files that are missing?? That might be an unintentional function.

I guess, the first thing that comes to mind is using a drop-folder for new files.

Can you help us understand the use case? Why does it become updated externally, but also need to be updated in ProGet? It's just not one we imagined... so understanding will help give a appropriate solution

Thanks,
Alana

Hi @itops_6398 ,

That's definitely not right; searching for "numpy" should return results basically the same as this: https://pypi.org/search/?q=numpy

However, I just tried it, and I'm not seeing any packages come up. This used to work, so I think there must be some outage or issue with pypis' search api.

Let's give this a day and see if it resolves itself, then we can investigate further.

Thanks,
Alana

Hi @admin_9486 ,

That URL is for a Web Page within ProGet, and is not an API Endpoint; so what you're trying to do will not work and is not supported.

Also, re-indexing is not something you should ever need to do unless the file or database store has become corrupted some how. So even if it were possible to "script" such a thing, it wouldn't make sense.

Cheer

Thanks for the additional feedback; this is definitely something we're revisiting for ProGet 2024. As you described, a lot of the pieces are there - they just aren't wired together very well in some cases. We're working on redesigning this to be more cohesive.

A lot of the existing behavior has to do with performance. For example, /packages/from-purl?pUrl=... is a bit of a "hack" that we used. It's the navigation URL, then then redirects you to the proper URL. This is because finding the URL of a package can be expensive, especially when there are 1000's of packages in a SBOM and multiple candidate feeds for each package. Doing big joins across multiple feeds on the FeedPackages is something we need to very carefully do.

On the same performance note, "Pulling from the source" can be expensive -- especially when you aggregate multiple feeds into one using connectors. So we need to be really careful before doing that.

But like I said, this is something we are redesigning in ProGet 2024; you can think of ProGet SCA features as a "2.0" right now (I guess "1.0" was the old package consumers feature?), and hopefully "3.0" will be a lot closer to getting it right.

Cheers,
Alana

Hi @avoisin_5738 ,

How did you create this .nupkg file?

If you rename the .nupkg file to a .zip file, will it open a zip file in Windows or with a tool like 7zip?

If you can open it as a zip file on your workstation, then please test if you can import the file into ProGet using this method: https://docs.inedo.com/docs/proget-bulk-import-with-droppath

Thanks,
Alana

Hi @MaxCascone ,

No problem; I deleted the "alternate" accounts that were auto-created, and then I changed your forum email to the new one.

Cheers,
Alana

Hi @schuettecarsten_9427 ,

There's no limitation about that in ProGet... the * is something the NuGet client resolves.

Based on the error, it looks like there's a problem with Visual Studio communicating to ProGet. No further information is provided; maybe this is related to ProGet having an issue communicating to Nuget.org through your connector. You'd need to dig into that a bit further.

I'd also make sure to use the V3 API Endpoint. The query that's shown in the error is the v2/ODATA endpoint.

Cheers,
Alana

@philippe-camelio_3885 oh wow, great find.

Hmmm..... so I guess that must be some kind of UI bug with not trimming the newlines

variable.ListValues = variable.Type == VariableTemplateType.List ? txtListValues.Value?.Split(new[] { Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries) : null;

I'm sure this should be fixed eleswhere too, but for now I'll just add | StringSplitOptions.TrimEntries and I guess that will at least fix the UI bug

Hi @Jon,

There haven't been any major changes recently. What version did you upgrade from?

Here are the recent changes to Otter:
https://my.inedo.com/downloads/issues?Product=Otter

Assuming you were on v2022 before, I would suggest to rollback, and see if it helps.

Cheers,
Alana

Hi @ssuenaga_1020 ,

Thanks for the detailed report; this seems easy enough to fix, so we'll get this addressed in the next maintenance release (2023.20) via PG-2499.

Cheers,
Alana

Hi @e-rotteveel_1850,

Thanks for reporting this; we'll get this fixed via PG-2498 in ProGet 2023.20 (in two weeks).

Cheers,
Alana

Hi @philippe-camelio_3885 ,

We tried to reproduce this in the latest version of Otter, but are not having any luck :(

Here is my Job Template; note that you can get the JSON for job template under Admin > Raft Repositories > Browse.

{
  "Description": "",
  "JobConfiguration": {
    "ScriptId": "Default::Script::z-z-test/test-hello.otter",
    "ServerTargeting": 1,
    "ServerNames": [
      "LOCALHOST"
    ],
    "ServerRoleNames": [],
    "EnvironmentNames": [],
    "Variables": {},
    "Arguments": ""
  },
  "TemplateUsage": 1,
  "JobVariables": [
    {
      "Name": "params",
      "Description": "",
      "InitialValue": "%(env:item1)",
      "Type": "List",
      "Usage": "Input",
      "ListValues": [
        "%(env:item1)",
        "%(env:item2)",
        "%(env:item3)"
      ],
      "ListRestrict": true
    }
  ]
}

Here is my OtterScript. I wanted to verfify that env would be written, which it was.

foreach $i in @MapKeys(%params)
{
    Log-Debug Hello $i;
}

It's possible it was fixed by something else since the version you used, but I don't know.

Might be worth upgrading; if you still have the issue, can you send your JSON of the Job TEmplate?

Thanks,
Alana

Hi @mness_8576,

Unfortunately it's not going to be possible/practical to troubleshoot issues inside of ProGet when there are possibly problematic gateway appliances in the way. In general I can't imagine how there would be issues if there was only ProGet.

We know there's at least one HTTP gateway server between the build server and ProGet (which explains the 504 error), but there may also be a gateway between ProGet and nuget.org.

So I would first find out where these gateways are, and bypass them. If that's not possible, then you'll have to see what's happening on the gateway side. There are typically logs issued, etc.

We've seen everything from content filters to cache servers to DDoS protectors cause problems here. It's really difficult to guess.

Hope that helps,
Alana

@lucas-almeida_8120 it's hard to say; it's likely due to some kind of operating system configuration or issue that's preventing the service from restarting. We haven't seen this happen very often, and restarting the web is not a common activity.

It's probably not worth spending time investigating or trying to resolve, especially given the age of the server. You'd probably have to look at the Windows service logs to find what the issue might be.

Hi @mness_8576,

A 504 would be coming from some kind of intermittent network equipment (gateway), most typically a proxy server or SSL Offloading, etc. It's not issued by ProGet or IIS.

We've see this quite a bit - and usually the gateway has a timeout configured. Once reached, it terminates the connection, giving a 504 to the client and a "disconnect" to the server. The only way to fix this is to find that intermediate device and disable the timeout.

Best,
Alana

Hi @jw,

Thanks, good points! I'll try to explain the inner-workings a little more.

I would ask first is why ProGet does need the full package in its cache for the SCA tracking to work correctly.

This gets a little tricky and confusing; SCA checks for deprecation, vulnerabilities, and licenses.

• Vulnerabilities only need the Package Id + Version, so no package is needed
• License is generally stored in the package manifest file (e.g. nuspec file)
• Deprecation is server-side metadata, which means ProGet's value may be different than the connector's value

So the design thought was, if only vulnerabilities were detected it would be weird. Hence, missing package issue only. Maybe this assumption is incorrect.

Maybe a parameter to SBOM uploading/analyzing could be introduced that enables downloading the required metadata or packages (depending on what is really needed) before analysis is started

This unfortunately gets pretty complicated...

• ProGet will only store metadata for local/cached packages
• Package caching is an optional setting (not all feeds use it)
• There is no mechanism for automatically pull or cache packages

Designing a mechanism inside of ProGet to automatically download would not trivial at all for this use case. A PowerShell script to handle this on a one-off basis would probably be much easier.

FYI -- we are considering some kind of job that automatically download certain, trusted packages. This is in the context of "package approval workflows" and basically allowing something like AWSSDK (which frequently publishes new versions) to just always promote a new version. That's not exactly related to SCA/SBOM however.

Alana