Posts made by atripp

Oh my bad... yes it seems a bug. I looked at GitHub in the UI, anwyays I think it's fixed.

Can you try the new pre-release version?

You can manually install, or point your extensions feed (Advanced SEttigns) to to https://proget.inedo.com/upack/PrereleaseExtensions/

https://proget.inedo.com/packages/overview?feedId=1&packageGroup=inedox&packageName=GitLab&packageVersion=1.7.6-RC.2

There are two fields on a GitLab Branches List Variable Source

• From GitHub resource ("ResourceName") should be the name of a Secure Resource, which defines your api url, project name, etc.

• ProjectName is optional, and may override the project name

Based on the message, I think, the resource name isn't filled out?

This is the sort of error you can expect when the application code and database are not the same version.

Had you installed the the same version you had on the old server, it wouldn't be a problem. But fortunately it's an easy fix I think.

Just follow the Database Manual Installation Instructions and run inedosql against your database. Then it'll get updated to the right version.

FYI please review the Backing Up & Restoring as well :)

Hi Markus,

The package metadata records are in the database, but the files are on disk.

The error is thrown when the operating system reports file-not-found when loading a file.

Why is the operating system reporting that, it's hard to say. With Docker, there are two things to consider:

• The package store (typically Admin > Advanced Settings > Root Package Store) refers to files within the container
• The docker run -v command maps a directory outside of the container to one within the container

Perhaps the directory outside of the contaienr doesn't have the file, or is a typo or something? Please let us know what you find...

Hope that helps.
Alana

Hello; at first, what version of ProGet is this?

Whoops, I accidentally forked the topic, so I'm just going to quote it in here instead

@smolkov-anton_1636 said in Unlisted NuGet Packages showing as listed in feed:

@atripp Hello.
I have got the same behaviour too.
With fiddler i see that all versions i unlisted before have property
<d:Listed m:type="Edm.Boolean">false</d:Listed>
, but still apeear in VisualStudio, nuget list and even Rider.
Version - 5.3.0 (Build 126-5.3.0).

I think this one is related:
https://forums.inedo.com/topic/2675/owner-has-unlisted-the-package-on-nuget-org-but-proget-still-shows-it-in-the-list-of-availible-versions/4

@smolkov-anton_1636 said in Unlisted NuGet Packages showing as listed in feed:

@atripp Hello.
I have got the same behaviour too.
With fiddler i see that all versions i unlisted before have property
<d:Listed m:type="Edm.Boolean">false</d:Listed>
, but still apeear in VisualStudio, nuget list and even Rider.
Version - 5.3.0 (Build 126-5.3.0).

I think this one is related:
https://forums.inedo.com/topic/2675/owner-has-unlisted-the-package-on-nuget-org-but-proget-still-shows-it-in-the-list-of-availible-versions/4

@smolkov-anton_1636 said in Unlisted NuGet Packages showing as listed in feed:

The same is for new nuget V3 API.
Steps to reproduce:

1. Unlist all versions of package in proget UI
2. List only one version / push new version

Expected result:
Visual Studio UI shows only one version which was listed

Observed result:
Visual Studio UI shows all the versions

UPDATE in 2023 - see Inedo Products and Prometheus support to learn more about how to integrate our products together

Thanks for the suggestion! It's the first time I heard about Prometheus; do you think this would be easy to do?

We do integrate with a took called Loupe by Gibraltar Software, which does provide great, details, and information about performance.

Hi Simon;

The Linux-based error is an error-logging-an-error sort of problem, so we can just get rid of the underlying error and be OK.

The underlying error in this case is a bug, and it's that Impersonate User is required (even though it shouldn't be). Just put in a user that has ViewPackage privilege to the target feed.

Actually you don't even need an API Key for this endpoint, just view package is enough.

Best,

Alana

Not in particular; .NET does garbage collection, in the background, as needed; so it's not uncommon to see processes "swell" to consume available memory and then "contract" overtime, as memory usage (percentage-wise) gets high.

I'm not aware of how to do it; ProGet does not provide monitoring at this network level. Do you have an ngix proxy in front of it? We've heard that's very important, and it can help deal with some other mono bugs around not closing keep-alive connections quickly. And and also lets you have SSL as well.

This sounds like it's network-traffic related, and likely the server is getting overwhelmed...

See How to Prevent Server Overload in ProGet to learn more.

HI Simon;

The error appears to be occurring while logging the API body; can you try disabling that as see if it helps?

Admin > Api Keys > Edit > Uncheck Log request body , Uncheck Log response body

Is this ProGet for Linux, by chance? There was a related error in an earlier version of ProGet regarding this.

Hello;

I think that error is coming from choco client.

ProGet will respond with a 403: Invalid API Key error response when an invalid API is sent, then terminate the request. A 401 will cause the NuGet client to prompt for credentials.

ni Nik,

There could certainly be a timeout... but if that were the case, it wouldn't be causing an "authentication" error.

You can check errors in ProGet, Admin > Diagnostic Center, perhaps something is logged? If not, then ProGet isn't making the error.

We have seen intermediate servers (proxy, traffic shaping, etc), terminate long-running connections and return a strange/inappropriate error (i.e. something other than Gateway Timeout) to the client. It's really hard to guess. You can install an intermediate tool, like Fiddler or Wireshark, to capture traffic from your workstation. Maybe the raw traffic will give you a better clue.

Hello;

That's a good question; we can't reproduce this, if the file is instantaneous through the browser, we can be totally assured that the problem isn't ProGet finding and sending that .deb file over the web. apt install isn't any more complicated than that.

Can you put a sort of Proxy Server or traffic monitor between the two? Maybe it's network related? Maybe at a TCP/SSL packet level? Maybe apt has a sort of throttle? I guess you could also try to curl the file from whatever server you're doing apt install on to further track it down?

hope thsi helps track it down, please share what you find in case anyone else experiences it.

Alana

HI Marc

These errors are unrelated to license restrictions or your license key.

Check out the errors on Admin > Diagnostic Center, they should be logged.

There must have been some configuration change on your server or network, perhaps a proxy server or new firewall restriction, that is preventing your ProGet server from retrieving the packages.

Alana

Hello; it's the first request for this, but we're going to investigate adding this to ProGet as PG-1702. We'll update as we make progress on investigation.

Thanks! Cool "badge site" (shields.io) that you use for the status image like this

NIce ideas, but a couple concerns; currently we use ApplicationName as the querystring parameter, which then searches for the current active build or that application.

To be able to use $CommitId, it would make it more complicated. It would mean we would need to find a build with a variable named $CommitId and with that value. And at this point, then, we could just allow any number of build variables to be used, which might be nice. $Branch for example.

But a concern is performance. That could be a lot of variables to search, so having an ApplicationName filter would be important (build variables are indexed by application Id); it could still be hundreds of thousands of records rows search even in one application.

We could limit it to active builds, which would reduce searching by many orders of magnitudes and be not a problem.

Hello;

Actually, I think one of those public repositories you are connecting to is a ProGet instance, but it's (incorrectly) configured to warn when ProGet Free connects to it.

We're in touch with the vendor now and are providing them with the proper key to allow it.

Thanks.
Alana

Hello;

When you edit a connector on ProGet Free, that messaging ("ProGet Free is only permitted to have a connector to public repositories...") is informational. It's always displayed when creating or editing a connector, if your license key is free.

If you create a connector to another ProGet Free instance... or if another ProGet Free connects to your instance... then ProGet will detect this, and record a license violation. The License Violations Documentation should give some more information about how this works, but in any case, those violations are "informational only" and do prevent functionality of the software. It just displays the banner to make you aware of the restriction and violation.

What information is being shown under Admin > License Key > License Violations? That should record IP address and descriptions.

How are you changing the license key? When you change the license key (doesn't matter to what... even an invalid string), it will delete all recorded license violations.

Thanks,
Alana

Hmm, for this, then it seems -Repository is the argument?

This sounds like a nice idea, and I think something we could implement easily. Of course, figuring out the new specifications / documentation is always the hard part I think :)

How might you suggest changing the CI Status Badges we have now?

I'm envisioning adding in a querystring parameter for Stage or something? If you could suggest us how to change the docs / API, that would go a long way in helping us get this implemented :D

It looks like you're not specifying the -Source argument? I'm not totally sure, but here's the docs on Publish-Module might help diagnose.

Hi Nik;

There's nothing in ProGet that would yield this behavior, and it's not behavior that others experience, so I think it must be something between ProGet and your workstation. Maybe proxy server, request filtering, etc.?

Just to check, you are using chunked uploads with Docker, right? It's been the default for years, but i guess it can still be disabled. The same intermediate server may be stripping out ProGet's support of chunks? I don't know -- https://github.com/docker/distribution/blob/master/docs/spec/api.md#chunked-upload

We've also had reports of strange errors happening on random blobs from one or two customers. They discovered it was the Request Filtering feature enabled in IIS. The error was different, and it had to do with blocking query strings with :’s in them. But maybe it's this?

hope this helps

Otter uses Inedo.SDK 1.6, where as BuildMaster is on Inedo.SDK 1.7. The major addition to 1.7 is SecureResources and SecureCredentials, which replace ResourceCredentials. However, it's still backwards compatible.

The current (1.7) version of GitLab::Get-Source has two changes:

• Changed Credentials property to From (but keeps Credentials as an alias)
• Added From as the default property, so it will be used as argument right after the operation name

This keeps it backwards compatible.

We will add SDK 1.7 to Otter later this year, but in the mean time you can use the Otter version code and create a ResourceCredential in BuildMaster of the same name.

For this case, you're right -- the Instance ID would be the way to go. On the new BuildMaster server, you can set the Agent.InstanceId field (any string is fine, "bm62" would work) , so this way, both BuildMaster instances won't collide.

We are using our "Customer Success Program" to provide upgrade assistance for BuildMaster 6.2, so the best way to get started on this would be to talk to your account representative.

You can also submit a ticket and put 6.2 UPGRADE in the "How can we help" section, and we'll route that to the right people :)

Indexing the symbol server wouldn't make a difference. I would try clearing your connector cache.

It's very unlikely that this is a regression in 5.3; this is a very common use case, nothing strange about the package, and I can't reproduce it. So it's likely configuration or network related.

To rollback, please follow the restore instructions: https://docs.inedo.com/docs/proget/installation/backing-up

I think this is related to the foreign layer. This is a known issue, it's actually not an "error", but "by design" of Docker.

https://medium.com/@amlys/solution-docker-push-fails-with-skipping-foreign-layer-a119e03bdf0d

I think this will fix it.

Hello;

That "error" message is to be expected; that's what ProGet returns when you visit to /v2 with a web browser. But the error sounds like a sort of SSL/certificate configuration problem.

After searching for "tls: no renegotiation", here is an article I found about that might help: https://www.digicert.com/news/2011-06-03-ssl-renego/

Hi Nik, ProGet does indeed support pushing Windows containers.

What sort of errors are you receiving?

Oh I see; in this case, please remove your credentials from the connector. They're not required for NuGet.org; ProGet does not push to connectors, only download files.

It's a common mistake and we will try to make it clear in the UI.

First, upgrade to BuildMaster 6.1 (see KB#1163 Upgrading to BuildMaster 6.1; this is a relatively low risk, as BuildMaster 6.1 still has all of the legacy features included. There are several "bridge features" to help get you to 6.2

Then, consider a upgrade or migration strategy to BuildMaster 6.2 (see KB#1766 Upgrading to BuildMaster 6.2). Depending on your legacy feature usage, you may want to do a direct upgrade, or you may want to migrate.

And lastly, please keep in mind this note from the docs on BuildMaster v6.2 Migration:

Because this upgrade is more involved than previous upgrades, special support for it has been included with paid licenses. If you are interested in learning more, use the Submit Ticket Form and put 6.2 UPGRADE in the "How can we help" section to let us know when you are going to upgrade. We also recommend you include the Legacy Features Dashboard Report Logs in the body of the ticket.

Hello; I'm not able to reproduce this.

I can successfully download Microsoft.EntityFrameworkCore 3.1.3 without an issue from ProGet 5.3.0-m5.

Can you navigate to the package in your instance, and hit the Download button?

Maybe it was a temporary NuGet outage?

Hello; i don't know how easy it is to implement, but I put I added this for an internal assessment; https://inedo.myjetbrains.com/youtrack/issue/BM-3595

We'll try to evaluate in the coming weeks. Stay tuned!

@philippe-camelio_3885 I responded to this before realizing it was deleted; so I restored the post and moved it to my username. hope that's ok :)

The SHExec operation does not consider the error code for success; it appears to be relatively easy to change the code to do so (SHUtil::ExecuteScriptAsync returns an error code), but I think (I could be wrong) exit codes from shell scripts aren't so consistent?

In any case, it appears if a script writes to the error log (stream=2) then it will fail.

Thanks for the update! I've noted this in the docs, and linked to this discussion :)

https://github.com/Inedo/inedo-docs/commit/d24087911584bbda833314084a58c2ae1ff41c39

Hello
Howto raise an error in Buildmaster for a failed SHEXEC ?

I have an ansible playbook which failed (RC=2) but the Pipeline step is OK while it should be in error

I can't figure out how to manage this.

Here is the log:

...
INFO : 2020-05-03 13:46:43Z - fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Source template/site.conf.j2 not found"}
INFO : 2020-05-03 13:46:43Z - 	to retry, use: --limit @/tmp/bm/extranet-monespace/ansible/main.retry
INFO : 2020-05-03 13:46:43Z - PLAY RECAP *********************************************************************
INFO : 2020-05-03 13:46:43Z - 127.0.0.1                  : ok=4    changed=2    unreachable=0    failed=1   
INFO : 2020-05-03 13:46:43Z - Exit code is: 2
DEBUG: 2020-05-03 13:46:43Z - Script completed.
DEBUG: 2020-05-03 13:46:43Z - Deleting temporary script file (/tmp/otter/scripts/d2f7283d77c94cfb89151ca2e3004212)...
DEBUG: 2020-05-03 13:46:43Z - Temporary file deleted.
DEBUG: 2020-05-03 13:46:43Z - Cleaning up...
DEBUG: 2020-05-03 13:46:43Z - Deleting /tmp/otter/_E174799 on VM111021...
DEBUG: 2020-05-03 13:46:43Z - /tmp/otter/_E174799 on VM111021 deleted.
DEBUG: 2020-05-03 13:46:43Z - Cleanup complete.

Any ideas ?

What do you think about it? We use GitLab for our internal projects, but would like to understand the choices between GOGS/Gitea and the "big players" like GitHub/GitLab.

I know we are very interested in helping organizations use as many different source control platforms as possible, and GOGS seems pretty interesting! So keep us in the loop, as far as how your integration goes with GOGS, it might be worth us building a specific integration. I've also heard of Gitlea, but really not sure about their popularity in organizations, or if it's worth investing/investigating how to build better/first-class integrations to our products

Aww, that's too bad. Unfortunately Repository Webhooks aren't exposed in the SDK at this point, so it's not nearly as easy to try building/testing it on your own. We'll definitely consider adding this in a future SDK release however.

Thanks @kvasudevan_8753 - that definitely will do it :)

@nik-reiman_6009 package interaction is done through the package-specific, third-party apis; we only provide minimal documentation for these APIs, as they are generally either already documented elsewhere. We simply implement those API.

So in this case, if you search for how to delete a nuget package from the api, the first result is from Microsoft which details how to Push and Delete packages from a NuGet API

What I mean by "republish" it is to republish it to your ProGet feed. This local package will then "mask" the connector package, and ought to behave fine.

In this case, since it's just one package behaving like this, and there's such an easy work around (download package, edit metadata, republish), it's really not worth the investment to even investigate how easy/difficult it would be to fix. If it becomes more widespread, we'll investigate.

This has to be done on NuGet and PowerShell feeds for some old and quirky packages (like ones that have a 1.01.1 and 1.1.1 version). It may be trivial, but it could introduce the exact regression you described (people using build metadata in their package.json), or other (unknown) regressions, and it's just not worth the risk to our users.

The Webhooks are GitLab and GitHub specific, but if Gogs mocks/uses the GitHub pattern it should work.

Thanks, that makes sense where it comes from.

Since this is the only package with a problem, I wonder if you could just "republish" it with the build-metadata number missing?

We're a bit hesitant to make change that could impact this. For all we know, this works in older versions of the npm client, or newer versions of the npm client, or the yarn client, etc., and users are relying on this behavior.

Hello;

Sounds like there's a lot to unpack...

First, we've got some major changes in ProGet 5.3 that should help out quite a bit, but I've included a lot of resources in this post to consider as well. Consider that, as a private registry, ProGet is designed to host "private images" that you manage (which can be build from "public images"), and it's really not designed as a "registry mirror/cache". These are two different usecases.

The authentication issues, that might be different problem and might be worth a new thread?

Repository Mirroring in ProGet

From But here is our current thoughts/documentation on repository mirroring in Proget:

There is the ability to "mirror" hub.docker.org, but this feature was really only designed to enable Docker, Inc. to host public registries in sensitive regions like China (so you can mirror to registry.docker-cn.com instead).
Unfortunately, the Docker client will only use the configured "mirror" for building and pulling; images are always pushed to hub.docker.org.
Docker, Inc. does not seem to be interested in enabling this, and we aren't particularly keen on forking the Docker client to enable this. Thus, it seems there's no sense in ProGet supporting this anytime soon

Base Image Pattern

On Dockerhub, there are a lot of problematic images that cause a new breed of "worked on my Docker" problems in companies. openjdk:11 is a great example of this problem. The connector "problems" in ProGet are a symptom of the problem, but we strongly recommend you to not use "third-party" container image directly, but instead follow our "base image" development pattern.

First, and foremost, openjdk:11 currently references what's called a "fat" manifest, which is a sort of "pointer" to other manifests that will be used depending on the run-time environment of Docker. There are currently four different environment-specific images that this "fat" manifest points to, so when you build an image on top of it (e.g. FROM openjdk:11), your image will be "locked" to whatever environment configuration the build machine (or even workstation) had at the time the command was run. This may be different than the machine you run the image on.

Even more problematic, however, is that 11 is a time-sensitive tag, and what 11 refers to keeps changing. Today, it points to four different versions of four different images (as of 7 days ago), but tomorrow... it could point to 3 totally different images. Who knows? It's a repository maintained by a third party, and 11 means what they say it means, at their discretion.

The Dockerhub is meant to be the "authority" on this repository, and this is where connectors are a challenge. ProGet caches certain informations about these images, but when the meaning of these things change, it defeats the purpose of caching. If you disable caching, it should work... but then you've already , but what's the point then? You're downloading huge image files constantly then.

So instead, here's how we recommend you develop a practice around third-party base images.

docker pull openjdk:11
docker tag openjdk:11 proget.mycompany.com/dev-images/openjdk-linux64:11.0.6-rc.1
docker push proget.mycompany.com/devimages/mycomp/openjdk-linux64:11.0.6-rc.1

Even better would be to use a Dockerfile (e.g. FROM openjdk:11) that lets you customize the image as needed, but this a good start. Then, after you/developers are happy with that version of that image, then you can "re-tag" from 11.0.6-rc.1 to 11.0.6. In ProGet 5.3, these operations are made easier, and "virtual" tags are automatically generated. But the important thing is that you treat "tags" as immutable, and provide consistent images for your users.

Note that, the repository name is mycomp/openjdk-linux64. It's important to not use only openjdk-linux64, since that name is identical to _/openjdk-linux64 and library/openjdk-linux64. The library/empty/_ namespace are intended only for "Dockerhub-certified" images

Docker is supposed to solve "works on my machine", but unless you treat tags as immutable, you'll keep getting "works on my Docker when I ran it," and that's even harder to debug. ProGet is "exposing" the underlying problems with these machine-specific, time-specific tags, but I guess it's better to discover it early on, then in Production.

Hello, you can definitely delete packages form the API!

But please note, from the API Docs page:

API for Third-party Packages and Feed Types

In order to support third-party package formats types like NuGet, npm, etc., ProGet implements a variety of third-party APIs. We only provide minimal documentation for these APIs, as they are generally either already documented elsewhere. However, you can generally find the basics by searching for specific things you'd like to do with the API, such as "how to search for packages using the NuGet API" or "how to publish an npm package using the API".

hi Ryan,

Can you try Active Directory (New)? The Active Directory with Multiple Domains will be deprecated in 5.3, and the code behind the scenes is totally different anyways. So, it might behave differently.

Regardless, based on what you wrote, it seems to be a problem of NETBIOS mapping.

Basically, the LOGON_USER string contains DOMAIN\username, and ProGet needs map DOMAIN to the actual domain. Sometimes the automatic querying doesn't work (permissions, not configured on domain, etc), so you can specify it in an advance property in the Active Directory (New) provider.

So please try this if it doesn't work once switching to Active Directory (New).

Hmm, it's strange. Is this the only package and version that's acting like this? The "+" implies a sort of build-metadata in SemVer, but i'm not sure if npm supports this?

Anyways the npmjs.org version doesn't seem to have the text 4.12.20 anywhere, at all; any idea where that's coming from?

OK; at this point, I think it's going to best to schedule some one-one-one review time with our engineering team.

I will start the process on this end, but can you please open a ticket at here - https://my.inedo.com/tickets/new

You can write "6.2 UPGRADE" in the "How can we help", and include any other emails you'd like us to be in touch with in the form as well. Then we will work and get stuff scheduled!