Via HTTP Post, unable to promote ProGet package

jstarbird_7831

Not sure if this is a Buildmaster HTTP Post issue or a ProGet API issue.
I've tried several iterations of the HTTP Post to try and promote a package on ProGet.
I've followed everything I can find on both subjects but have been unable to get it to work.

What I'm not sure about is how to enter JSON in the BuildMaster HTTP post function. I've tried TextData and FormData.

The issue I'm having is that I keep getting a 401 unauthorized however I am putting in an API Key that is configured for Promotion API on ProGet.

Here is what I have been trying:

    Post-Http http://progetserver/api/promotions/promote
    (
        ContentType: application/json,
        TextData: key: "API Key entered here", 
            packageName: "Package Name",
            user: "Admin username",
            version: "54.0.0",
            fromFeed: "src feed",
            toFeed: "dest feed",
            comments: "This package promoted by BuildMaster",
        LogRequestData: true,
        LogResponseBody: true
    );

I've also tried it with FormData but that didn't work, which I would expect in this case.

Product: BuildMaster
Version: 5.5.3

One cause for a 401 is Windows Integrated Authentication; if you have this enabled on ProGet, then you'll want to make sure the BuildMaster server is running under a domain account that can access.

To use FormData, you'll want to use a Map expression.

Post-Http http://progetserver/api/promotions/promote
(
    ContentType: application/json,
    FormData: %(key: "API Key entered here", 
        packageName: "Package Name",
        user: "Admin username",
        version: "54.0.0",
        fromFeed: "src feed",
        toFeed: "dest feed",
        comments: "This package promoted by BuildMaster"),
    LogRequestData: true,
    LogResponseBody: true
);

Alana,
I had tried that. However, just realized I had been looking at BuildMasters user manager instead of ProGet. The user I am using didn't have the correct permission.

So I gave that user Promote Packages permission but it still didn't work. This login is the user that BuildMaster runs as.
I even tried giving that account Admin rights on the ProGet server, still didn't work.

I would look at the response body error message; if it has HTML in it, then the problem is integrated security. But if you can install NuGet packages (or otherwise access the server from BuildMaster), then that's not the problem. When WIA is enabled, IIS will block all requests without authentication before it even gets to ProGet.

That said, the API does not pay attention to the user; only the API key is used. That API key must be configured to allow for this sort of access under Admin > API KEys.

Alana,
I can publish to ProGet from Buildmaster all the time so it's only doing this API for promoting that isn't working.
The response body is HTML.

The API key I had setup for just Promotion API access but I just changed to allow unrestricted access, still didn't work.

Is there any information of note in Admin > Security & Authentication > API Keys & Access Logs > Access Logs tab in the ProGet instance? The request and response bodies of any API request should be logged there.

Nothing is being logged there unless i just go to the web browser and try the /api/promotions/promote url directly. Then an error gets logged about needing the API key.
When I run it from the Buildmaster post function, nothing is logged. It's like something is preventing it from connecting to that server yet there is nothing setup to do so.

No firewalls on either system. Buildmaster can push/pull from ProGet feeds just fine. It's just this method that isn't working for some reason.

some more info, I tried just using a Post with all the params in the URL as per your doc page for urlencoded, that didn't work either.

So I went ahead and did it directly in the web browser and now get a better error. It's saying now that the API does not permit access to Package Promotion API yet this is the URL I used:

http://progetserver/api/promotions/promote?key=realapikey&packageName=Sci.Framework.Service.Sm.Core&version=41.0.0&fromFeed=Development&toFeed=Released&user=username&comments=This+package+promoted+to+Release+by+BuildMaster

Meant to add to the above, in the access log it looks like the API key is the problem. It has a Plus sign in the middle so it's seen as a space in it.
I tried replacing the plus sign in the api key with %2b but then I get a Method Not Supported error.

Yes, any query string arguments need to be URL encoded. The method not supported error is expected because that endpoint only allows HTTP POST. You'll have to use something like Fiddler or Postman to do a POST. As a tip, when I do test POSTs in Fiddler, I always capture a GET request to the URL then drag the captured request to the Composer tab, where it can be converted to a POST... a lot easier than creating the request from scratch :)

I also notice Alana's example has ContentType: application/json, when it should be application/x-www-form-urlencoded if you're sending key-value pairs with FormData.

If you wanted to send JSON, you'd have to send that as TextData and build the JSON object as a string e.g.

TextData: >> {
  fromFeed: "fromFeedName",
  toFeed: "toFeedName",
  ... etc
}>>

Tod,
I was trying the URL directly based on the ProGet promotion API doc that showed that so if that isn't supported you may want to update the doc. Also a bit more detailed example there might be helpful. Of course the ultimate solution would be to just add a Promote Build function to the ProGet extension in BuildMaster! I've already submitted a feature request for that one.

Anyway, I tried what you said and still hit the same error. I'll try it in Fiddler when I can but I could only try it from BuildMaster at the moment. Here's what I put in:

        Post-Http http://progetserver/api/promotions/promote
        (
            ContentType: application/json,
            TextData: >> { 
                key: "apikeyvalue",
                packageName: "Sci.Framework.Service.Sm.Core",
                version: "41.0.0",
                fromFeed: "Development",
                toFeed: "Released",
                user: "adminuseraccount",
                comments: "This package promoted to Release by BuildMaster"
            }>>,
            LogRequestData: true,
            LogResponseBody: true
        );

Thanks.

We will add that operation at some point. The problem is that the API doesn't send 401s (all forbidden API keys result in a 403). The 401s are coming from IIS or the integrated web server before the request goes through PG - so my only guess is that the BuildMaster service account must not have access to the ProGet server. Note the HTTP operations currently run from the BuildMaster server only, so executing in a for server block will still be sent from the BuildMaster service (this will likely change in a future InedoCore extension as it's a bit counter-intuitive).

Another thing I noticed is that the JSON representation doesn't support the key argument. You can either supply "API_Key": "yourApiKey" in the JSON, or add a new operation argument RequestHeaders: %(X-ApiKey: "yourApiKey") that doesn't need to be escaped.

Unfortunately with Windows Integrated Auth, it cannot be disabled for some URLs... it's all or nothing (though we're working on a resolution to this currently, especially for npm which doesn't support WIA at all).

Tod,
the buildmaster server has full access to our ProGet server, the user each service runs as is the same domain account and it is full system admin on both as well.

I did try moving the API key to the Request Headers as you suggested but still get the exact same error.

Both servers are using the IIS setup done by BuildMaster and ProGet installers, are there any new settings that need to be made for it that may not have occurred in an upgrade?

Thanks.

The 401s are occurring before the request reaches the API (it's blocked by the web server), so it doesn't really matter what conent you are sending.

Unfortunately, sometimes just setting the service account to a domain account isn't enough to get integrated authentication working; I suggest to logon to the server with that service account, and then try to access the web UI. You should not be prompted for a password from the browser.

Windows Integrated Auth can be very complex to get working between service accounts; are these running on the same machine? Getting a machine to "talk to itself" is a challenge, and you may need to use setspn or another domain admin tool to do so.

Well to add to all this, I can use the Jenkins plugin using the exact same account without issue. It's using the Native API I believe, at least that's which API Key it has but it does work.
For some reason this seems to only be an issue from BuildMaster.

I'm not familiar with this SetSPN but after reading it I'm not to sure what we're trying to do with it. The account being used is what both Windows services are running as on their respective machines. I have logged into both with that account as well.

You mentioned using the UI on the Proget server, which UI did you mean? Progets or BuildMaster? I did try both and did not have any issue so not sure what it was you wanted me to try.

The other thing is I can, from the BuildMaster server, enter the URL - http://progetserver/api/json/Connectors_GetConnectors?API_Key=nativeAPIKey
and that works. But from BuildMaster it does not.

I can't say why Jenkins works, the problem is the client (BuildMaster service), server (ProGet website), or both. Since the application happens below the application layer, Diagnosing it is not trivial unfortunately. When WIA is enabled, all our server software "sees" is an integrated request.

I would also try to restart the BuildMaster machine as well, maybe that will do something the domain and service principal names.

If all else fails, you'll will need to monitor network traffic; you can do this by:

1. installing Fiddler on BuildMaster machine
2. running it
3. editing proxy settings on buildmaster to use a proxy with windows settings
4. stop/restart the service

you should see requests being pushed through fiddler, then on to their destination. This will give you more insight into where the request is not being handled...

Before I do that, does it matter what the App Pool runs as for each service? it will be next week before i have a chance to work with Fiddler.

I finally had a chance to run Fiddler and it's not showing any attempt to talk to the ProGet server via HTTP post.
I see other requests occurring by BuildMaster, updates, Jira, but not the HTTP Post to promote.
So for some reason BuildMaster isn't even making an attempt.

So after searching online I did find that if I Enable Anonymous Authentication on the ProGet server that this then works.
However, big problem here is that this then Breaks Windows Authentication whenever you browse to the ProGet server.

I would hope you have a way to configure these since they are both your products, what would I change in IIS to handle both situations?

Discussion should be moved to a new post or followed here:

http://inedo.com/support/questions/6554