Assets do not return Last-Modified header (anymore?)

pfeigl

We use the Asset feature to host ZIP files for use as "Jenkins Custom Tools".
Long story short, Jenkins uses the Last-Modified header on the original download of a file to identify whether to send the "If-Modified-Since" header for future requests to the same file.

We are pretty sure, that earlier versions of ProGet did actually return this header, as we have files on Jenkins slaves (back from april 2021) which have the correct modified header set.

Somewhere between back than and today (we update our instance pretty regular), this must have changed, as any request to the asset files in ProGet no longer return the Last-Modified header. This is a kind of a major problem for us, because Jenkins is downloading the same files over and over again.

Was this change by intention and is there any way to get this enabled again? The only change in your releases notes which might be remotely connected to this topic is this
https://inedo.myjetbrains.com/youtrack/issue/PG-2068

Thanks for your help!

atripp

Hi @pfeigl ,

The handler for asset file downloading hasn't changed recently.

The last major changes were in ProGet 6.0, where (among other things), the ability to control client-side caching was added. The change you found (PG-2068) fixed a bug related to UTC/Local time differences in those cache headers.

In ProGet 2022, we changed the overall platform (.NET Framework -> .NET6). The platform is what's responsible for reading/responding to cached/head requests.

I'm not sure what the behavior was prior to ProGet 2022... but if you're finding that caching isn't working as expected, I would inspect the cache control headers and see if you can find what the underlying issue is. So far as we can tell/test, it's working as it's supposed to now.

Cheers,
Alana

joerg.holle_8050

Hi,

seconding @pfeigl here:

we also moved a system of ours from .NET Framework -> .NET6 recently and experienced some changes in header-properties behaviour, in this case Content-Length not being filled anymore by the FileStreamResultExecutor of .net6.0 even though the Stream which is passed into the FileStreamResult has a length defined.
That field had been filled before with .Net Framework !
In the end we filled Content-Length in these special cases in our WebApi within our code to ensure identical behaviour for our clients.

I'm very convinced that ProGet's behaviour regarding Last-Modified was different before ProGet 2022 !

So, while I understand your statement "The platform is what's responsible for reading/responding to cached/head requests." I would still very much appreciate Inedo working around this behaviour in the code or at least documenting this as breaking change.

(The former I would prefer obviously ;-) because it would help our build process based on Jenkins as mentioned above.)

pfeigl

@atripp Thanks for your answer. I checked the caching options before posting here, but they don't really help in this case. Unfortunately Jenkins does not support the ETag header (and the other caching options). The whole implementation is based around the Last-Modified header.
See the implementation here for reference:
https://github.com/jenkinsci/jenkins/blob/5f7a4ee04e7a28bfb97805f2e5829ea0cd3f1c9c/core/src/main/java/hudson/FilePath.java#L980

The conditional requests RFC (7232) also states

An origin server SHOULD send Last-Modified for any selected
representation for which a last modification date can be reasonably
and consistently determined, ...

I understand that the platform is responsible for responding to cache request headers. The Last-Modified header however is not something that is a response to a certain given request header but needs to simply be returned. Only the If-Modified-Since header is one, that the server has to actively responde to (and ProGet rightfully does so already).

Anyways, I guess our question simply is: Is it reasonable for you to (re-)add this header in a future version? It feels like a simple change, as the asset UI already shows this exact field.

(If it helps anything, we are actually a paying customer, just chose to use the forum instead of a support incident).

Thanks,
Philipp

atripp

Thanks for the additional insight! To clarify... I know very little about how caching works, and I was just reporting what changed recently so we can track down where to look :)

So just to confirm... you're saying that this used to work in ProGet 6.0, but it's not working after you upgraded to ProGet 2022? If that's the case, then it would very likely be the platform change.

@pfeigl said in Assets do not return Last-Modified header (anymore?):

Anyways, I guess our question simply is: Is it reasonable for you to (re-)add this header in a future version? It feels like a simple change, as the asset UI already shows this exact field.

Yes, we just need to track down exactly what the issue is :)

Our platform code does seem to look for an If-Modified-Since header, and then sends a 304 if the dates are within a minute of each other. So I guess that works.

But then this code, when sending response headers, looks pretty suspicious to me.... I wonder if he should be setting the Last-Modified header instead of Date

pfeigl

This actually looks a little supsicious, but it's hard to tell without knowing what this.Cache actually is. Is this code kind of "general" to all requests in ProGet or is it specific to Assets? For assets I would have expected for the Last-Modified header to get it's value from the asset itself and not from any cache - but again, I don't know enough about your code base to really comment on this.

The spec on Date however is pretty clear (https://www.rfc-editor.org/rfc/rfc9110.html#section-6.6.1) that this header is not related to the actual content but more about when the response was generated. This information is very rarely of much interest tbh.

Last-Modified however is directly related to the date and time when the content of the requested document (file in this case) has changed.

rhessinger

Hi @pfeigl,

I did some investigation and it does look to be related to our platform upgrade. We will have a fix ready to be released in ProGet v2022.20.

Thanks,
Rich

pfeigl

This is great news, thanks for the super good feedback and fast responses! Looking forward to the next release

rhessinger

Hi @pfeigl,

Always happy to help! The new release is scheduled to be available on Friday. After upgrading to it, please inform us if the issue you were experiencing has been resolved or not.

Thanks,
Rich

pfeigl

Hi @rhessinger ,

we have finally found time to install the update and validate the fix you made.
We can confirm that everything is working as expected now.

Short remark which we realised while testing: HEAD requests still do not send the Last-Modified header, while GET requests do. This might be by intention, but I thought I'd share this.

Thanks for your help with this!

rhessinger

Hi @pfeigl,

Thanks for letting us know. It looks like this is related to how HEAD requests were handled. I went and added that header to head requests also. This will be tracked in PG-2300 and will be released in ProGet 2022.25.

Thanks,
Rich