Skip to content

CDN: File Headers

File headers are the essence of CDN workload and behavior. Whenever a file will be (or not) cached, some details are essential in order to decide what to do.

For more detailed resource, check out Mozilla Caching docs - please note that this is a platform agnostic documentation. It will explain how network, resources fetching, browser headers and content validation is done (and also a lot more).

Most Relevant Headers

  • Cache-Control: MDN;
  • Pragma: MDN for backward compatibility. It will take care of public information for cache, if resource is safe to cache or not (e.g. sensitive information), how long it will be cached, if revalidation is required or not, etc.
  • Cross-origin Resource Sharing (CORS): MDN
  • Access-Control-Allow-Origin: MDN
  • Access-Control-Allow-Methods: MDN
  • Access-Control-Allow-Headers: MDN
  • Access-Control-Allow-Credentials: MDN

Where Headers Should Come From?

Content Served from Origin

Server should provide all relevant headers from the web-server side (nginx, Apache). Sometimes we can use application to determine dynamic-content headers as well as other information, like credential-related headers.

Content from Storage

In the current setup, AWS uses S3 to handle CORS headers. Then, we should handle Cache related headers on Lambda@Edge level

Secret Tip

Most of the work on CDN relies on headers and how they are passed between origin -> CDN -> Client/Viewer.

CDNs usually limit the headers that are used to fetch a resource as a way to improve performance (by default).

If you master what headers are needed and where they can be added/removed, it will not be that hard to work with CDNs.

You should configure your CDN to handle query strings, filter unnecessary headers, add missing ones or even override headers (check Lambda@Edge)

Testing Headers

Origin

sh
curl -I 'https://www.interaction-design.org/favicon.ico'

-- output --

HTTP/2 200
server: nginx/1.17.3
date: Mon, 11 May 2020 22:20:07 GMT
content-type: image/x-icon
content-length: 1150
last-modified: Fri, 08 May 2020 19:56:32 GMT
etag: "5eb5b970-47e"
expires: Wed, 10 Jun 2020 22:20:07 GMT
cache-control: max-age=2592000
cache-control: public
accept-ranges: bytes

CDN

For response headers (what CDN sent to you)

sh
curl -I https://public-media.interaction-design.org/images/app/safari-pinned-tab.svg

-- output --

HTTP/2 200
content-type: image/svg+xml
content-length: 4929
date: Mon, 11 May 2020 22:22:22 GMT
last-modified: Fri, 20 Sep 2019 02:01:33 GMT
etag: "b06527ba67d44305a068b678d013091f"
cache-control: public, max-age=15552000
x-amz-version-id: No2_li_fk1k61K8cxk3ZrQ7HpJh1Nisq
accept-ranges: bytes
server: AmazonS3
vary: Accept-Encoding
x-cache: Miss from cloudfront
via: 1.1 90afcdb26518f969b68e124515efb74c.cloudfront.net (CloudFront)
x-amz-cf-pop: YUL62-C1
x-amz-cf-id: hFm_EMlPY46KOFX4Q0Y9wTx7HtfLeGBFpFwdAlM00bH_J1SaE8bVuQ==

To follow redirects (output in a chain)

sh
# note we use the option with an extra argument "L".
curl -IL https://staging.ixdf-dev/tv/course_videos/v2/practical-accessibility-for-web-and-1-2-10-principles-of-accessibility-for.mp4

-- output --

HTTP/2 307
server: nginx/1.17.3
content-type: text/html; charset=UTF-8
location: https://dqt1nhr0dcwgx.cloudfront.net/tv/course_videos/v2/practical-accessibility-for-web-and-1-2-10-principles-of-accessibility-for.mp4?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9kcXQxbmhyMGRjd2d4LmNsb3VkZnJvbnQubmV0L3R2L2NvdXJzZV92aWRlb3MvdjIvcHJhY3RpY2FsLWFjY2Vzc2liaWxpdHktZm9yLXdlYi1hbmQtMS0yLTEwLXByaW5jaXBsZXMtb2YtYWNjZXNzaWJpbGl0eS1mb3IubXA0IiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNTg5MzIyMjg3fX19XX0_&Signature=DR~HUt-5fBfExwwRPsBAqdKoEHBnsGKtTcgDR-rP0JGmyw314l1OPMyORtQFryc1vkgFfALJSB9MRIJmJ9Jst~jif7t65Im8PvKtEW1NNJyCj7QGKcEzKZqMlRVTnh0OQSxYRrZoq3vhFqfTUETuJBcDwg3uzSOBlqg-VC9Kgc93ZPPBjQWLMzcXaKzy8HhsnU5zPcUyoI7HWwTQv0cxrleRUf5KksQDPNUHZz~r47YLYuufXHBAf~rjnJDIVppg5ER6CI25cXcB97PNYLu2XkBkVvcf3qOcYKxvEfL2zBHmdmZlb0Plbcqx2aaldUaN4XWRR6obaseNR~Rc5O5IuA__&Key-Pair-Id=APKAJZGMDJHQOVEWTKAA
cache-control: no-cache, private
date: Mon, 11 May 2020 22:24:47 GMT
x-robots-tag: none
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
strict-transport-security: max-age=31536000; includeSubDomains

HTTP/2 200
content-type: video/mp4
content-length: 174804645
date: Mon, 11 May 2020 22:24:48 GMT
last-modified: Tue, 04 Jun 2019 13:47:41 GMT
etag: "268b55cdf69f7dcd6408463ef25dbf3f-11"
accept-ranges: bytes
server: AmazonS3
cache-control: public, max-age=15552000
x-cache: Miss from cloudfront
via: 1.1 c9b84ddda58ae2e75aede1ce65a69ca3.cloudfront.net (CloudFront)
x-amz-cf-pop: YUL62-C1
x-amz-cf-id: sKRWeCJ1yoDaRtmexI4TNMhffK_hUZeuGQWqCtsOPswighdQRR5LAQ==