Cloudfront

Xyne · 2010-11-14 20:47:40

I would like to know why the Wiki's CSS files are now served from Cloudfront servers. Why are we relying on a third party that "operates on a pay-as-you-go basis"? I can't imagine that bandwidth is an issue considering that the server serves binary packages.

Then there's the additional niggle of letting Amazon know every time I visit the wiki, but that's admittedly not that big a deal... still, given that part of the motivation for the recent move to HTTPS was to increase user privacy, this is a bit strange.

Don't jump on me for asking. I'm just curious.

Allan · 2010-11-14 22:46:00

I can not remember the full details, but it actually gets rid of a lot of bandwidth (~1GB per day from memory and costs very little. This frees up bandwidth for other tasks (such as faster rsync to mirrors).

To get full details, you would have to ask on arch-general.

vaccaaa · 2011-04-20 09:16:49

The most wonderful thing is that the cloudfront server is not accessible from everywhere... Here in Italy I have to use a proxy to get access to archwiki...

ngoonee · 2011-04-20 09:56:07

vaccaaa wrote:

The most wonderful thing is that the cloudfront server is not accessible from everywhere... Here in Italy I have to use a proxy to get access to archwiki...

That is indeed strange, is there some internet censorship going on in Italy?

vaccaaa · 2011-04-20 19:14:34

I understood what the problem was: a strange cache update in opendns... If I change to any other dns server it works well...Hope they (opendns) solve this problem soon... More info here: http://www.archlinux.it/forum/viewtopic.php?id=11475

cactus · 2011-04-20 19:51:31

Xyne wrote:

I would like to know why the Wiki's CSS files are now served from Cloudfront servers. Why are we relying on a third party that "operates on a pay-as-you-go basis"? I can't imagine that bandwidth is an issue considering that the server serves binary packages.

As far as I know, the cost is very low (something like 20$ a month), and yet saves origin bandwidth (which can be a burden on those who donate the servers/bandwidth), as well as makes the site load much faster.

As for 'the server serves binary packages', remember that binary packages are mirrored, but the site content comes from the origin (which also handles rsync for the mirrors).

Then there's the additional niggle of letting Amazon know every time I visit the wiki, but that's admittedly not that big a deal... still, given that part of the motivation for the recent move to HTTPS was to increase user privacy, this is a bit strange.

I believe that HTTPS was added to prevent session hijacking and sniffing (think firesheep).

At worst amazon cloudfront servers see a referer header saying that you loaded a particular page, and then the css/js/image is cached for a while by your browser. If you wish to disable that referer, some browsers provide a means to do so.
example for firefox (found via a google search result): http://cafe.elharo.com/privacy/privacy- … n-firefox/

I am not sure why browsers default to sending referer headers for https sites anyway (at least chrome is doing so). I have no problem with http, but it seems to me like a browser side issue to do so for https..

I don't know of a way to disable https referer sending in chrome.

cactus · 2011-04-20 19:54:58

vaccaaa wrote:

I understood what the problem was: a strange cache update in opendns... If I change to any other dns server it works well...Hope they (opendns) solve this problem soon... More info here: http://www.archlinux.it/forum/viewtopic.php?id=11475

opendns plays havok with CDNs in general, since many of them rely on anycast dns to find the closest POP location to serve the edge cached content from. Most people get much better streaming performance (and browsing performance due to CDN closeness) if they use their ISP's dns servers, or if they run their own recursor (unbound, bind, etc).

EDIT: Looking at opendns again (been a while) it seems they use anycast to locate the nearest one of their dns servers, so the problem is less severe, depending on how close one of their closest dns service nodes are to your ISPs primary peering location (your general point of network egress).

Last edited by cactus (2011-04-20 19:57:13)

bangkok_manouel · 2011-04-23 01:43:36

ah, I was wondering why the wiki looked like crap since a few days...

cactus · 2011-04-23 16:35:28

hmm.. it looks like cloudfront doesn't handle content negotiation very well.

curl -I 'https://d11xdyzr0div58.cloudfront.net/wiki-1.16.0/skins/archlinux/arch.css'
HTTP/1.0 200 OK
Date: Thu, 14 Apr 2011 20:04:05 GMT
Content-Encoding: gzip
Expires: Thu, 20 Sep 2012 10:43:22 GMT
Last-Modified: Tue, 21 Sep 2010 15:43:23 GMT
ETag: "879ed68dcbb4611c7c857c18c0dabe1b"
Accept-Ranges: bytes
Content-Type: text/css
Content-Length: 1144
Server: AmazonS3
Age: 764797
X-Cache: Hit from cloudfront
X-Amz-Cf-Id: 367491558ce9971478f8603d80e15f676359bfdec444e7543132e174a50e0bb160ec2af381af8a4a
Via: 1.0 c36847c5252e758d61b94a1d396be659.cloudfront.net:11180 (CloudFront), 1.0 19045f733446956507c11872d4f5914f.cloudfront.net:11180 (CloudFront)
Connection: close

Curl didn't request gzip content, but cloudfront supplied it anyway.
I think the problem is that arch is using s3 as the origin, and not a 'custom origin' pointed to the arch servers. Thus the css files appear to have been uploaded as gzipped (even though the name does not reflect that), so that cloudfront can send gzip files to end users (faster, less bandwidth). However, if a client cannot handle gzip data, or specifically asks (content negotiation) for uncompressed delivery, cloudfront will still send it compressed.

To the people who are having rendering issues....what browsers are you using? and are you using any custom settings that may interfere with content negotiation?

see: http://www.nomitor.com/blog/2010/11/10/ … loudfront/

firecat53 · 2011-04-23 18:44:52

This appears to be a bug in gnutls and/or libwebkit that just affects the webkit browsers (dwb, jumanji, et. al.) I just downgraded gnutls from 2.12.2-1 to 2.10.5-1, and the affected sites (Arch Wiki, Bitbucket, Github) are back to normal (dwb/libwebkit). I think it's related to this bug per this thread.

Scott

Last edited by firecat53 (2011-04-23 18:47:38)

cactus · 2011-04-23 23:15:21

good sleuthing firecat53!

skottish · 2011-04-25 01:03:24

gnutls 2.12.3 fixes the above problem.

jasonwryan · 2011-04-25 08:00:45

skottish wrote:

gnutls 2.12.3 fixes the above problem.

...and what a welcome fix it is. It is odd how annoying it is to browse unstyled pages in a graphical browser, when using w3m to do the same thing is completely satisfactory. I put it down to feeling discriminated against by 'The Cloud'...

Arch Linux

#1 2010-11-14 20:47:40

Cloudfront

#2 2010-11-14 22:46:00

Re: Cloudfront

#3 2011-04-20 09:16:49

Re: Cloudfront

#4 2011-04-20 09:56:07

Re: Cloudfront

#5 2011-04-20 19:14:34

Re: Cloudfront

#6 2011-04-20 19:51:31

Re: Cloudfront

#7 2011-04-20 19:54:58

Re: Cloudfront

#8 2011-04-23 01:43:36

Re: Cloudfront

#9 2011-04-23 16:35:28

Re: Cloudfront

#10 2011-04-23 18:44:52

Re: Cloudfront

#11 2011-04-23 23:15:21

Re: Cloudfront

#12 2011-04-25 01:03:24

Re: Cloudfront

#13 2011-04-25 08:00:45

Re: Cloudfront

Board footer