We've recently had reports of a couple of sites that have been timing out when accessed through CensorNet. Even customers who've tried adding the URL to the filter bypass cannot get the pages to download. Instead, all that they get is an "Upstream proxy did not respond in time" message.

The only solution you can do is to add the URL to the client browser's "Do not use proxy for" URL list.

The reason for this is a misbehaving webserver.

The brief explanation is that the remote end (ie the website) isn't playing nicely with the proxy information being sent from the CensorNet server. What happens is that instead of closing one connection, and continuing loading the page, the remote site is waiting for CensorNet to close the connection. Which makes no sense. It would be like expecting a user to hit "Stop" when loading a webpage when they think the remote server has finished sending them all the information they need.

The above explanation sounds a little dramatic, so if you want a more technical explanation, please read on.

We used http://www.halifax.co.uk to test this.

According to: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.10, the Connection header is used by the "sender" (the web server in this case), to indicate that the connection will be closed after the completion of the current response. However the web server is not closing the connection, so squid is hanging there waiting for the socket to close.

The RFC it implies it's the senders responsibility to close the socket first, not the recipient. So here we have a Mexican stand off, squid is waiting for the server to close the socket, and the server is waiting for squid to close the socket.

(Note, squid is the caching-proxy that censornet uses as its end-point proxy.)

-- BigMick - 08 Jan 2010