How not to do URL redirects (… the way Quora does) [Update: Quora has fixed the issue]

If you were running an online platform or a service wherein you deal with user generated content, you would often want to keep a track of links that your users post and the corresponding clicks on those. We, as webmasters, love analytics of any sort because it can be put to great use – to make better products and to make lives simpler. Trouble begins when we start doing it the wrong way.

Some WebEngage customers complained that website referrer based rules in targeting for surveys were not working when they specified referring links from Quora (#get-more-context-here). We investigated the problem and realized it was one of those bad engineering practices that people use while choosing to do URL redirects to external websites from their web applications.

What’s the wrong doing?
Quora redirects in such a way that the destination website doesn’t come to know what the original site referrer was. Take this Quora thread for example - http://www.quora.com/What-are-everyday-apps-that-use-cloud-computing. As you can see, the answer by Mat Ellis contains a link to Gigaom – http://gigaom.com/2010/06/08/how-zynga-survived-farmville/. Quora overrides it and converts the link to http://www.quora.com/_/redirect?url=http%3A%2F%2Fgigaom.com%2F2010%2F06%2F08%2Fhow-zynga-survived-farmville%2F&sig=4f01ab. Overriding links is absolutely okay.

This is where the problem starts. Underneath is a snippet of the response (header and body) sent by Quora for such redirect requests -

Response Headers
HTTP/1.1 200 OK
Server: PasteWSGIServer/0.5 Python/2.7.2
Date: Thu, 19 Jan 2012 12:37:52 GMT
Content-Type: text/html; charset=utf-8
Pragma: no-cache
Cache-Control: no-cache
Content-Encoding: gzip
Content-Length: 135
 
Response Body
<html>
  <head>
    <meta http-equiv="refresh" 
     content="0; url=http://gigaom.com/2010/06/08/how-zynga-survived-farmville/">
  </head>
</html>

As you can see, Quora emits a response body for those requests. This means the actual redirect happened to Gigaom as a page refresh from within the browser when the current location was http://www.quora.com/_/redirect... instead of the old school 302 browser redirects. Now, if the Gigaom site or any script on Gigaom’s pages wanted to know which URL’s are users on their site coming from, all they would get to know is the URL http://www.quora.com/_/redirect... They would never ever come to know where the user actually came from, e.g. in this case it should have been http://www.quora.com/What-are-everyday-apps-that-use-cloud-computing.

A simple mistake and such loss of precious information.

If Quora would have simply redirected to the destination site from their backend, there would have been no problems at all. We understand that at times you don’t have a choice but to redirect via an HTML response body to the destination site. In that case, you should do it the way Google does it for its search results (Oh, you know that Google tracks your search clicks, right?). They changed their redirection mechanism a bit since the launch of new suggest feature. As you keep searching on Google with their suggest feature on, you only change a fragment (portion after #) in the URL. Browsers (read User Agents) don’t add such anchors, as they are called, to HTTP request header called Referrer. To counter this, Google composes their redirect URLs in such a way that it gives you the original search query in the redirect URL. Based on this particular parameter a lot of products thrive – the biggest being Google Analytics which tells you about search queries that led to your website. The point is simple – preserve and pass on the context for downstream applications to work as expected.

Why are we complaining?

For the uninitiated, WebEngage is an in-site short survey tool. We let you create surveys and display those on your website in a “targeted” manner – basically, we let you filter based on multiple things like visitors geography, first time visitors, pages on your website, user’s browser etc. One of the targeting parameters is Referring site (images below).

Specifying referrer based targeting for a WebEngage survey

Short survey in action

For users trying to specify Quora URLs in the referring site section, it wouldn’t work as expected. We found out why and hence this blog post.

As online content sharing platforms keep growing, user are being offered innovative ways to share information. It becomes all the more important for these platforms to realize that there are hundreds and thousands of applications which empowers their users and others in variety of ways. Every single engineering decision they make, affects not only their own application but also the ecosystem around it. Moreover, getting it right is no rocket science either. Just stick to basics; there’s no need to re-invent because Sir Tim Berners-Lee has done all the hard work for us decades ago.

We rest our case here.

Update (20th Jan, 2012): After an uproar on Hacker News and a massive support for the cause, someone finally asked this question on Quora – http://www.quora.com/Why-does-Quora-redirect-to-URLs-in-a-way-that-loses-the-original-referrer. We’ll update this post when an official reply comes from Quora.

Update (28th Jan, 2012): We are pleased to announce that Quora has fixed its redirection logic to the old school way. This means no more broken web. We truly appreciate this gesture from Quora and thank the developer community for standing behind us.


11 responses to “How not to do URL redirects (… the way Quora does) [Update: Quora has fixed the issue]

  1. Nicolas

    I’m completely unrelated to Quora, but I ran through unexpected behaviours with redirects several times. It tought me that doing redirects the *bad* way (client-side) is useful when you want to set cookies during the redirect process.

    The standard (http://www.ietf.org/rfc/rfc2109.txt) says that during HTTP 3xx redirects, browsers must ignore the Set-Cookie headers and redirect to the target URL, for privacy reasons. Which can be pretty smart or pretty useless depending on the use-case.

    Maybe Quora sets a cookie during that redirect (even though it doesn’t seem so on your specific example), and they need to do it that way ?

    • avlesh

      Exactly our question Nicolas. If one had to really set a cookie in the re-direction process, there can be workarounds. Our argument is simple – there should be no loss of information downstream. Applications should be extra careful in passing the right context.

    • kevin

      I can’t see any reason why quora would need to set a cookie at this point, and you would easily be able to see one if it was set. Perhaps they use server side analytics and want to capture the page view.

  2. Anand

    What would be the reason to do an HTML redirect instead of the HTTP 302? I didn’t quite get your explanation of how Google does redirects. Are you saying they add a # to the redirect url?

    • avlesh

      > What would be the reason to do an HTML redirect instead of the HTTP 302
      There can be reasons. Especially if you *had* to execute some client side JS code before redirecting.

      > I didn’t quite get your explanation of how Google does redirects. Are you saying they add a # to the redirect url?
      I tried to explain it in detail but the post was getting lengthier. Will write a separate one. In brief, when you search on Google these days, notice the URL of the page. It just says Google.com/#blah..blah. If Google did a 302 redirect, all you’d get to see in the referrer is Google.com because the browser ignores everything beyond the # in Referrer HTTP header. To circumvent this, Google does redirects via HTML by adding a lot of context to the URL.

  3. Good to know Quora does it. I was not aware.

    It seams such a basic mistake that I doubt Quora did that mistake. I’m sure they have a reason for that, and it’s probably not a technical one.

    Did you try to consider that maybe Quora wants to avoid linked pages from getting the referral for some reason.

  4. Pingback: Quora

  5. the Webengage product looks a lot similiar to KissInsights .. :)

  6. Pingback: Quora Bugs: How do I resolve the error "This webpage has a redirect loop" for my Quora board? - Quora

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="" highlight="">