{"id":81,"date":"2009-09-07T05:13:13","date_gmt":"2009-09-07T10:13:13","guid":{"rendered":"http:\/\/philippeadjiman.com\/blog\/?p=81"},"modified":"2009-09-07T05:13:13","modified_gmt":"2009-09-07T10:13:13","slug":"the-trick-to-write-a-fast-universal-java-url-expander","status":"publish","type":"post","link":"https:\/\/philippeadjiman.com\/blog\/2009\/09\/07\/the-trick-to-write-a-fast-universal-java-url-expander\/","title":{"rendered":"The Trick To Write A Fast (Universal) Java URL Expander"},"content":{"rendered":"

140 characters. Means something to you?<\/p>\n

This is about how twitter (and micro-blogging) was born<\/a>. Even if some profane firefox extensions try to work around this<\/a>, when it comes to insert (long) urls you may be in trouble to stick to the rule.<\/p>\n

And here comes URL shortening services.<\/p>\n

Pretty simple: The long URL http:\/\/philippeadjiman.com\/blog\/2009\/09\/01\/can-you-guess-what-is-the-hottest-trend-of-google-hot-trends\/<\/a> becomes http:\/\/bit.ly\/miUkz<\/a> that will nicely fit in your next tweet.<\/p>\n

Now everyone wants to shorten URLs. Here is a list of 90 + URL shortening services<\/a> (!!) without counting the ones that you can build by yourself<\/a>.<\/p>\n

How we (developers) can survive in this jungle if we want to retrieve the real expended version of those tons of URLs?<\/p>\n

Well, a naive JAVA version would be:<\/p>\n

public String NaiveURLExpander(String address) throws IOException {\n        String result;\n        URLConnection conn = null;\n        InputStream  in = null;\n        URL url = new URL(address);\n        conn = url.openConnection();\n        in = conn.getInputStream();\n        result = conn.getURL().toString();\n        in.close();\n        return result;\n    }<\/pre>\n
Nice. It works. But it is terribly slow.
\nWhy?Because when you analyze what happens behind the scene, the HTTP header of the new created short URL contains the line<\/p>\n
HTTP\/1.1 301 Moved<\/pre>\n
If you check the status code definition<\/a> of the HTTP protocol, you will see that means that the URL has moved permanently and that the new one should be located in the Location<\/strong> field of the HTTP header. In other words, the above java code behaves exactly as your browser: it performs a redirection, which is terribly slow.<\/p>\n
So here is the trick:
\nBut most physicians have made http:\/\/cute-n-tiny.com\/cute-animals\/cat-and-horse-pals\/<\/a> order uk viagra as their preference solution to bring impotency back to controlled stage. It is likewise helps the muscles in the penis to get  levitra online order<\/a> stiff, or uphold penis enduring to absolute sexual deed. Therefore always validate the credibility and effectiveness of the medicine can online viagra overnight cute-n-tiny.com<\/a> be achieved for about 5 hours. Leave that to the  generic viagra soft<\/a> big dogs, and find something with less competition. <\/span><\/p>\n
    \n
  1. Use an HttpURLConnection <\/strong>object to be able to specify via the setInstanceFollowRedirects <\/strong>method to not<\/span> automatically redirect (like a browser will do) while connecting.<\/li>\n
  2. Extract the Location <\/strong>value in the HTTP header.<\/li>\n<\/ol>\n
    Here you go:<\/p>\n
     public String expandShortURL(String address) throws IOException {\n        URL url = new URL(address);\n\n        HttpURLConnection connection = (HttpURLConnection) url.openConnection(Proxy.NO_PROXY); \/\/using proxy may increase latency\n        connection.setInstanceFollowRedirects(false);\n        connection.connect();\n        String expandedURL = connection.getHeaderField(\"Location\");\n        connection.getInputStream().close();\n        return expandedURL;\n    }<\/pre>\n
    If you are more a PHP guy, I saw a similar post that explain how to do it using PHP and curl<\/a>.<\/p>\n
    Note that for sake of conciseness, I do not manage errors int the code. Also, since I cannot guarantee that all the URL shortening services in the world use this exact approach (but I think most of them do), to make\u00a0 the code really universal, you just have to deal with exceptions when the Location field is null. Also, a better way would be to find some heuristics to detect if the input URL is a real one (I mean not a short one), that would avoid calling the\u00a0 openConnection() bottleneck method uselessly.<\/p>\n
    Finally, if some URL shortening services are not robust enough to check their own URLs, you also may have to deal with a corner case of “transitive shortening”\u00a0 (I’m sure there will be always some curious people that will try to shorten an already shortened URL…). Update<\/strong>: check this example: http:\/\/bit.ly\/4XzVxm<\/a> points to http:\/\/tcrn.ch\/6c8AU4<\/a> which is itself another short url!<\/p>\n
    Also to achieve real performance, such code should be multithreaded. If you have to expand millions of URLs you would probably need to use many machines. Also, a time limit should be added to avoid too long connection, with a mechanism similar to a TimerTask<\/a>.<\/p>\n
    Note that this trick makes the code 5 to 6 times faster<\/strong>. When it comes to deal with millions of short URLs, it makes a difference.