How do you Programmatically Download a Webpage in Java

by

I’d use a decent HTML parser like Jsoup. It’s then as easy as:

String html = Jsoup.connect("http://stackoverflow.com").get().html();

It handles GZIP and chunked responses and character encoding fully transparently. It offers more advantages as well, like HTML traversing and manipulation by CSS selectors like as jQuery can do. You only have to grab it as Document, not as a String.

Document document = Jsoup.connect("http://google.com").get();

You really don’t want to run basic String methods or even regex on HTML to process it.

See also:

  • What are the pros and cons of leading HTML parsers in Java?

Leave a Comment