Stack Overflow archive
11 scoreaccepted

Java JSoup error fetching URL

score
11
question views
20.6K
license
CC BY-SA 3.0

Set the user-agent header:

java
.userAgent("Mozilla")

Example:

js
Document document = Jsoup.connect("https://stackoverflow.com/questions/11970938/java-html-parser-to-extract-specific-data").userAgent("Mozilla").get();
Elements elements = document.select("span.hidden-text");
for (Element element : elements) {
  System.out.println(element.text());
}

Stack Exchange

Inbox

Reputation and Badges

source: https://stackoverflow.com/a/7523425/1048340


Perhaps this is related: https://meta.stackexchange.com/questions/277369/a-terms-of-service-update-restricting-companies-that-scrape-your-profile-informa

Originally posted on Stack Overflow. Public user contributions are licensed under Creative Commons Attribution-ShareAlike.