Java Web Scraping Tutorial (2026): jsoup, Step by Step

Java has a reputation for ceremony, but web scraping is one place it's refreshingly direct, thanks to jsoup — a single library that fetches a page and parses it, then hands you CSS selectors as familiar as the ones in your browser's dev tools. No separate HTTP client, no separate parser. This tutorial uses jsoup and nothing else, and I ran every snippet against the live site; the output you see is what I got, including the one character-encoding gotcha that trips up almost everyone.

We'll scrape books.toscrape.com, a sandbox built for practice: 1,000 fictional books over 50 pages, no terms-of-service worries, no bot detection. Prefer no code? An AI website scraper does this from a pasted URL — compared honestly at the end.

What do I need to scrape a website in Java?

A JDK (I'm on a recent LTS) and the jsoup library. You don't need Maven or Gradle to start — download the single jar and put it on the classpath:

curl -O https://repo1.maven.org/maven2/org/jsoup/jsoup/1.18.3/jsoup-1.18.3.jar
javac -cp jsoup.jar Books.java
java -cp .:jsoup.jar Books

On Windows the classpath separator is ; rather than :. In a real project you'd let Maven or Gradle manage jsoup as a dependency, but the jar-on-the-classpath route keeps the focus on scraping, not build config.

How do I fetch and parse a page with jsoup?

Here's what makes jsoup pleasant: Jsoup.connect(url).get() both downloads the page and returns a parsed Document in one call. Set a User-Agent and a timeout while you're at it, since the default agent gets blocked and no timeout means a hung request can stall forever:

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;

Document doc = Jsoup.connect("https://books.toscrape.com/")
        .userAgent("Mozilla/5.0 (compatible; MyScraper/1.0)")
        .timeout(20000)
        .get();

System.out.println(doc.title());

That prints the page title. Now query with CSS selectors. Inspect the page first: each book is an <article class="product_pod">, with the title in h3 a (its title attribute), the price in .price_color, and stock in .instock.availability. A record is a tidy container for each row:

import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.util.*;

record Book(String title, String price, String availability) {}

Elements pods = doc.select("article.product_pod");
List<Book> books = new ArrayList<>();
for (Element el : pods) {
    books.add(new Book(
        el.selectFirst("h3 a").attr("title"),
        el.selectFirst(".price_color").text(),
        el.selectFirst(".instock.availability").text()));
}
System.out.println("Found " + books.size() + " books");

select returns all matches; selectFirst returns the first within an element. Reading the title from the title attribute rather than the truncated link text gives the full name. That found all twenty books on the page.

How do I scrape multiple pages?

Extract the logic into a method, then loop the numbered pages and stop at the first empty one:

static List<Book> scrapePage(String url) throws IOException {
    Document doc = Jsoup.connect(url)
            .userAgent("Mozilla/5.0 (compatible; MyScraper/1.0)")
            .timeout(20000)
            .get();
    List<Book> books = new ArrayList<>();
    for (Element el : doc.select("article.product_pod")) {
        books.add(new Book(
            el.selectFirst("h3 a").attr("title"),
            el.selectFirst(".price_color").text(),
            el.selectFirst(".instock.availability").text()));
    }
    return books;
}

public static void main(String[] args) throws Exception {
    List<Book> all = new ArrayList<>();
    for (int page = 1; page <= 2; page++) {
        List<Book> books = scrapePage(
            "https://books.toscrape.com/catalogue/page-" + page + ".html");
        if (books.isEmpty()) break;          // ran past the last page
        System.out.println("page " + page + ": " + books.size() + " books");
        all.addAll(books);
        Thread.sleep(1000);                   // be a polite guest
    }
    System.out.println("TOTAL: " + all.size() + " books");
}

Two pages, kept light:

page 1: 20 books
page 2: 20 books
TOTAL: 40 books

The Thread.sleep(1000) is the courtesy that keeps you from hammering the server — raise the loop to 50 and you'd collect all 1,000 books, but keep the pause.

The encoding gotcha: why £ becomes ?

Run the code above, print a price, and you may see ?51.77 instead of £51.77. This trips up nearly everyone, so it's worth being explicit: jsoup read the page correctly — it handles UTF-8 fine. The mangling happens on the way out, because Java's console and default file writer may use a non-UTF-8 platform encoding. Fix both ends by being explicit about UTF-8:

import java.io.*;
import java.nio.charset.StandardCharsets;

// UTF-8 for console output
PrintStream out = new PrintStream(System.out, true, StandardCharsets.UTF_8);
out.printf("%-40s %s%n", books.get(0).title(), books.get(0).price());

With the UTF-8 PrintStream, that line printed cleanly:

A Light in the Attic                     £51.77

Same lesson applies to files: write them through an explicit UTF-8 writer, never a bare FileWriter that inherits the platform default.

How do I export the data to CSV?

Java has no built-in CSV writer, but a flat table is simple to write by hand — just quote the fields, and use the UTF-8 writer from above so prices survive:

try (Writer w = new OutputStreamWriter(
        new FileOutputStream("books.csv"), StandardCharsets.UTF_8)) {
    w.write("title,price,availability\n");
    for (Book b : all) {
        w.write("\"" + b.title() + "\"," + b.price() + ",\"" + b.availability() + "\"\n");
    }
}

The resulting file opened correctly, pounds intact:

title,price,availability
"A Light in the Attic",£51.77,"In stock"
"Tipping the Velvet",£53.74,"In stock"

For data with quotes or newlines inside fields, use a library like Apache Commons CSV that escapes every case; for clean tabular data, this is enough.

When does jsoup stop working?

The familiar wall: JavaScript. jsoup fetches the server's HTML and never runs a script, so a page that renders its content in the browser hands jsoup a document without your data. View source to check — if the values aren't there, no selector will find them. For those pages you drive a real browser with Selenium or HtmlUnit, at a real cost in speed and setup. The other wall is serious anti-bot protection, which a User-Agent won't clear.

Do you actually need to write Java for this?

For a static site, jsoup is a clean, type-safe tool, and inside an existing Java or Android app it's the obvious choice. But scraping's real cost is upkeep, not the first program: selectors break when the markup shifts, JavaScript pages pull in Selenium and the infrastructure to run it, and "check this every morning and tell me what changed" means building scheduling, storage, and diffing yourself — the genuinely hard 80%.

An AI scraper absorbs all of that. Website Scraper renders the page (JavaScript included), extracts by understanding the content rather than a product_pod selector you maintain, and turns any scrape into a monitored alert with no infrastructure on your side. If Java is your stack, this tutorial is the right path. If you only want the data out, paste the URL into the no-code scraper. The how to scrape data from a website guide covers the workflow, and the Python, PHP, JavaScript, and Go tutorials show the same job in other languages.