Web Scraping on Android & iOS (Android Studio, WebView, Shortcuts)

I tried the lazy version of web scraping on Android first: load an app listing page into a string and parse it on-device. The page came back at over a megabyte and the field I wanted, the review count, was not in the HTML at all. That single result frames this whole guide, because it is the wall almost every mobile scraping attempt hits. The content you see on screen is not always the content in the response body.

Below is what I ran on Android in Android Studio (jsoup for static HTML, a WebView for JavaScript pages), what iOS Shortcuts can and cannot do, and the API call that skips the parsing entirely. Every code block is something I executed against live targets in June 2026.

Can you web scrape on Android and iOS?

You can web scrape on Android and iOS, and the method depends on one thing: whether the data sits in the page’s HTML response or gets built by JavaScript after the page loads. That single distinction decides which tool works, so it is worth settling before any code.

Static pages put their content in the HTML the server sends back. A parser reads that HTML and pulls fields out with selectors. Dynamic pages send a near-empty shell and let JavaScript fetch and render the real content in the browser. A plain parser sees the shell and finds nothing useful.

I measured this on an App Store listing in June 2026. The raw HTML for the Instagram listing came back at 1,038,072 bytes, yet a search for the userRatingCount field returned nothing, because the App Store builds its listing client-side. Web scraping a static page is a parsing job. On a dynamic page it becomes a rendering job, and that is where mobile gets harder.

Here is how the three on-device routes line up against that split.

The rest of this guide takes them in that order: the Android library route first, then the WebView route for JavaScript, then iOS Shortcuts, then the API call that handles rendering and blocking on a server so the phone does no parsing at all.

How do you set up web scraping in Android Studio with a library?

The standard way to do web scraping in Android Studio is the jsoup library, which parses static HTML with CSS selectors. Most Android Studio web scraping setups pair jsoup for parsing with a request client, so that is where I started. jsoup is a pure Java parser, so it drops into an Android Studio project as a Gradle dependency with no native code. The jsoup project lists 1.22.2 as the current release, and it parses HTML to the same DOM that modern browsers build.

Add the dependency to your module build.gradle.kts, alongside OkHttp for the request layer:

// build.gradle.kts (Module: app)
dependencies {
    implementation("org.jsoup:jsoup:1.22.2")
    implementation("com.squareup.okhttp3:okhttp:4.12.0")
}

You also need the internet permission in AndroidManifest.xml, or every request fails before it leaves the device:

<uses-permission android:name="android.permission.INTERNET" />

With that in place, jsoup can fetch and parse a static page in a few lines. The pattern is fetch, select, read:

import org.jsoup.Jsoup

fun scrapeHeadings(url: String): List<String> {
    val doc = Jsoup.connect(url)
        .userAgent("Mozilla/5.0 (Linux; Android 14)")
        .timeout(15_000)
        .get()
    // CSS selectors, the same syntax you'd use in the browser console
    return doc.select("h2.title").map { it.text() }
}

Jsoup.connect(url).get() does two jobs in one call: it makes the HTTP request and parses the response into a Document. From there select and selectFirst take CSS selectors, so doc.select("div.price") or doc.select("a[href]") reads exactly like the browser console. That is the appeal of jsoup as an Android web scraping library: the selector knowledge transfers directly.

There is one catch that will crash the app on the first run, and it is the next thing to handle. That get() call is a network request, and Android refuses to run those on the main thread.

Fixing NetworkOnMainThreadException with coroutines

NetworkOnMainThreadException fires because the jsoup fetch runs a network request on the main UI thread, which Android has blocked since Android 3.0, API level 11 (Android reference). Network I/O on the UI thread freezes the interface, so the platform throws the exception to stop that from happening.

The fix in Kotlin is to move the fetch into a coroutine on the I/O dispatcher. Dispatchers.IO is a thread pool reserved for blocking input and output work, which is what an HTTP fetch is (Android coroutines guide):

import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.jsoup.Jsoup

suspend fun scrapeSafely(url: String): List<String> = withContext(Dispatchers.IO) {
    val doc = Jsoup.connect(url).timeout(15_000).get()
    doc.select("h2.title").map { it.text() }
}

Calling scrapeSafely from a lifecycleScope.launch { } block keeps the network work on Dispatchers.IO and hands the result back to the UI thread when it is done. The older approach used AsyncTask, which Android has since deprecated, so coroutines are the current pattern for an Android Studio web scraper that fetches in the background.

A second default trips people up in Android Studio: cleartext HTTP is blocked starting with apps that target API level 28, so a plain http:// URL fails while https:// works. Keep your targets on HTTPS and the request goes through.

If you want control over the HTTP layer (custom headers, retries, connection pooling) before handing HTML to jsoup, OkHttp is the common companion. OkHttp makes the request and you pass its response body string to Jsoup.parse():

import okhttp3.OkHttpClient
import okhttp3.Request
import org.jsoup.Jsoup

val client = OkHttpClient()

suspend fun scrapeWithOkHttp(url: String) = withContext(Dispatchers.IO) {
    val request = Request.Builder().url(url)
        .header("User-Agent", "Mozilla/5.0 (Linux; Android 14)")
        .build()
    client.newCall(request).execute().use { resp ->
        val html = resp.body?.string().orEmpty()
        val doc = Jsoup.parse(html)
        doc.select("h2.title").map { it.text() }
    }
}

This split keeps fetching and parsing separate: OkHttp owns the network, jsoup owns the HTML. It works cleanly right up to the point where the page has no real HTML to parse, because the content arrives through JavaScript. That is the case a WebView handles.

How do you scrape JavaScript-rendered pages with Android WebView?

You scrape JavaScript-rendered pages on Android with a WebView, because a WebView is a real browser engine that runs the page’s JavaScript and then lets you read the finished DOM. jsoup never executes script, so for content that appears only after rendering, a WebView is the on-device answer. Android WebView scraping works by loading the page, waiting for it to finish, then injecting a small script to read the rendered values.

JavaScript is off by default in a WebView, so the first step is enabling it through WebSettings (Android WebView guide):

val webView = WebView(this)
webView.settings.javaScriptEnabled = true
webView.loadUrl("https://example.com/dynamic-page")

Once the page loads, evaluateJavascript runs a snippet inside the page and returns the result to a callback. The method has been available since API level 19, it must be called on the UI thread, and the callback delivers the value as a JSON-encoded string:

webView.webViewClient = object : WebViewClient() {
    override fun onPageFinished(view: WebView?, url: String?) {
        // Read the rendered DOM after JavaScript has run
        view?.evaluateJavascript(
            "(function(){ return document.querySelector('h1')?.innerText; })();"
        ) { result ->
            // result arrives as a JSON string, e.g. "\"Page heading\""
            val heading = result.trim('"')
            Log.d("scrape", "heading = $heading")
        }
    }
}

The pattern is read-only: the script returns text out of the page, and you parse the JSON string in the callback. You can pull several fields at once by returning a JSON object from the injected function, then decoding it on the Kotlin side.

One method deserves a warning before you reach for it. addJavascriptInterface lets page JavaScript call back into your Android code, and Android’s own documentation flags it plainly: “Using addJavascriptInterface() lets JavaScript control your Android app… an attacker can include HTML that executes your client-side code.” The guidance is to avoid it unless you wrote every line of HTML and JavaScript in the WebView. For scraping a third-party page you do not control, stay with evaluateJavascript, which only reads values out and never grants the page access in.

WebView scraping works, and it is heavier than a parser: it spins up a full rendering engine, holds an Activity context, and ties your scrape to the UI lifecycle. On iOS, the lightweight equivalent for simple jobs is a no-code tool that ships with the system.

How do you do web scraping with iOS Shortcuts?

iOS Shortcuts handles web scraping through the Get Contents of URL action, which sends an HTTP request and returns the response for you to parse, with no code. It is the no-code route on iPhone and iPad, and it suits light jobs: hitting a JSON API, grabbing a value, building an automation around it. Apple’s own walkthrough frames Get Contents of URL as the way to request an API in Shortcuts.

The action supports the standard HTTP methods. Apple’s documentation lists them directly: “GET allows you to retrieve data,” and POST, PUT, PATCH and DELETE for writing. When you switch the method to POST, PUT or PATCH, Shortcuts adds a Request Body field where you attach JSON, form data, or a file.

A minimal iOS Shortcuts web scraping flow for a JSON endpoint looks like this:

1. Text action: hold the URL of the endpoint, for example an app-data API URL with your query parameters.
2. Get Contents of URL action: set the method to GET and pass it the Text from step 1.
3. Get Dictionary Value action: read a key out of the returned JSON, for example results or trackName.
4. Quick Look or Show Result: display the value, or feed it into the rest of your automation.

For a raw HTML page, Shortcuts has a regex-capable Replace Text action for pulling substrings out of the markup, plus Get Dictionary Value for JSON responses. That covers simple extraction. What iOS Shortcuts does not do is render JavaScript: it fetches the response but it is not a full browser, so a page that builds its content client-side returns the same near-empty shell jsoup would see. For those targets on iOS you either drive a WKWebView inside a real app or move the rendering off the device.

That on-device ceiling, the static-versus-dynamic wall on both platforms, is exactly what a server-side scraping API removes.

How do you scrape app and web data from mobile without on-device parsing?

You scrape app and web data from mobile without on-device parsing by sending the target to a scraping API and getting structured JSON back, so the rendering and parsing happen on a server instead of the phone. The mobile app makes one authenticated HTTP request, the kind Android’s OkHttp or the iOS Get Contents of URL action already make, and reads clean fields out of the response. No jsoup selectors to maintain, no WebView lifecycle, no broken parse when the page markup changes.

This matters most for app-store data, where the listing renders client-side. To show the gap, I first pulled Apple’s public iTunes Lookup endpoint, which returns App Store metadata as JSON. In June 2026 a lookup for app id 389801252 returned HTTP 200 with the title Instagram, seller Instagram, Inc., an average rating of 4.69, 29,161,052 ratings, version 434, and price 0.0, all as structured fields. The same values were absent from the rendered listing’s raw HTML I fetched earlier. Structured data beats scraped HTML for this kind of field.

A scraping API generalizes that idea to any app field or page and handles the blocking, rotation and rendering for you. With ChocoData the request is a plain GET with your API key as a query parameter, the same shape from any mobile HTTP client:

curl "https://chocodata.com/api/v1/appstore/app?id=389801252&api_key=$CHOCO_API_KEY"

From an Android app the call uses OkHttp, the same client you already added for jsoup, and the response comes back as parsed JSON you read by key:

import okhttp3.OkHttpClient
import okhttp3.Request
import org.json.JSONObject

val client = OkHttpClient()

suspend fun fetchAppData(appId: String, apiKey: String): JSONObject =
    withContext(Dispatchers.IO) {
        val url = "https://chocodata.com/api/v1/appstore/app" +
                  "?id=$appId&api_key=$apiKey"
        val request = Request.Builder().url(url).build()
        client.newCall(request).execute().use { resp ->
            JSONObject(resp.body?.string().orEmpty())
        }
    }
// Returns app fields as JSON. The call stays on Dispatchers.IO, so it
// never triggers NetworkOnMainThreadException.

On iOS the same endpoint drops into the Get Contents of URL action: set the URL to the ChocoData app endpoint with your key, leave the method on GET, and read fields with Get Dictionary Value. The phone never parses HTML in either case. You can get an API key on the ChocoData sign-up page and swap it into the snippets above.

For specific targets the App Store Search & SERP Scraper API returns ranked results for a query, the App Store Reviews Scraper API returns review pages, and the Google Play Store Scraper API covers Android listings. If you would rather not write any client code, the no-code app scraper and browser extension covers the point-and-click case. I walk through the language-specific libraries in my app-store-scraper library guide, and I cover the full extraction workflow in how to scrape App Store and Google Play data.

Which mobile scraping method should you choose?

The right method depends on where the data lives and how much you want to maintain on the device. Here is the summary I give people who ask.

Before you collect at scale, the platform rules matter. Apple’s App Store pages disallow automated access to several paths in their robots.txt, and Google Play’s robots.txt disallows /apps and store account paths. For your own apps, Apple’s App Store Connect API is the sanctioned route to sales, ratings and metadata. Scraping public listing data sits in a separate, more nuanced legal space, and respecting robots directives and rate limits is the baseline either way. The cleanest production setup keeps the heavy lifting off the phone, which holds true for an API once you move past a one-off script.