Can Your Browser Run AI? Checking WebGPU and On-Device AI Performance

AI features that summarize text, sort photos, or transcribe speech are moving into ordinary web pages. What still surprises many people is that some of them now run inside your own browser instead of on a remote server. So can your browser really run AI? In short, yes, if the model is small. How comfortable it feels, though, varies a lot from device to device. This guide explains the ideas behind that difference, WebGPU and on-device AI, the conditions you need, and how to check your own device with Web Bench.

What WebGPU is and how it differs from WebGL

WebGPU is a modern web standard that lets a page talk to the GPU directly. For years, the usual way to use the GPU in a browser was WebGL. As the name suggests, WebGL was designed around graphics, that is, drawing things on screen. If you only wanted the GPU to do math, you had to disguise the calculation as a drawing job.

WebGPU differs in two ways. First, its design reflects modern native graphics interfaces such as Vulkan, Metal, and Direct3D 12, so the way work is handed to the GPU is more explicit and efficient. Second, it officially supports compute shaders. A compute shader is a program that spreads pure calculation across the many cores of a GPU, with no drawing involved. Doing the same arithmetic over and over on a huge amount of data, such as matrix multiplication, suits it well, and matrix multiplication is the bulk of what an AI model does.

• WebGL: an older API centered on graphics rendering, widely used across browsers.
• WebGPU: a newer API built for both graphics and general-purpose computing, where compute shaders can express parallel work like AI directly.
• In common: both let a web page use your device's GPU without installing anything.

In short, WebGPU is the proper road for running calculations on the GPU from a browser, which makes it a good match for in-browser AI. It is not switched on everywhere in the same way, however. Support depends on the browser version, operating system, graphics driver, and settings, and it keeps changing over time. For that reason this guide does not claim that any particular browser works. It shows you how to check for yourself.

Why run AI in the browser: the benefits of on-device AI

On-device AI means that, instead of sending a request to a model hosted on a server, you download the model to your own device and run it there. Doing this in a browser means there is nothing to install. There are three main benefits.

1. Privacy: the sentences or photos you enter are processed on the spot and do not have to leave your device. That matters most for features that handle sensitive content.
2. Offline use: once a model has been downloaded, it may keep working where the connection is poor. Note that if the browser cache is cleared or storage runs low, the model may need to be downloaded again.
3. Server cost: because your device does the computing, the people running the service do not need a fleet of inference servers. That makes AI features realistic for free services and small sites too.

There are trade-offs, of course. The model has to be downloaded, which costs time and data the first time, and your device's performance sets the ceiling for speed and quality. That is why small, compact models are the usual choice rather than huge ones. It is also why it helps to estimate beforehand how comfortable a given device will be.

What you need to run AI in a browser

A few things have to line up for in-browser AI to run well. Go through them one by one.

• WebGPU support: the browser must offer WebGPU, and it must actually be able to obtain a graphics adapter. Even an up-to-date browser can have it turned off because of a driver or a setting.
• Memory: the model needs room to live. Even a small model takes memory for the downloaded files and for intermediate values during the calculation. A phone with little free memory, or a desktop with many tabs open, feels the pressure sooner.
• shader-f16: an optional WebGPU feature that lets shaders use 16-bit floating-point numbers (f16). Numbers take half the space, so the same job needs less memory and bandwidth, which helps AI. Not every GPU and driver offers it, so some devices have it and others do not.
• Storage and network: you need space to keep the model files and should allow for the data used on the first download. On a metered mobile connection, it is better to try on Wi-Fi.
• Power and heat: a laptop in power-saving mode or on battery may run slower, and a phone slows itself down when it gets hot.

You do not have to dig through settings pages to find out which of these apply. Web Bench's result screen shows which GPU path was used along with the features and limits of the WebGPU adapter, and any test that was skipped because it was unsupported comes with a reason. Reading that information gives you clues about what your device is missing.

When WebGPU is missing: the WASM fallback

A device without WebGPU is not necessarily shut out of browser AI. There is a fallback called WASM (WebAssembly). WASM is a compact binary code format that browsers run quickly on the CPU, and many AI runtimes use it as a safety net when no GPU path is available.

The difference is where the calculation happens. On the WebGPU path, the many cores of the GPU work in parallel. On the WASM path, the CPU does the work. It is common for the same model to run much more slowly on WASM. That is not a malfunction, just a difference in execution path, and it is why putting a WebGPU result and a WASM result side by side can mislead you.

• WebGPU path: calculation happens on the GPU. On supported devices you can usually expect a smoother experience.
• WASM path: calculation happens on the CPU. It is more likely to work anywhere, but it can be slow.
• When WebGPU is unavailable and Web Bench runs on WASM instead, it labels the result as a WASM fallback.

How to enable WebGPU

How the AI inference test in Web Bench works

AI inference in Web Bench is optional. The standard test already covers CPU, memory, GPU, browser, and storage, and AI is something you add only if you want it. Here is how it behaves.

• It runs only if you agree to download the models. If you leave the consent checkbox unchecked, nothing is downloaded.
• After you agree, the models are fetched from Hugging Face's public CDN. A browser cache is reused when available, but depending on storage space and browser settings, files may be downloaded again.
• The inputs are a fixed sentence and a fixed image. Your own files and personal data are never used, and because every device is measured with the same input, results can be compared fairly.
• The text model is SmolLM2-135M-Instruct and the image model is MobileNetV4. Both are published under the Apache-2.0 license, which allows commercial use, and you can check their sources on the licenses page.
• Download size depends on the format. For the text model, the q4f16 format is about 117MB and q4 is about 181MB, while the image model is about 15.1MB. Tokenizer and runtime files are separate, and if both text formats are tried, the total download can reach about 350MB.
• Before you start, the screen shows a download of about 130MB. The actual amount depends on which format your device supports, so also watch the per-file sizes on the progress screen.
• If you measure AI, you choose whether it counts toward the overall score, and if you did not agree, or the run was stopped or failed, it is left out of the total automatically. The ranking tables are likewise split into with-AI and without-AI.
• It runs on the GPU if WebGPU is available and on WASM if not, and the result says which path was used.

The text model format is chosen to suit your device. If the WebGPU adapter supports shader-f16, the q4f16 format is used. Otherwise, or on the WASM path, the q4 format is used. If loading q4f16 fails, Web Bench falls back to q4 and tells you so on the progress screen. As a result, each result records both the path (WebGPU or WASM fallback) and the text format (q4f16 or q4). Only compare results where both match.

Each item gets a warm-up followed by three rounds. If you hide the tab in the middle, that round is discarded, and when you return the test reuses the cache and starts again from the warm-up. Leaving the tab in front while it runs gives the most reliable numbers.

Reading the AI results and the GPU compute tests

The AI inference test reports four values.

• Time to first token (ms): the time from starting generation until the first piece of text (a token) appears. Lower is better. It is close to the feeling of pressing a button in a chatbot and waiting for a reaction.
• Prompt processing (tokens/s): how fast the model reads the text you give it. It is the number of input tokens divided by the time of the first calculation pass, and higher is better. It matters for features that take long documents as input.
• Text generation (tokens/s): the throughput over the 127 tokens after the first one while generating 128 tokens. Higher is better, and it maps directly to how fast text flows onto the screen.
• Image classification, one image (ms): the time to classify a single fixed 224x224 image. Preprocessing is excluded, and lower is better.

A token is the unit a model uses to cut text up; think of it as roughly a word or a piece of a word. These values come from a small model, SmolLM2-135M, so they do not mean a larger model would run at the same speed. They are a reference for whether a small AI feature will feel comfortable, nothing more.

Looking at the GPU Compute category alongside makes the picture much easier to read. GPU Compute measures speed when you hand calculation, rather than graphics, to the GPU, and it is closest to heavily parallel work such as in-browser video processing and AI inference.

• Matrix multiplication, f32 and f16: shown in GFLOPS, higher is better. It is the test most similar to the core calculation of AI. The f16 version runs only when shader-f16 is available.
• Image convolution, particle simulation, and parallel sum: shown as elements processed per second, higher is better.
• Every GPU compute test reads back its result buffer and compares it with an answer computed on the CPU. A fast but wrong result does not count toward the score.

If the GPU compute tests were skipped, that can be a sign that WebGPU is missing or a required feature is not available. If only the f16 matrix multiplication is missing, your environment probably has WebGPU but not shader-f16. In that case it is natural for the AI text model to run in q4 rather than q4f16.

Judging how well your device suits AI work

The hardest part is knowing what to make of the numbers. Reading them in the following order keeps the judgment reasonable.

1. Check the path first. See whether the run used WebGPU or the WASM fallback. If it was the WASM fallback, the CPU did the work without help from the GPU, so slower values are natural.
2. Check the format. q4f16 and q4 are different conditions, so compare results with the same format.
3. Look at the GPU compute scores. If things like matrix multiplication do well, the device has the parallel muscle, and AI throughput is likely to follow.
4. Match text generation (tokens/s) to your use case. If text flows slowly, conversational features can feel sluggish. Tasks that finish in one shot, like classification or a short summary, are easier to live with at lower speeds.
5. Look at time to first token. It is the waiting time before anything reacts after a click, so it has a large effect on how comfortable the feature feels.
6. Look at the image classification time. For features that process many photos in a row, such as organizing or filtering a gallery, the per-image time adds up quickly.

Values can change on the same device depending on conditions. A laptop in battery mode, another tab using the GPU, or a device that has heated up can all produce lower numbers. Rather than fixating on one number, repeat the test two or three times under the same conditions and read the range. To compare a phone with a laptop, or an old device with a new one, match the path and format first and measure in the same browser.

There is a reason this guide does not draw a line such as "above this score, it works." The speed you need depends on the model and the use case, and the scores are only reference values for fixed inputs. The most accurate judgment is to hold your device's numbers up against the feature you actually want to use.

A checklist for WebGPU and on-device AI support

Finally, here is a list of things to check before you try browser AI on your device.

1. Is your browser updated to the latest version?
2. Does the Web Bench result screen show the GPU path as WebGPU? If it shows only WebGL2, WebGPU may be turned off or unsupported in your environment.
3. Is the WebGPU adapter information (features and limits) shown? Check whether shader-f16 is supported.
4. Did the GPU compute tests, especially matrix multiplication in f32 and f16, run rather than being skipped? If they were skipped, read the reason on the result screen.
5. Do you have enough free memory and storage? Close other heavy tabs and apps before trying.
6. On a laptop, plug in the power and use a high-performance mode. On a phone, connect the charger and try somewhere cool.
7. Can you download the models over Wi-Fi rather than mobile data? About 130MB is announced, and depending on the format, more may be downloaded.
8. Did you measure two or three times with AI turned on to see the range? Note down both the path (WebGPU or WASM fallback) and the text format (q4f16 or q4).

Web Bench is a free tool that measures CPU, memory, GPU, browser, and storage performance in your browser with no install and no login. The quick test takes about 30 seconds and the standard test about 2 to 3 minutes, and AI inference is something you agree to and add only if you want it. Every measurement is processed on your device, and results go to a server only if you agree to join the ranking. Start with the standard test without AI to see your GPU path and GPU compute results, then add AI inference when you are curious.

Open Web Bench