The market's technical baseline: what we found on 35 stores

On 7 and 8 August 2026 we measured the technical state of live sites — online stores and service sites across a range of niches. On 17 September 2026 we repeated the same measurement, and every figure below comes from the repeat. No admin access, no hosting password, no involvement from the owners. Everything described below is what a site hands to anyone who opens it: response codes, time to first byte, sitemap, canonical, structured data, hreflang, certificate and the PHP version in the headers.

This is not a piece about a bad market. It is a description of the baseline every new store competes against. Most of the gaps below can be closed in one evening, which is why they read better as a list of your own opportunities than of someone else's mistakes.

How we measured, and why from the outside

Every measurement is a request to a public address and a reading of the response. Headers, the HTML of the homepage, robots.txt, the sitemap, and a product page wherever one could be opened. Three reasons to work this way.

This is how a crawler sees a site. Whatever is invisible from the outside does not exist for indexing, no matter how tidy the admin panel is inside.

It is the only way to compare yourself with a competitor. You will never get into their panel, yet from the outside you see exactly what Google sees.

It is fast. A few minutes per site. So the full technical sweep — PHP version, whether the sitemap is usable, certificate state — covers 35 sites. The per-niche breakdowns below are separate samples of four to eight sites each, taken with the same set of checks. They are different sets, and wherever their numbers disagree we say so.

Two limits of the method are worth naming up front. Five sites answered our script with an anti-bot stub page instead of content, and one more did not answer at all — all six are excluded from every denominator, which is why the sweep covers 35 sites and not 41. And breadcrumb markup is looked for not only on the homepage but on a product page too, so that point below carries two figures.

What an external measurement cannot see gets its own section at the end. The list is longer than we would like.

Six gaps that keep repeating

No canonical on the homepage at all

A finding that repeats in every niche: 23 of the 35 homepages carry no canonical. A canonical is the line where a page tells search «this is my real address». Without it, every variant of the same page is entitled to count as a separate one: with and without www, with a sort parameter, with a campaign tag in the link, with and without a trailing slash.

The homepage is the simplest page on a site and the first one anyone thinks about. If the canonical is missing there, it is certainly missing in the catalog and the filters — and the catalog is exactly where duplicates are born. We covered the mechanics and the checks separately: canonicalization and removing duplicate URLs.

Breadcrumbs on the screen, nothing in the code

The «Home → Category → Product» trail is drawn almost everywhere. The markup that explains that trail to search is almost nowhere: 33 of the 35 homepages have none, and if a product page counts too, 20 of 35 have none anywhere.

Eight niches in a row without a single exception on the homepage. The homepage, though, is not where breadcrumbs usually belong, so we look at a product page separately: the markup turns up more often there, and even then 20 sites of 35 have none anywhere. A result like that usually means not carelessness but that the store template never covered the topic, and nobody opened the question after installation.

The sitemap: missing, borrowed or far too large

Across the 35-site sweep, seven have no usable sitemap, and on two more the sitemap points to an outside domain. Search receives a list of addresses that do not belong to that site.

The other side of the problem is size. Medians by niche:

A store with a few thousand products and a sitemap of ninety thousand addresses does not have ninety thousand pages. It has the same products laid out across filters, sort orders and tags. What to do about it is in the breakdown of sitemaps and robots.txt, and you can look at your own quickly with this tool.

PHP that no longer receives security updates

Two sites in the 35-site sweep. In the building materials sample, four of six. One more turned up in each of jewelry out of four, stationery out of four and construction firms out of eight.

The gap between «two of thirty-five» and «four of six» is not an error but two different samples. And it is telling in itself: a market average hides the niches where an outdated build is the normal state of affairs. This is no longer a search question. An unsupported version means nobody will patch the vulnerabilities found in it.

The server's first response

Time to first byte is not site speed yet — only how long the server thinks before the first character of the answer. Images, styles and scripts come after that.

The spread between niches is more than threefold, and catalog size does not explain it: footwear, with a median catalog of 12,762 addresses, answers faster than legal sites with a median of 524. In furniture, the first response takes longer than a second on two sites out of four.

The product is not marked up

A product page could be opened on 25 sites. Ten of them carry Product markup, ten carry Offer price markup, five have their specifications marked up and three have a rating. On three product pages out of five, search can read neither what the thing is nor what it costs.

Separately, some sites carry no structured data at all — eleven of thirty-five: three of seven in optics, two of four in stationery, two of six in household chemicals. What this looks like in code is in the breakdown of structured data.

What it costs the owner

No percentages here: we measured code, not the till. But the mechanism behind each point is direct.

Duplicates eat the crawl. A crawler arrives with a limited budget of requests. If the sitemap walks it through tens of thousands of variants of one catalog, that budget goes on copies while new products wait their turn for days.

Without Product and Offer the snippet stays plain text. Price, availability and rating in the results are read from markup. No markup, no lines — and next to you stands a competitor who has them.

Without breadcrumb markup the path is invisible in the results. Instead of a readable «section → subsection → product» the user sees a bare URL.

A slow first response multiplies into everything else. Seven hundred milliseconds to the first byte happens before the browser has downloaded a single file. Everything else is added on top of that number.

An old PHP version is no longer a marketing matter. It is an open risk to the customer and order database.

The wider list of things worth going through before a season is collected in our technical checklist for an online store.

Fifteen minutes to check your own

Every command reads your site's public response and changes nothing.

1. Canonical on the homepage.

curl -s https://your-site.com/ | grep -io '<link[^>]*canonical[^>]*>'

Empty output is your first job. A line pointing at another domain, or carrying a parameter, is a job too.

2. Whether a sitemap is declared, and what is in it.

curl -s https://your-site.com/robots.txt | grep -i sitemap
curl -s https://your-site.com/sitemap.xml | grep -c '<loc>'

Compare the second number with the count of products and categories in your admin panel. A gap of several times over means duplicates.

3. Duplicates inside the sitemap itself.

curl -s https://your-site.com/sitemap.xml \
  | grep -o '<loc>[^<]*' | sort | uniq -d | wc -l

It should be zero.

4. First response.

curl -s -o /dev/null -w 'to first byte: %{time_starttransfer} s\n' \
  https://your-site.com/

Three runs in a row, take the worst. There is something to compare against: the niche medians above run from 208.4 to 711.3 ms.

5. Markup on a product page.

curl -s https://your-site.com/product-url | grep -c 'application/ld+json'

Zero means there is no markup at all. One or more means you still need to look at what exactly is marked up.

Response codes across a list of addresses are easy to collect with the status code checker. If after these five points you want the full picture, we run a technical audit starting from the same checks, only across every page rather than the homepage.

Honest limits

An external measurement misses the main thing. It does not know how many visitors you have or where they come from, how many orders are lost in the cart, how your managers work, or whether analytics is configured at all. It sees the facade and draws conclusions about the facade.

A median across four to eight sites is a reference point, not statistics. It shows what counts as normal in a niche and gives no right to claim that the whole market looks that way. One site in a sample can be an outlier, and usually at least one is.

The presence of a tag does not mean it is correct. We recorded whether a canonical exists and whether structured data exists, not whether they point where they should. Correctness is checked by hand, page by page.

And the main one: not a single item on this list converts into money directly. We do not know what a missing Offer costs a particular store, and neither does anyone who quotes a percentage. What we do know is that all of it is visible from the outside, measurable in minutes, and in most cases fixed faster than it gets discussed.

We measure our own site with the same set of checks and published the result together with the commands: a site audit in one day.