Short answer: Technical SEO is the infrastructure work that makes sure Google can crawl your site, index its pages and understand them correctly. The core checks are: robots.txt and the sitemap, pages returning the correct HTTP status codes, canonical tags, HTTPS, mobile-friendliness, page speed, content arriving from the server as ready-made HTML, and structured data.
1. Can Google crawl your site?
- robots.txt: Open
yoursite.com/robots.txt. Make sure important sections have not been accidentally closed off withDisallow. Blocking private areas such as the admin panel is correct. - robots.txt can't do noindex: Google does not support a
noindexrule in robots.txt. If you don't want a page to be indexed, add<meta name="robots" content="noindex">to the page and do not block that page in robots.txt; if you block it, Google will never see the noindex tag. - Server access: A firewall, CDN or bot protection may be blocking search engine crawlers. Run a live test with the URL Inspection tool in Search Console.
2. Is your sitemap correct?
An XML sitemap should list the canonical URLs you want indexed. Redirected URLs, URLs returning 404 and noindex URLs should not be in the sitemap. A single sitemap can contain at most 50,000 URLs and be at most 50 MB uncompressed; larger sites use multiple sitemaps and a sitemap index file. The sitemap's URL should be declared in robots.txt with a Sitemap: line and submitted to Search Console.
3. Do your pages return the correct status code?
| Situation | Correct code | Common mistake |
|---|---|---|
| Page exists | 200 | — |
| Moved permanently | 301 or 308 | Using 302 (temporary) |
| Page doesn't exist | 404 or 410 | A page that says "Not found" but returns 200 (soft 404) |
| Server error | 5xx | If it keeps recurring, crawling slows down |
Soft 404s are especially common on modern JavaScript sites: the screen says "page not found," but the server returns 200. Google treats these pages as low-value content.
4. Does every page have a single correct URL?
If the same content opens at several URLs, such as with and without www, over http and https, or with and without a trailing slash, Google decides on its own which one to show. The right approach:
- Choose a single preferred URL format and redirect all others to it with a 301.
- Add a self-referencing canonical tag to every page:
<link rel="canonical" href="https://yoursite.com/page"> - Always use this preferred URL in your internal links as well.
5. Are HTTPS and mobile-friendliness in place?
Google indexes sites based on their mobile version. If content is missing on mobile, text is hard to read or horizontal scrolling is required, your rankings are affected too. All pages should load over HTTPS with a valid certificate, and http URLs should redirect to https.
6. Is your content delivered ready from the server?
If a page's main text is only created after JavaScript runs in the browser, some search engines, and especially some AI crawlers, may not be able to see that content. To check, right-click the page and choose "View page source": if the headings and main text are visible in the source, your content is being delivered ready from the server. In frameworks such as Next.js, server-side rendering or static generation solves this problem.
7. Are your title, description and heading hierarchy in order?
- Every page should have a unique
<title>tag; titles longer than about 60 characters get truncated in search results. - The meta description is not a required ranking signal, but it affects click-through rate; keep it to around 160 characters.
- Each page should have exactly one H1, with subheadings following as H2 and H3 in a logical order.
- Images should have alt text that describes their content.
8. Speed and structured data
For page experience, review your Core Web Vitals metrics, and for schemas suited to your page types, see our structured data guide.
Which tools should you use for these checks?
| Tool | What for? |
|---|---|
| Google Search Console | Indexing status, crawl errors, sitemaps, live testing |
| PageSpeed Insights / Lighthouse | Speed, accessibility, basic SEO checks |
| Rich Results Test | Structured data |
| "View page source" in the browser | Whether content comes from the server, canonical and meta tags |
Curious about your site's technical SEO health? As part of our SEO / GEO / AEO service, we crawl your site, report the findings in order of priority and implement the fixes.
Sources
Frequently Asked Questions
No. Every new page, design change or plugin can introduce new problems. You should check your Search Console reports once a month and run a crawl after any major change.
Not entirely. robots.txt blocks crawling, but a URL that receives links from other sites can still appear in the index. To remove it from the index, add a noindex meta tag to the page and do not block it in robots.txt.
Not directly. A sitemap makes it easier for Google to find your pages and notice updates; rankings depend on content quality and other signals.