The noindex you cannot see: X-Robots-Tag and disappearing pages
A noindex sent as an HTTP header renders nothing in the page source. How to find it and why pages vanish without explanation.
Ashish Khan9 min read
Ashish Khan articles
- articles
- 43
- page
- 2/3
- in the whole archive
- 156
Every article, newest first
We Scanned 148 Businesses in 20 Cities. Only 16% Sign Their Work.
A 148-site study across 20 cities and five countries. Structured data is near-universal at 85%. Author attribution sits at 16.2% - and India leads the world on it.
Ashish Khan4 min read
The Web Is Blocking AI Training, Not AI Search: What 506 Top Sites Actually Do
We checked the robots.txt of 506 of the world's most-visited websites against 14 AI crawlers. Sites block AI training crawlers roughly twice as often as retrieval crawlers.
Ashish Khan5 min read
Keyword Research Guide: How to Find the Right Keywords for SEO
A step-by-step keyword research guide: how to find keywords, evaluate search volume and competition, analyze search intent, and prioritize your keyword list.
Ashish Khan16 min read
SEO for Hotels: How to Increase Direct Bookings from Search
A complete guide to SEO for hotels: how to increase direct bookings from Google search, optimize your Google Business Profile, build travel directory citations, create room and amenity pages, and reduce OTA…
Ashish Khan8 min read
YouTube SEO: How to Rank Your Videos in 2026
A complete YouTube SEO guide: how to optimize video titles, descriptions, thumbnails, tags, and engagement signals to rank higher in YouTube search results in 2026.
Link Building Strategies: 7 White-Hat Methods That Work in 2026
A guide to link building strategies for 2026: linkable assets, guest posting, digital PR, broken link building, and which strategies to avoid.
On-Page SEO Guide: How to Optimize Every Page for Search
A complete on-page SEO guide covering title tags, meta descriptions, headings, content optimization, images, internal linking, URL structure, and schema markup.
SEO for Lawyers: A Complete Guide to Law Firm SEO
A complete guide to SEO for lawyers and law firms: how to rank higher on Google, optimize your Google Business Profile, create practice-area pages, publish legal content, and attract more clients from…
SEO Expert India: How to Hire the Right SEO Expert
A complete guide to hiring an SEO expert in India: what they do, why India offers a cost advantage, how to find and choose the right expert, pricing, and what results to…
Hire SEO Experts India: A Complete Guide
A complete guide to hiring SEO experts from India: why India offers a cost advantage, how to find and evaluate candidates, pricing, engagement models, and what to look for when hiring Indian…
Ashish Khan8 min read
SEO Freelancer India: How to Find and Hire the Right Freelancer
A complete guide to hiring an SEO freelancer in India: what they do, how to find and evaluate the right freelancer, pricing, pros and cons, and how freelancers compare to agencies and…
Ashish Khan8 min read
SEO Trends 2026: What’s Changing and How to Adapt
The top SEO trends for 2026: AI-powered search, Answer Engine Optimization, voice search, Core Web Vitals, structured data, content quality, and video SEO.
Ashish Khan9 min read
Voice Search SEO: How to Optimize Your Business for Voice Search
A guide to voice search SEO: how voice search differs from text search, how to optimize for conversational queries, featured snippets, and local voice search.
Ashish Khan9 min read
Technical SEO Guide: How to Build a Search-Friendly Website
A technical SEO guide covering crawlability, indexation, site speed, Core Web Vitals, mobile-friendliness, structured data, canonical tags, and site architecture.
Ashish Khan9 min read
Original research, data attached
Three studies of how websites look to AI systems. Each one publishes its method, its limitations and a CSV with a row per site.
Open the research hub-
1.95×
Training crawlers are blocked about twice as often as search crawlersrobots.txt of 506 top sites, 14 AI crawlers
Ashish Khan -
3%
Only 8 of 243 top homepages are structurally quotable by AINine weighted checks, median score 45 of 100
Sakshi Kumari -
16%
Of 148 businesses in 20 cities, only 16% sign their workAuthor and date declared on the homepage
Ashish Khan
Want it applied to your site? Ask the people who wrote it.
A free 30-minute call with Sakshi Kumari or Ashish Khan, the specialists who run MoxSEO and wrote 72 of these 156 articles. Bring the post that made you think, and your numbers.
A call is not an engagement. Plans start at $699 a month, and we never guarantee rankings.
On the callfrom this archive
Ashish KhanSEO Specialist. AI crawler access, city-market readiness, keyword research.
43articlesSakshi KumariSEO Specialist. AI answer extractability, attribution, knowledge panels.
29articles