This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: seo-aeo-best-practices
|
||||
description: SEO and AEO best practices for metadata, Open Graph, sitemaps, robots.txt, hreflang, JSON-LD structured data, EEAT, and content optimized for search engines and AI answer surfaces. Use this skill when implementing page SEO, technical SEO, schema markup, international SEO, AI-overview readiness, or improving content for Google, ChatGPT, Perplexity, and similar assistants.
|
||||
---
|
||||
|
||||
# SEO & AEO Best Practices
|
||||
|
||||
Principles for optimizing content for both traditional search engines (SEO) and AI-powered answer engines (AEO). Includes Google's EEAT guidelines and structured data implementation.
|
||||
|
||||
## When to Apply
|
||||
|
||||
Reference these guidelines when:
|
||||
- Implementing metadata and Open Graph tags
|
||||
- Creating sitemaps and robots.txt
|
||||
- Adding JSON-LD structured data
|
||||
- Optimizing content for featured snippets
|
||||
- Preparing content for AI assistants (ChatGPT, Perplexity, etc.)
|
||||
- Evaluating content quality using EEAT principles
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### SEO (Search Engine Optimization)
|
||||
Optimizing content to rank well in traditional search results (Google, Bing).
|
||||
|
||||
### AEO (Answer Engine Optimization)
|
||||
Optimizing content to be selected as authoritative answers by AI systems.
|
||||
|
||||
### EEAT (Experience, Expertise, Authoritativeness, Trustworthiness)
|
||||
Google's framework for evaluating content quality.
|
||||
|
||||
## References
|
||||
|
||||
Start with the one reference that matches the task, such as technical SEO, structured data, EEAT, or AI-answer readiness. See `references/` for detailed guidance:
|
||||
- `references/eeat-principles.md` — EEAT implementation and author schema
|
||||
- `references/structured-data.md` — JSON-LD patterns (Article, FAQ, Breadcrumb, Product)
|
||||
- `references/technical-seo.md` — Technical SEO checklist (metadata, sitemaps, hreflang, robots.txt)
|
||||
- `references/aeo-considerations.md` — AI/AEO considerations (AI Overviews, crawler management)
|
||||
@@ -0,0 +1,159 @@
|
||||
# AI/AEO Considerations
|
||||
|
||||
Answer Engine Optimization (AEO) prepares content to be selected as authoritative answers by AI systems like ChatGPT, Perplexity, Google AI Overviews, and Bing Copilot.
|
||||
|
||||
## How AI Selects Answers
|
||||
|
||||
AI systems evaluate content based on:
|
||||
|
||||
1. **Clarity:** Is the answer direct and easy to extract?
|
||||
2. **Authority:** Is the source trustworthy?
|
||||
3. **Comprehensiveness:** Does it fully address the question?
|
||||
4. **Recency:** Is the information up to date?
|
||||
5. **Structure:** Can the AI parse and understand it?
|
||||
|
||||
## Content Structure for AI
|
||||
|
||||
### Direct Answers First
|
||||
Lead with the answer, then explain.
|
||||
|
||||
**Bad:**
|
||||
> The history of JavaScript dates back to 1995 when Brendan Eich... [500 words later] ...JavaScript runs in the browser.
|
||||
|
||||
**Good:**
|
||||
> JavaScript is a programming language that runs in web browsers. It was created in 1995 by Brendan Eich...
|
||||
|
||||
### Clear Headings
|
||||
Use descriptive H2/H3 headings that match user questions.
|
||||
|
||||
**Bad:** "Overview" → "Details" → "More Information"
|
||||
**Good:** "What is X?" → "How does X work?" → "When should you use X?"
|
||||
|
||||
### Lists and Tables
|
||||
AI extracts structured information more easily than prose.
|
||||
|
||||
```markdown
|
||||
## Benefits of Structured Content
|
||||
|
||||
- **Reusability:** Use content across channels
|
||||
- **Flexibility:** Change presentation without changing content
|
||||
- **Scalability:** Manage large content volumes
|
||||
```
|
||||
|
||||
### FAQ Format
|
||||
Question-answer pairs are ideal for AI extraction.
|
||||
|
||||
```typescript
|
||||
// Schema for AI-friendly FAQs
|
||||
defineType({
|
||||
name: 'faq',
|
||||
type: 'document',
|
||||
fields: [
|
||||
defineField({ name: 'question', type: 'string' }),
|
||||
defineField({ name: 'answer', type: 'text' }),
|
||||
defineField({ name: 'category', type: 'reference', to: [{ type: 'faqCategory' }] }),
|
||||
]
|
||||
})
|
||||
```
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### Structured Data (Critical)
|
||||
JSON-LD helps AI understand content type and relationships.
|
||||
|
||||
```typescript
|
||||
// FAQ structured data
|
||||
const faqSchema = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
mainEntity: faqs.map(faq => ({
|
||||
"@type": "Question",
|
||||
name: faq.question,
|
||||
acceptedAnswer: {
|
||||
"@type": "Answer",
|
||||
text: faq.answer
|
||||
}
|
||||
}))
|
||||
}
|
||||
```
|
||||
|
||||
### Canonical Content
|
||||
Ensure AI finds your authoritative version, not copies.
|
||||
|
||||
- Set canonical URLs
|
||||
- Avoid duplicate content across pages
|
||||
- Use `rel="canonical"` for syndicated content
|
||||
|
||||
### Freshness Signals
|
||||
AI systems prefer current information.
|
||||
|
||||
- Display publish and update dates prominently
|
||||
- Update content regularly with substantive changes (superficial updates like changing dates without meaningful edits can be counterproductive)
|
||||
- Use `dateModified` in structured data
|
||||
|
||||
## Content Quality Signals
|
||||
|
||||
### Author Credentials
|
||||
AI systems increasingly check author authority.
|
||||
|
||||
- Display author name and credentials
|
||||
- Link to author profiles
|
||||
- Include author structured data
|
||||
|
||||
### Citations and Sources
|
||||
Linking to authoritative sources increases trust.
|
||||
|
||||
- Cite primary sources
|
||||
- Link to studies, documentation, official sources
|
||||
- Avoid circular citations (sites citing each other)
|
||||
|
||||
### Comprehensive Coverage
|
||||
AI prefers content that fully answers questions.
|
||||
|
||||
- Cover related questions users might have
|
||||
- Include definitions for technical terms
|
||||
- Address common misconceptions
|
||||
|
||||
## Google AI Overviews
|
||||
|
||||
Google's AI Overviews (formerly SGE) now appear in many search results. To optimize:
|
||||
|
||||
- **Be the cited source:** AI Overviews cite specific pages. Concise, authoritative answers increase citation likelihood.
|
||||
- **Structure for extraction:** Use clear headings, direct answers, and lists that AI can easily parse.
|
||||
- **Cover follow-up questions:** AI Overviews often address related queries. Anticipate and answer them on the same page or link to dedicated pages.
|
||||
- **Monitor in Search Console:** Google Search Console provides data on AI Overview impressions and clicks.
|
||||
|
||||
## AI Crawler Management
|
||||
|
||||
Make conscious decisions about which AI systems can crawl your content:
|
||||
|
||||
- **robots.txt directives:** Use `User-agent: GPTBot`, `ClaudeBot`, `PerplexityBot`, `Google-Extended` to control access.
|
||||
- **Allowing crawlers** increases chances of being cited as a source in AI responses.
|
||||
- **Blocking crawlers** prevents content from being used in AI training (but may reduce AI citations).
|
||||
- Review your policy regularly — this is one of the most actively evolving areas of SEO.
|
||||
|
||||
## Measuring AEO Success
|
||||
|
||||
### Monitor AI Mentions
|
||||
Track when AI assistants cite your content:
|
||||
- Use Google Search Console's AI Overview data for impression and click tracking
|
||||
- Monitor referral traffic from AI platforms (Perplexity, ChatGPT, Bing Copilot)
|
||||
- Search for your brand + "according to" in AI assistants
|
||||
- Consider third-party AEO tracking tools for comprehensive monitoring
|
||||
|
||||
### Track Zero-Click Queries
|
||||
If AI answers questions directly, traditional rankings matter less.
|
||||
|
||||
### Featured Snippet Capture
|
||||
Featured snippets often become AI answers. Track which you own.
|
||||
|
||||
## AEO vs SEO Balance
|
||||
|
||||
AEO and SEO largely align—quality content serves both. Key differences:
|
||||
|
||||
| Aspect | SEO Focus | AEO Focus |
|
||||
|--------|-----------|-----------|
|
||||
| Goal | Rank on page 1 | Be THE answer |
|
||||
| Format | Varies | Direct, structured |
|
||||
| Length | Often longer | Concise + comprehensive |
|
||||
| Links | Link building | Source citations |
|
||||
@@ -0,0 +1,127 @@
|
||||
# EEAT Principles
|
||||
|
||||
Google's EEAT framework (Experience, Expertise, Authoritativeness, Trustworthiness) guides how content quality is evaluated. This applies to both SEO rankings and AI answer selection.
|
||||
|
||||
## The Four Pillars
|
||||
|
||||
### Experience
|
||||
First-hand or life experience with the topic.
|
||||
|
||||
**Signals:**
|
||||
- Personal anecdotes and case studies
|
||||
- "I tested this" content
|
||||
- Real-world results and screenshots
|
||||
- User-generated reviews
|
||||
|
||||
**Implementation:**
|
||||
- Include author bios with relevant experience
|
||||
- Add "About the Author" sections
|
||||
- Feature customer testimonials
|
||||
- Show real examples, not just theory
|
||||
|
||||
### Expertise
|
||||
Knowledge and skill in the subject area.
|
||||
|
||||
**Signals:**
|
||||
- Credentials and qualifications
|
||||
- Depth of content coverage
|
||||
- Technical accuracy
|
||||
- Citations to authoritative sources
|
||||
|
||||
**Implementation:**
|
||||
- Display author credentials
|
||||
- Link to primary sources
|
||||
- Cover topics comprehensively
|
||||
- Keep content technically accurate and updated
|
||||
|
||||
### Authoritativeness
|
||||
Recognition as a go-to source in the field.
|
||||
|
||||
**Signals:**
|
||||
- Backlinks from respected sites
|
||||
- Mentions in industry publications
|
||||
- Social proof and follower counts
|
||||
- Brand recognition
|
||||
|
||||
**Implementation:**
|
||||
- Build thought leadership content
|
||||
- Contribute to industry publications
|
||||
- Maintain consistent publishing
|
||||
- Develop recognizable brand voice
|
||||
|
||||
### Trustworthiness
|
||||
Accuracy, transparency, and legitimacy.
|
||||
|
||||
**Signals:**
|
||||
- Clear authorship and contact info
|
||||
- Accurate, fact-checked content
|
||||
- Secure website (HTTPS)
|
||||
- Privacy policy and terms
|
||||
|
||||
**Implementation:**
|
||||
- Display clear author attribution
|
||||
- Include publication and update dates
|
||||
- Provide contact information
|
||||
- Use HTTPS and maintain security
|
||||
|
||||
## Sanity Implementation
|
||||
|
||||
```typescript
|
||||
// Author schema with EEAT signals
|
||||
defineType({
|
||||
name: 'author',
|
||||
type: 'document',
|
||||
fields: [
|
||||
defineField({ name: 'name', type: 'string' }),
|
||||
defineField({ name: 'role', type: 'string' }),
|
||||
defineField({ name: 'bio', type: 'text' }),
|
||||
defineField({ name: 'credentials', type: 'array', of: [{ type: 'string' }] }),
|
||||
defineField({ name: 'image', type: 'image' }),
|
||||
// sameAs: used for schema.org Person structured data output
|
||||
defineField({ name: 'sameAs', type: 'array', of: [{ type: 'url' }],
|
||||
description: 'Canonical profile URLs (LinkedIn, Twitter, etc.) for schema.org Person'
|
||||
}),
|
||||
// socialLinks: used for display purposes (platform icons, labels)
|
||||
defineField({
|
||||
name: 'socialLinks',
|
||||
type: 'array',
|
||||
of: [{ type: 'object', fields: [
|
||||
defineField({ name: 'platform', type: 'string' }),
|
||||
defineField({ name: 'url', type: 'url' })
|
||||
]}],
|
||||
description: 'Social links for display in the UI. Use sameAs for structured data output.'
|
||||
}),
|
||||
]
|
||||
})
|
||||
|
||||
// Content with EEAT metadata
|
||||
defineType({
|
||||
name: 'post',
|
||||
fields: [
|
||||
defineField({ name: 'author', type: 'reference', to: [{ type: 'author' }] }),
|
||||
defineField({ name: 'publishedAt', type: 'datetime' }),
|
||||
defineField({ name: 'updatedAt', type: 'datetime' }),
|
||||
defineField({
|
||||
name: 'reviewedBy',
|
||||
type: 'reference',
|
||||
to: [{ type: 'author' }],
|
||||
description: 'Expert reviewer for fact-checking'
|
||||
}),
|
||||
defineField({
|
||||
name: 'sources',
|
||||
type: 'array',
|
||||
of: [{ type: 'url' }],
|
||||
description: 'Citations and references'
|
||||
}),
|
||||
]
|
||||
})
|
||||
```
|
||||
|
||||
## YMYL Considerations
|
||||
|
||||
"Your Money or Your Life" topics (health, finance, legal, safety) require extra EEAT rigor:
|
||||
|
||||
- Medical content reviewed by healthcare professionals
|
||||
- Financial advice from certified experts
|
||||
- Legal content reviewed by attorneys
|
||||
- Clear disclaimers where appropriate
|
||||
@@ -0,0 +1,183 @@
|
||||
# Structured Data (JSON-LD)
|
||||
|
||||
Structured data helps search engines and AI understand your content. JSON-LD is the recommended format.
|
||||
|
||||
## Why Structured Data Matters
|
||||
|
||||
- **Rich snippets:** Enhanced search result appearance
|
||||
- **Knowledge panels:** Featured information boxes
|
||||
- **AI training:** Better content understanding
|
||||
- **Voice search:** Answer selection for voice queries
|
||||
|
||||
## Common Schema Types
|
||||
|
||||
### Article / Blog Post
|
||||
|
||||
```typescript
|
||||
import { Article, WithContext } from 'schema-dts'
|
||||
|
||||
const articleSchema: WithContext<Article> = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Article",
|
||||
headline: post.title,
|
||||
description: post.excerpt,
|
||||
image: post.image?.url,
|
||||
datePublished: post.publishedAt,
|
||||
dateModified: post.updatedAt,
|
||||
author: {
|
||||
"@type": "Person",
|
||||
name: post.author.name,
|
||||
url: post.author.url
|
||||
},
|
||||
publisher: {
|
||||
"@type": "Organization",
|
||||
name: "Your Company",
|
||||
logo: {
|
||||
"@type": "ImageObject",
|
||||
url: "https://example.com/logo.png"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### FAQ Page
|
||||
|
||||
```typescript
|
||||
import { FAQPage, WithContext } from 'schema-dts'
|
||||
|
||||
const faqSchema: WithContext<FAQPage> = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
mainEntity: faqs.map(faq => ({
|
||||
"@type": "Question",
|
||||
name: faq.question,
|
||||
acceptedAnswer: {
|
||||
"@type": "Answer",
|
||||
text: faq.answer // Plain text, use pt::text() in GROQ
|
||||
}
|
||||
}))
|
||||
}
|
||||
```
|
||||
|
||||
### Organization
|
||||
|
||||
```typescript
|
||||
import { Organization, WithContext } from 'schema-dts'
|
||||
|
||||
const orgSchema: WithContext<Organization> = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
name: "Your Company",
|
||||
url: "https://example.com",
|
||||
logo: "https://example.com/logo.png",
|
||||
sameAs: [
|
||||
"https://twitter.com/company",
|
||||
"https://linkedin.com/company/company"
|
||||
],
|
||||
contactPoint: {
|
||||
"@type": "ContactPoint",
|
||||
telephone: "+1-555-555-5555",
|
||||
contactType: "customer service"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Product
|
||||
|
||||
```typescript
|
||||
import { Product, WithContext } from 'schema-dts'
|
||||
|
||||
const productSchema: WithContext<Product> = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Product",
|
||||
name: product.name,
|
||||
description: product.description,
|
||||
image: product.images,
|
||||
offers: {
|
||||
"@type": "Offer",
|
||||
price: product.price,
|
||||
priceCurrency: "USD",
|
||||
availability: "https://schema.org/InStock"
|
||||
},
|
||||
aggregateRating: product.rating ? {
|
||||
"@type": "AggregateRating",
|
||||
ratingValue: product.rating.average,
|
||||
reviewCount: product.rating.count
|
||||
} : undefined
|
||||
}
|
||||
```
|
||||
|
||||
### Breadcrumb
|
||||
|
||||
```typescript
|
||||
import { BreadcrumbList, WithContext } from 'schema-dts'
|
||||
|
||||
const breadcrumbSchema: WithContext<BreadcrumbList> = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "BreadcrumbList",
|
||||
itemListElement: breadcrumbs.map((crumb, index) => ({
|
||||
"@type": "ListItem",
|
||||
position: index + 1, // schema.org positions are 1-based
|
||||
name: crumb.title,
|
||||
item: `https://example.com${crumb.path}`
|
||||
}))
|
||||
}
|
||||
```
|
||||
|
||||
## Combining Multiple Schemas (@graph)
|
||||
|
||||
Real-world pages often need multiple schema types. Use `@graph` to combine them. The `@context` is defined once at the top level — omit it from individual schema generators when used inside `@graph`:
|
||||
|
||||
```typescript
|
||||
const pageSchema = {
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
generateArticleSchema(post), // No @context needed here
|
||||
generateBreadcrumbSchema(breadcrumbs),
|
||||
generateOrganizationSchema(),
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation in Next.js
|
||||
|
||||
```typescript
|
||||
// Component to render JSON-LD
|
||||
// Ensure data comes from trusted sources (your CMS).
|
||||
// If data could contain user-generated content, strip HTML tags
|
||||
// and escape special characters before passing to JSON.stringify.
|
||||
function JsonLd({ data }: { data: WithContext<Thing> }) {
|
||||
return (
|
||||
<script
|
||||
type="application/ld+json"
|
||||
dangerouslySetInnerHTML={{ __html: JSON.stringify(data) }}
|
||||
/>
|
||||
)
|
||||
}
|
||||
|
||||
// Usage in page
|
||||
export default function PostPage({ post }) {
|
||||
return (
|
||||
<>
|
||||
<JsonLd data={generateArticleSchema(post)} />
|
||||
<article>...</article>
|
||||
</>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## GROQ for Plain Text
|
||||
|
||||
Structured data often needs plain text, not rich text:
|
||||
|
||||
```groq
|
||||
*[_type == "faq"]{
|
||||
question,
|
||||
"answer": pt::text(answerRichText) // Convert Portable Text to plain string
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Tools
|
||||
|
||||
- [Google Rich Results Test](https://search.google.com/test/rich-results)
|
||||
- [Schema.org Validator](https://validator.schema.org/)
|
||||
@@ -0,0 +1,188 @@
|
||||
# Technical SEO Checklist
|
||||
|
||||
Essential technical SEO elements for modern web applications.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- Metadata
|
||||
- Sitemaps
|
||||
- Canonical URLs
|
||||
- Redirects
|
||||
- Performance
|
||||
- Robots.txt
|
||||
- International SEO
|
||||
|
||||
## Metadata
|
||||
|
||||
### Title Tags
|
||||
- Unique per page
|
||||
- 50-60 characters
|
||||
- Primary keyword near the beginning
|
||||
- Brand name at the end (optional)
|
||||
|
||||
### Meta Descriptions
|
||||
- Unique per page
|
||||
- 150-160 characters
|
||||
- Include call-to-action
|
||||
- Contain relevant keywords
|
||||
|
||||
### Open Graph
|
||||
```html
|
||||
<meta property="og:title" content="Page Title" />
|
||||
<meta property="og:description" content="Description" />
|
||||
<meta property="og:image" content="https://example.com/image.jpg" />
|
||||
<meta property="og:url" content="https://example.com/page" />
|
||||
<meta property="og:type" content="article" />
|
||||
```
|
||||
|
||||
### Sanity + Next.js Implementation
|
||||
|
||||
```typescript
|
||||
export async function generateMetadata({ params }): Promise<Metadata> {
|
||||
const { data } = await sanityFetch({
|
||||
query: PAGE_QUERY,
|
||||
stega: false, // Critical: no stega in metadata
|
||||
})
|
||||
|
||||
return {
|
||||
title: data.seo?.title || data.title,
|
||||
description: data.seo?.description,
|
||||
openGraph: {
|
||||
images: data.seo?.image ? [{
|
||||
url: urlFor(data.seo.image).width(1200).height(630).url(),
|
||||
width: 1200,
|
||||
height: 630,
|
||||
}] : [],
|
||||
},
|
||||
robots: data.seo?.noIndex ? 'noindex' : undefined,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Sitemaps
|
||||
|
||||
Dynamic sitemap from CMS content:
|
||||
|
||||
```typescript
|
||||
// app/sitemap.ts
|
||||
import { MetadataRoute } from 'next'
|
||||
|
||||
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
|
||||
const pages = await client.fetch(`
|
||||
*[_type in ["page", "post"] && defined(slug.current) && seo.noIndex != true]{
|
||||
"url": select(
|
||||
_type == "page" => "/" + slug.current,
|
||||
_type == "post" => "/blog/" + slug.current
|
||||
),
|
||||
_updatedAt
|
||||
}
|
||||
`)
|
||||
|
||||
return pages.map(page => ({
|
||||
url: `https://example.com${page.url}`,
|
||||
lastModified: new Date(page._updatedAt),
|
||||
// Note: changeFrequency and priority are largely ignored by Google
|
||||
// but may be used by other search engines
|
||||
}))
|
||||
}
|
||||
```
|
||||
|
||||
## Canonical URLs
|
||||
|
||||
Prevent duplicate content issues:
|
||||
|
||||
```typescript
|
||||
export async function generateMetadata({ params }): Promise<Metadata> {
|
||||
return {
|
||||
alternates: {
|
||||
canonical: `https://example.com/${params.slug}`,
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Redirects
|
||||
|
||||
CMS-managed redirects:
|
||||
|
||||
```typescript
|
||||
// next.config.ts
|
||||
async redirects() {
|
||||
const redirects = await client.fetch(`
|
||||
*[_type == "redirect" && isEnabled == true]{
|
||||
source,
|
||||
destination,
|
||||
permanent
|
||||
}
|
||||
`)
|
||||
return redirects
|
||||
}
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
[Core Web Vitals](https://web.dev/articles/defining-core-web-vitals-thresholds) impact rankings:
|
||||
|
||||
- **LCP (Largest Contentful Paint):** < 2.5s
|
||||
- **INP (Interaction to Next Paint):** < 200ms
|
||||
- **CLS (Cumulative Layout Shift):** < 0.1
|
||||
|
||||
### Image Optimization (Next.js example)
|
||||
- Use `next/image` with Sanity URL builder
|
||||
- Serve WebP/AVIF formats
|
||||
- Implement LQIP blur placeholders
|
||||
- Set explicit dimensions
|
||||
|
||||
### Font Loading (Next.js example)
|
||||
```typescript
|
||||
// Prevent layout shift
|
||||
import { Inter } from 'next/font/google'
|
||||
const inter = Inter({ subsets: ['latin'], display: 'swap' })
|
||||
```
|
||||
|
||||
## Robots.txt
|
||||
|
||||
```
|
||||
# public/robots.txt
|
||||
User-agent: *
|
||||
Allow: /
|
||||
Disallow: /api/
|
||||
Disallow: /studio/
|
||||
|
||||
# AI crawlers — allow or block based on your content strategy
|
||||
# Uncomment to block specific AI crawlers:
|
||||
# User-agent: GPTBot
|
||||
# Disallow: /
|
||||
# User-agent: ClaudeBot
|
||||
# Disallow: /
|
||||
# User-agent: PerplexityBot
|
||||
# Disallow: /
|
||||
# User-agent: Google-Extended
|
||||
# Disallow: /
|
||||
|
||||
Sitemap: https://example.com/sitemap.xml
|
||||
```
|
||||
|
||||
**AI crawler considerations:** Decide whether AI training crawlers should access your content. Blocking `Google-Extended` prevents AI training use while still allowing Google Search indexing. Review your policy regularly as this landscape evolves.
|
||||
|
||||
## International SEO (hreflang)
|
||||
|
||||
For multi-language sites, implement hreflang tags to indicate language/region variants:
|
||||
|
||||
```typescript
|
||||
export async function generateMetadata({ params }: { params: Promise<{ lang: string; slug: string }> }): Promise<Metadata> {
|
||||
const { lang, slug } = await params
|
||||
return {
|
||||
alternates: {
|
||||
canonical: `https://example.com/${lang}/${slug}`,
|
||||
languages: {
|
||||
'en': `https://example.com/en/${slug}`,
|
||||
'de': `https://example.com/de/${slug}`,
|
||||
'x-default': `https://example.com/en/${slug}`,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Include all language variants in sitemaps with `hreflang` annotations for proper indexing.
|
||||
Reference in New Issue
Block a user