Skip to content

Commit 22dfd46

Browse files
committed
fix(seo): correct OG image URL, fix homepage title, and add AI-friendly robots.txt
- Change og:image to absolute URL (https://33jsconcepts.com/og-image.png) - Add thumbnails config and article:author to docs.json - Fix homepage title from '33 JavaScript Concepts' to 'Learn JavaScript' to prevent duplication - Create robots.txt allowing AI search engine bots (GPTBot, ClaudeBot, PerplexityBot)
1 parent 64b1159 commit 22dfd46

3 files changed

Lines changed: 59 additions & 2 deletions

File tree

docs/docs.json

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,17 +13,21 @@
1313
"dark": "#C9B83C"
1414
},
1515
"favicon": "/favicon.ico",
16+
"thumbnails": {
17+
"appearance": "dark"
18+
},
1619
"seo": {
1720
"indexing": "navigable",
1821
"metatags": {
1922
"og:type": "website",
2023
"og:site_name": "33 JavaScript Concepts",
2124
"og:locale": "en_US",
22-
"og:image": "/og-image.png",
25+
"og:image": "https://33jsconcepts.com/og-image.png",
2326
"twitter:card": "summary_large_image",
2427
"twitter:site": "@leonardomso",
2528
"twitter:creator": "@leonardomso",
2629
"twitter:image": "/og-image.png",
30+
"article:author": "Leonardo Maldonado",
2731
"author": "Leonardo Maldonado",
2832
"keywords": "JavaScript, JS, learn JavaScript, JavaScript tutorial, JavaScript concepts, JavaScript fundamentals, web development, programming, coding, closures, promises, async await, event loop, DOM, prototypes",
2933
"language": "en",

docs/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "33 JavaScript Concepts"
2+
title: "Learn JavaScript"
33
sidebarTitle: "Welcome"
44
description: "Master JavaScript with 33 core concepts. Clear explanations, practical examples, and curated resources for developers at any level."
55
---

docs/robots.txt

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Belt-and-suspenders robots.txt for 33jsconcepts.com
2+
#
3+
# PRIMARY FIX: Cloudflare dashboard configuration is required to override
4+
# Cloudflare-managed AI bot blocking. This file serves as the origin-level
5+
# directive and fallback if Cloudflare managed robots is disabled.
6+
#
7+
# Cloudflare Dashboard Paths:
8+
# - Security -> Settings -> Bot traffic -> robots.txt setting
9+
# - AI Crawl Control -> Crawlers (allow/block per crawler)
10+
# - AI Crawl Control -> Robots.txt tab (check managed status)
11+
#
12+
# This file allows ALL major search engines and AI crawlers to index content.
13+
14+
# Default rule: Allow all bots to crawl everything except Cloudflare internal paths
15+
User-agent: *
16+
Allow: /
17+
Disallow: /cdn-cgi/
18+
19+
# Explicit allow directives for major search engine bots
20+
User-agent: Googlebot
21+
Allow: /
22+
23+
User-agent: Bingbot
24+
Allow: /
25+
26+
User-agent: Yandexbot
27+
Allow: /
28+
29+
User-agent: DuckDuckBot
30+
Allow: /
31+
32+
# Explicit allow directives for AI search engine bots
33+
# These override any Cloudflare-managed blocks at the origin level
34+
User-agent: GPTBot
35+
Allow: /
36+
37+
User-agent: ChatGPT-User
38+
Allow: /
39+
40+
User-agent: ClaudeBot
41+
Allow: /
42+
43+
User-agent: anthropic-ai
44+
Allow: /
45+
46+
User-agent: PerplexityBot
47+
Allow: /
48+
49+
User-agent: Google-Extended
50+
Allow: /
51+
52+
# Sitemap for search engine discovery
53+
Sitemap: https://33jsconcepts.com/sitemap.xml

0 commit comments

Comments
 (0)