Future Modules planning docs
This commit is contained in:
IGNY8 VPS (Salman)
505
gsc_integratin.md
Normal file
505
gsc_integratin.md
Normal file
@@ -0,0 +1,505 @@
|
||||
# Google Search Console Integration
|
||||
## Complete Implementation Plan for IGNY8
|
||||
|
||||
**Version:** 1.0 | **January 2026**
|
||||
|
||||
---
|
||||
|
||||
# 1. EXECUTIVE SUMMARY
|
||||
|
||||
## 1.1 What We're Building
|
||||
|
||||
| Capability | Description |
|
||||
|------------|-------------|
|
||||
| Connect Once | Single Google account connection manages all sites |
|
||||
| Auto-Index | Every IGNY8 article automatically submitted to Google |
|
||||
| Monitor | See which pages are indexed, pending, or have issues |
|
||||
| Performance | View clicks, impressions, keywords in IGNY8 |
|
||||
| Manual Control | Inspect and request indexing for any URL |
|
||||
|
||||
## 1.2 Problem vs Solution
|
||||
|
||||
**BEFORE:** Content published → Wait for Google → Maybe indexed in weeks/months → 43% NOT indexed after 1 year
|
||||
|
||||
**AFTER:** Content published → Auto-inspect → Auto-request indexing → Status synced to WordPress → 90%+ indexed in 2-4 weeks
|
||||
|
||||
## 1.3 API Choice: URL Inspection API
|
||||
|
||||
| Aspect | URL Inspection API (WE USE) | Indexing API (NOT USING) |
|
||||
|--------|----------------------------|--------------------------|
|
||||
| Restrictions | NONE - any URL | JobPosting/Events only |
|
||||
| Daily Quota | ~2,000/day | ~200/day |
|
||||
| Risk Level | Zero | High |
|
||||
|
||||
## 1.4 Integration Model
|
||||
|
||||
```
|
||||
IGNY8 APP (Central Hub)
|
||||
┌─────────────────────────────────────┐
|
||||
│ Google OAuth Token (ONE time) │
|
||||
└─────────────────┬───────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Site A │ │ Site B │ │ Site C │
|
||||
│(Plugin) │ │(Plugin) │ │(Plugin) │
|
||||
└─────────┘ └─────────┘ └─────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 2. ARCHITECTURE
|
||||
|
||||
## 2.1 System Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ IGNY8 APP │
|
||||
│ │
|
||||
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
|
||||
│ │GSC Service │ │Queue Serv. │ │Metrics Srv │ │
|
||||
│ │• OAuth │ │• URL queue │ │• Fetch data│ │
|
||||
│ │• Tokens │ │• Rate limit│ │• Cache │ │
|
||||
│ │• Mapping │ │• Status │ │• Filter │ │
|
||||
│ └─────┬──────┘ └─────┬──────┘ └─────┬──────┘ │
|
||||
│ └──────────────┼──────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌───────────────────────┐ │
|
||||
│ │ Background Workers │ │
|
||||
│ │ (Celery) │ │
|
||||
│ │ • Process queue │ │
|
||||
│ │ • Re-inspect URLs │ │
|
||||
│ │ • Sync to plugins │ │
|
||||
│ └───────────┬───────────┘ │
|
||||
└───────────────────────┼──────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┴─────────────┐
|
||||
▼ ▼
|
||||
┌──────────────────────┐ ┌──────────────────────┐
|
||||
│ GOOGLE APIS │ │ WORDPRESS SITES │
|
||||
│ • URL Inspection API │ │ • IGNY8 Plugin │
|
||||
│ • Search Analytics │ │ - Receive status │
|
||||
│ • Sites API │ │ - Display in WP │
|
||||
└──────────────────────┘ └──────────────────────┘
|
||||
```
|
||||
|
||||
## 2.2 Data Flow
|
||||
|
||||
```
|
||||
User connects Google → OAuth tokens stored → Fetch GSC sites → Map to IGNY8 sites
|
||||
│
|
||||
┌───────────────────────────────────────────┼───────────────────┐
|
||||
▼ ▼ ▼
|
||||
AUTO-INDEXING MANUAL INDEXING METRICS
|
||||
IGNY8 publishes User selects URLs User views
|
||||
│ │ performance
|
||||
▼ ▼ │
|
||||
Add to queue (P:100) Add to queue (P:50) ▼
|
||||
│ │ Fetch API
|
||||
└───────────────────┬───────────────────────┘ Cache
|
||||
▼
|
||||
Worker inspects
|
||||
│
|
||||
NOT INDEXED? → Request indexing → Schedule re-inspect
|
||||
│
|
||||
▼
|
||||
Sync to WordPress plugin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 3. FEATURES
|
||||
|
||||
## 3.1 Feature List
|
||||
|
||||
| # | Feature | Priority |
|
||||
|---|---------|----------|
|
||||
| 1 | GSC OAuth Connection | Critical |
|
||||
| 2 | Site Mapping | Critical |
|
||||
| 3 | URL Inspection | Critical |
|
||||
| 4 | Indexing Requests | Critical |
|
||||
| 5 | Auto-Indexing | Critical |
|
||||
| 6 | Queue Management | Critical |
|
||||
| 7 | Plugin Status Sync | High |
|
||||
| 8 | Search Metrics | High |
|
||||
| 9 | Manual URL Management | Medium |
|
||||
|
||||
## 3.2 OAuth Connection
|
||||
|
||||
**Flow:** User clicks Connect → Google OAuth → User allows → Tokens stored → Sites fetched
|
||||
|
||||
**Scopes:** `webmasters.readonly`, `webmasters`
|
||||
|
||||
**Data Stored:** user_id, google_email, access_token (encrypted), refresh_token (encrypted), token_expiry, status, connected_at
|
||||
|
||||
## 3.3 Site Mapping
|
||||
|
||||
**Logic:** For each IGNY8 site, check GSC for:
|
||||
1. sc-domain:example.com (Domain property) ← Preferred
|
||||
2. https://example.com (URL prefix)
|
||||
3. http://example.com (URL prefix)
|
||||
|
||||
**States:** matched ✓, manual 🔧, not_found ✗
|
||||
|
||||
## 3.4 URL Inspection & Indexing
|
||||
|
||||
**Decision Flow:**
|
||||
```
|
||||
Inspect URL → Parse Response
|
||||
│
|
||||
┌───────────┼───────────┐
|
||||
▼ ▼ ▼
|
||||
INDEXED NOT INDEXED ERROR
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
Done! Request indexing Show details
|
||||
│
|
||||
▼
|
||||
Schedule re-inspect (24-48 hrs)
|
||||
```
|
||||
|
||||
## 3.5 Auto-Indexing Pipeline
|
||||
|
||||
```
|
||||
IGNY8 publishes content
|
||||
│
|
||||
▼
|
||||
Create URL tracking record
|
||||
│
|
||||
▼
|
||||
Add to queue (Priority: 100)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ BACKGROUND WORKER │
|
||||
│ │
|
||||
│ Pick item → Check quota │
|
||||
│ │ │
|
||||
│ YES ─┴─ NO │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ Process Skip (retry tomorrow) │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Call URL Inspection API │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ NOT INDEXED? → Request indexing │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Sync to WordPress → Wait 3s → Next │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 3.6 Queue Management
|
||||
|
||||
**Rules:**
|
||||
1. One at a time (3 sec between calls)
|
||||
2. Priority order: 100 (IGNY8 auto) → 90 → 70 → 50 → 30 (re-inspect)
|
||||
3. Quota: 2,000/day, reset midnight PT
|
||||
4. Quota exhausted: items stay queued, resume after reset
|
||||
|
||||
## 3.7 Plugin Status Sync
|
||||
|
||||
**Flow:** Status changes → Batch collector (5 min) → POST to plugin → Store in WP DB → Display in admin
|
||||
|
||||
## 3.8 Search Metrics
|
||||
|
||||
**Metrics:** Clicks, Impressions, CTR, Position
|
||||
|
||||
**Dimensions:** Pages, Keywords, Countries, Devices
|
||||
|
||||
**Special:** Filter to show ONLY IGNY8 content performance
|
||||
|
||||
---
|
||||
|
||||
# 4. USER EXPERIENCE
|
||||
|
||||
## 4.1 First-Time Setup
|
||||
|
||||
1. **Discovery:** New "Search Console" menu in IGNY8
|
||||
2. **Welcome:** Benefits explained, "Connect" button
|
||||
3. **OAuth:** User selects Google account, grants permission
|
||||
4. **Mapping:** Shows matched/unmatched sites
|
||||
5. **Complete:** Ready to use
|
||||
|
||||
## 4.2 Indexing Dashboard
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Site: [example.com ▼] │
|
||||
│ │
|
||||
│ ┌─────────┬─────────┬─────────┬─────────┐ │
|
||||
│ │ TOTAL │ INDEXED │ PENDING │ ISSUES │ │
|
||||
│ │ 53 │ 30 │ 18 │ 5 │ │
|
||||
│ └─────────┴─────────┴─────────┴─────────┘ │
|
||||
│ │
|
||||
│ [IGNY8 Content] [All Site URLs] [Custom URL] │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────────────────┐ │
|
||||
│ │ □ TITLE/URL PUBLISHED STATUS ACTION │ │
|
||||
│ ├───────────────────────────────────────────────────┤ │
|
||||
│ │ □ Best SEO Practices Jan 15 ✓ Indexed │ │
|
||||
│ │ □ Blue Widget Pro Jan 14 ⏳ Pending │ │
|
||||
│ │ □ Content Marketing Jan 12 ➡ Requested │ │
|
||||
│ │ □ Red Widget Jan 10 ⚠ Noindex │ │
|
||||
│ └───────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Queue: 3 processing • Quota: 1,847/2,000 remaining │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 4.3 Performance Metrics
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Site: [example.com ▼] Date: [Last 28 days ▼] │
|
||||
│ │
|
||||
│ ┌───────────┬───────────┬───────────┬───────────┐ │
|
||||
│ │ CLICKS │ IMPR │ CTR │ POSITION │ │
|
||||
│ │ 1,234 │ 45,678 │ 2.7% │ 12.4 │ │
|
||||
│ │ ▲ +15% │ ▲ +8% │ ▲ +0.3% │ ▼ -2.1 │ │
|
||||
│ └───────────┴───────────┴───────────┴───────────┘ │
|
||||
│ │
|
||||
│ [Top Pages] [Top Keywords] [IGNY8 Content ★] │
|
||||
│ │
|
||||
│ □ Show only IGNY8 content │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────────────────────┐ │
|
||||
│ │ PAGE CLICKS IMPR CTR POSITION│ │
|
||||
│ │ ★ /blog/best-seo 342 12,456 2.7% 8.2 │ │
|
||||
│ │ ★ /product/widget 156 8,234 1.9% 15.4 │ │
|
||||
│ │ /about-us 89 5,678 1.6% 22.1 │ │
|
||||
│ └───────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ★ = IGNY8 content [Export to CSV] │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 4.4 WordPress Plugin Display
|
||||
|
||||
**Content List Column:**
|
||||
```
|
||||
□ TITLE DATE STATUS INDEX STATUS
|
||||
□ Best SEO Tips Jan 15 Published ✓ Indexed
|
||||
□ Widget Guide Jan 14 Published ⏳ Pending
|
||||
□ Marketing 101 Jan 12 Published ➡ Requested
|
||||
```
|
||||
|
||||
**Edit Screen Metabox:**
|
||||
```
|
||||
┌─────────────────────────────┐
|
||||
│ Search Console Status │
|
||||
│ Index Status: ✓ Indexed │
|
||||
│ Last Checked: 2 hours ago │
|
||||
│ ℹ Managed via IGNY8 │
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 5. DATA ARCHITECTURE
|
||||
|
||||
## 5.1 Database Schema
|
||||
|
||||
```
|
||||
USER (existing)
|
||||
│
|
||||
└──► GSC_CONNECTION (1:1)
|
||||
│ • google_email
|
||||
│ • access_token (encrypted)
|
||||
│ • refresh_token (encrypted)
|
||||
│ • status
|
||||
│
|
||||
└──► DAILY_QUOTA (1:N)
|
||||
• date
|
||||
• inspections_used
|
||||
• inspections_limit
|
||||
|
||||
SITE (existing)
|
||||
│
|
||||
└──► GSC_SITE_MAPPING (1:1)
|
||||
│ • gsc_property
|
||||
│ • mapping_type
|
||||
│ • status
|
||||
│
|
||||
└──► URL_INDEXING_RECORD (1:N)
|
||||
│ • url
|
||||
│ • source
|
||||
│ • status
|
||||
│ • last_inspection_result
|
||||
│
|
||||
└──► INDEXING_QUEUE (1:N)
|
||||
• queue_type
|
||||
• priority
|
||||
• status
|
||||
|
||||
PUBLISHED_CONTENT (existing)
|
||||
│
|
||||
└──► URL_INDEXING_RECORD (1:1)
|
||||
|
||||
METRICS_CACHE (linked to GSC_SITE_MAPPING)
|
||||
• metric_type
|
||||
• date_range
|
||||
• data
|
||||
• expires_at
|
||||
```
|
||||
|
||||
## 5.2 Status Definitions
|
||||
|
||||
| Status | Icon | Meaning |
|
||||
|--------|------|---------|
|
||||
| pending_inspection | ⏳ | In queue |
|
||||
| indexed | ✓ | In Google index |
|
||||
| not_indexed | ✗ | Not in index |
|
||||
| indexing_requested | ➡ | Request sent |
|
||||
| error_noindex | 🚫 | Has noindex tag |
|
||||
| error_blocked | 🚫 | Robots.txt blocks |
|
||||
| error_not_found | 🚫 | 404 error |
|
||||
|
||||
---
|
||||
|
||||
# 6. PROCESSING LOGIC
|
||||
|
||||
## 6.1 Queue Priority
|
||||
|
||||
| Priority | Type |
|
||||
|----------|------|
|
||||
| 100 | IGNY8 auto-inspection |
|
||||
| 90 | IGNY8 indexing request |
|
||||
| 70 | Manual indexing request |
|
||||
| 50 | Manual inspection |
|
||||
| 30 | Scheduled re-inspection |
|
||||
|
||||
## 6.2 Re-Inspection Schedule
|
||||
|
||||
| Check # | Delay | Action if not indexed |
|
||||
|---------|-------|----------------------|
|
||||
| 1 | 24 hours | Continue |
|
||||
| 2 | Day 3 | Continue |
|
||||
| 3 | Day 6 | Continue |
|
||||
| 4 | Day 13 | Final check |
|
||||
| 5 | STOP | Mark for manual review |
|
||||
|
||||
## 6.3 Quota Management
|
||||
|
||||
- Daily Limit: 2,000 inspections
|
||||
- Reset: Midnight PT
|
||||
- Exhausted: Items stay queued, resume after reset, user notified
|
||||
|
||||
---
|
||||
|
||||
# 7. API INTEGRATION
|
||||
|
||||
## 7.1 Google OAuth
|
||||
|
||||
- Redirect URI: `https://app.igny8.com/auth/google/callback`
|
||||
- Scopes: `webmasters.readonly`, `webmasters`
|
||||
|
||||
## 7.2 URL Inspection API
|
||||
|
||||
```
|
||||
POST https://searchconsole.googleapis.com/v1/urlInspection/index:inspect
|
||||
|
||||
Request:
|
||||
{
|
||||
"inspectionUrl": "https://example.com/page",
|
||||
"siteUrl": "sc-domain:example.com"
|
||||
}
|
||||
|
||||
Response fields: verdict, coverageState, robotsTxtState, indexingState, lastCrawlTime
|
||||
```
|
||||
|
||||
## 7.3 Search Analytics API
|
||||
|
||||
```
|
||||
POST https://searchconsole.googleapis.com/webmasters/v3/sites/{siteUrl}/searchAnalytics/query
|
||||
|
||||
Request:
|
||||
{
|
||||
"startDate": "2025-01-01",
|
||||
"endDate": "2025-01-28",
|
||||
"dimensions": ["page", "query"],
|
||||
"rowLimit": 1000
|
||||
}
|
||||
|
||||
Response: clicks, impressions, ctr, position
|
||||
```
|
||||
|
||||
## 7.4 Sites API
|
||||
|
||||
```
|
||||
GET https://searchconsole.googleapis.com/webmasters/v3/sites
|
||||
|
||||
Response: List of sites with siteUrl and permissionLevel
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 8. WORDPRESS PLUGIN UPDATES
|
||||
|
||||
## 8.1 New Components
|
||||
|
||||
- **Endpoint:** `POST /wp-json/igny8/v1/index-status-sync`
|
||||
- **Table:** url, post_id, status, last_checked, details
|
||||
- **UI:** Content list column, Edit screen metabox, Dashboard widget
|
||||
|
||||
---
|
||||
|
||||
# 9. ERROR HANDLING
|
||||
|
||||
| Category | Error | Action |
|
||||
|----------|-------|--------|
|
||||
| Connection | OAuth rejected | Retry |
|
||||
| Connection | Token expired | Reconnect |
|
||||
| Mapping | Site not in GSC | Link to GSC |
|
||||
| Indexing | Quota exhausted | Queue for tomorrow |
|
||||
| Indexing | noindex tag | Show warning |
|
||||
| Sync | Plugin unreachable | Retry later |
|
||||
|
||||
---
|
||||
|
||||
# 10. IMPLEMENTATION PHASES
|
||||
|
||||
| Phase | Weeks | Focus |
|
||||
|-------|-------|-------|
|
||||
| 1 | 1-2 | Core Connection (OAuth, site mapping) |
|
||||
| 2 | 3-4 | URL Inspection (queue, status tracking) |
|
||||
| 3 | 5-6 | Auto-Indexing (hooks, re-inspection) |
|
||||
| 4 | 7 | Plugin Status Sync |
|
||||
| 5 | 8-9 | Performance Metrics |
|
||||
| 6 | 10 | Polish & Optimization |
|
||||
|
||||
---
|
||||
|
||||
# 11. SUCCESS METRICS
|
||||
|
||||
| Metric | Target |
|
||||
|--------|--------|
|
||||
| GSC connection rate | 50% of active users |
|
||||
| Sites mapped | 80% of user sites |
|
||||
| Auto-indexing enabled | 70% of connected users |
|
||||
| Indexing success rate | >80% within 7 days |
|
||||
| Sync reliability | >99% |
|
||||
| API error rate | <1% |
|
||||
|
||||
---
|
||||
|
||||
# 12. GLOSSARY
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| GSC | Google Search Console |
|
||||
| URL Inspection | Checking URL status in Google's index |
|
||||
| Indexing Request | Asking Google to index a URL |
|
||||
| Impressions | Times page appeared in search |
|
||||
| Clicks | Times users clicked from search |
|
||||
| CTR | Click-through rate |
|
||||
| Position | Average ranking in search |
|
||||
| sc-domain | Domain property type in GSC |
|
||||
| Quota | Daily API usage limits |
|
||||
|
||||
---
|
||||
|
||||
**END OF DOCUMENT**
|
||||
Reference in New Issue
Block a user