23 KiB
23 KiB
Automation & Publishing Scheduling System
Last Updated: January 18, 2026
System: IGNY8 Celery Task Scheduling
Overview
IGNY8 uses Celery Beat to schedule two distinct but related systems:
- Automation Pipeline - Processes content through 7 stages (Keywords → Published)
- Publishing Scheduler - Schedules and publishes approved content to WordPress
┌─────────────────────────────────────────────────────────────────────────────┐
│ CELERY BEAT SCHEDULER │
│ (Persistent Schedule Store) │
└─────────────────────────────────────────────────────────────────────────────┘
│
┌─────────────────────────┼─────────────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Every Hour │ │ Every Hour │ │ Every 5 min │
│ at :05 │ │ at :00 │ │ │
└───────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ check_ │ │schedule_approved│ │process_scheduled│
│ scheduled_ │ │ _content │ │ _publications │
│ automations │ │ │ │ │
└───────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Automation │ │ Content gets │ │ Publishes to │
│ Pipeline │ │ scheduled_ │ │ WordPress via │
│ (7 stages) │ │ publish_at set │ │ API │
└───────────────┘ └─────────────────┘ └─────────────────┘
1. Automation Scheduling
Celery Task Configuration
| Task Name | Schedule | Purpose |
|---|---|---|
automation.check_scheduled_automations |
Every hour at :05 |
Check if any automation configs should run |
automation.check_test_triggers |
Every 5 minutes | Check for admin test triggers (exits early if none) |
How It Works
┌──────────────────────────────────────────────────────────────────────────┐
│ AUTOMATION SCHEDULING FLOW │
└──────────────────────────────────────────────────────────────────────────┘
Celery Beat AutomationConfig Result
│ │ │
│ (Every hour at :05) │ │
▼ │ │
┌─────────────┐ │ │
│ 02:05 check │ ─── Hour ────► scheduled_hour == 2? │
│ 03:05 check │ ─── Hour ────► scheduled_hour == 3? │
│ 04:05 check │ ─── Hour ────► scheduled_hour == 4? │
│ ... │ │ │
└─────────────┘ │ │
▼ │
┌────────────────┐ │
│ Match Found? │ │
└────────────────┘ │
│ │ │
Yes No │
│ │ │
▼ └─► Skip │
┌────────────────┐ │
│ Check Blocks: │ │
│ • 23hr block │ │
│ • Already │ │
│ running? │ │
└────────────────┘ │
│ │
Pass │
│ │
▼ │
┌────────────────┐ │
│ Start Run │ ────────────────────────► Run Started
│ • Set last_run │
│ • Queue task │
└────────────────┘
Hourly Matching Logic
The scheduler runs at :05 of every hour and checks if scheduled_hour == current_hour:
| Celery Runs At | Checks Hour | Example Config |
|---|---|---|
02:05 |
2 |
scheduled_time = 02:00 → Matches |
03:05 |
3 |
scheduled_time = 03:00 → Matches |
14:05 |
14 |
scheduled_time = 14:00 (2 PM) → Matches |
Note: Users select hour only (1-12 with AM/PM in UI), stored as HH:00 format. The :05 offset ensures the check happens after the configured hour begins.
Test Mode (Admin Feature)
Admins can trigger automations without waiting for the hourly schedule:
┌──────────────────────────────────────────────────────────────────────────┐
│ TEST TRIGGER FLOW │
└──────────────────────────────────────────────────────────────────────────┘
Admin Action Celery (Every Minute) Result
│ │ │
▼ │ │
┌─────────────────┐ │ │
│ Enable test mode│ │ │
│ Set trigger time│ │ │
│ (or "now") │ │ │
└─────────────────┘ │ │
│ │ │
└───────────────────────────────────▼ │
┌─────────────────┐ │
│check_test_ │ │
│triggers task │ │
└─────────────────┘ │
│ │
▼ │
┌─────────────────┐ │
│test_trigger_at │ │
│<= now? │ │
└─────────────────┘ │
│ │ │
Yes No │
│ └─► Wait │
▼ │
┌─────────────────┐ │
│Start test run │──────────────► Run Started
│trigger_type= │ (type='test')
│'test' │
│Clear trigger_at │
└─────────────────┘
Test mode fields:
test_mode_enabled- Enable test functionality for this configtest_trigger_at- When to trigger (set tonowfor immediate)
Admin actions:
- 🧪 Trigger test run - Sets both fields and triggers at next minute
- 🧹 Clear test mode - Disables test mode and clears trigger
Frequency Rules
| Frequency | When It Runs | Additional Condition |
|---|---|---|
daily |
Every day | scheduled_hour == current_hour |
weekly |
Mondays only | weekday() == 0 + hour match |
monthly |
1st of month only | day == 1 + hour match |
Duplicate Prevention (23-Hour Block)
After a successful run, the config won't run again for 23 hours:
if config.last_run_at:
time_since_last_run = now - config.last_run_at
if time_since_last_run < timedelta(hours=23):
# SKIP - already ran recently
Schedule Change Behavior
When scheduled_time, frequency, or is_enabled changes:
last_run_atis reset toNonenext_run_atis recalculated- Automation becomes eligible immediately at the next hourly check
Example:
Current time: 12:30
You change scheduled_time from 02:00 → 12:00 (12 PM)
Hour 12 was already checked at 12:05
→ Automation will run tomorrow at 12:05 (daily) or next valid day (weekly/monthly)
Pro tip: Use test mode to trigger immediately after schedule changes.
2. Publishing Scheduling
Celery Task Configuration
| Task Name | Schedule | Purpose |
|---|---|---|
publishing.schedule_approved_content |
Every hour at :00 |
Schedule approved content for future publishing |
publishing.process_scheduled_publications |
Every 5 min | Publish content where scheduled_publish_at <= now |
Publishing Flow
┌──────────────────────────────────────────────────────────────────────────┐
│ PUBLISHING SCHEDULING FLOW │
└──────────────────────────────────────────────────────────────────────────┘
EVERY HOUR (:00) EVERY 5 MINUTES
│ │
▼ ▼
┌────────────────────────┐ ┌────────────────────────┐
│schedule_approved_content│ │process_scheduled_ │
│ │ │publications │
└────────────────────────┘ └────────────────────────┘
│ │
▼ │
┌────────────────────────┐ │
│ For each Site with │ │
│ auto_publish_enabled: │ │
│ │ │
│ 1. Find approved │ │
│ content │ │
│ 2. Calculate slots │ │
│ based on settings │ │
│ 3. Set scheduled_ │ │
│ publish_at │ │
│ 4. Set site_status= │ │
│ 'scheduled' │ │
└────────────────────────┘ │
│ │
▼ ▼
┌───────────────────────────────────────────────┐
│ DATABASE │
│ ┌─────────────────────────────────────────┐ │
│ │ Content Table │ │
│ │ • status = 'approved' │ │
│ │ • site_status = 'scheduled' │ │
│ │ • scheduled_publish_at = <datetime> │ │
│ └─────────────────────────────────────────┘ │
└───────────────────────────────────────────────┘
│
▼
┌────────────────────────┐
│ process_scheduled_ │
│ publications │
│ │
│ WHERE: │
│ • site_status = │
│ 'scheduled' │
│ • scheduled_publish_at │
│ <= NOW │
└────────────────────────┘
│
▼
┌────────────────────────┐
│ Publish to WordPress │
│ via PublisherService │
│ │
│ On Success: │
│ • site_status = │
│ 'published' │
│ • Set wp_post_id │
└────────────────────────┘
Scheduling Modes
| Mode | Behavior |
|---|---|
immediate |
Content scheduled for now, picked up within 5 minutes |
time_slots |
Content scheduled at specific times (e.g., 9am, 2pm, 6pm) |
stagger |
Content spread evenly across publish hours |
Publishing Limits
| Limit | Description |
|---|---|
daily_publish_limit |
Max posts per day |
weekly_publish_limit |
Max posts per week |
monthly_publish_limit |
Max posts per month |
queue_limit |
Max items to schedule at once (default: 100) |
3. System Relationship
┌─────────────────────────────────────────────────────────────────────────────┐
│ COMPLETE CONTENT LIFECYCLE │
└─────────────────────────────────────────────────────────────────────────────┘
AUTOMATION PIPELINE PUBLISHING PIPELINE
(Every 15 min check) (Hourly + Every 5 min)
│ │
▼ │
┌─────────────────┐ │
│ Stage 1-6 │ │
│ Keywords → │ │
│ Content Created │ │
└─────────────────┘ │
│ │
▼ │
┌─────────────────┐ │
│ Stage 7 │ │
│ Auto-Approval │ status='approved' │
│ (if enabled) │ ─────────────────────────────────► │
└─────────────────┘ │
▼
┌─────────────────────┐
│schedule_approved_ │
│content (hourly) │
│ │
│ Sets: │
│ • scheduled_publish │
│ _at │
│ • site_status = │
│ 'scheduled' │
└─────────────────────┘
│
▼
┌─────────────────────┐
│process_scheduled_ │
│publications (5 min) │
│ │
│ Publishes to WP │
│ • site_status = │
│ 'published' │
└─────────────────────┘
4. Celery Beat Schedule Summary
| Task | Schedule | Crontab |
|---|---|---|
automation.check_scheduled_automations |
Every 15 min | minute='0,15,30,45' |
publishing.schedule_approved_content |
Hourly | minute=0 |
publishing.process_scheduled_publications |
Every 5 min | minute='*/5' |
5. Key Configuration Tables
AutomationConfig
| Field | Type | Purpose |
|---|---|---|
is_enabled |
Boolean | Enable/disable scheduling |
frequency |
Choice | daily, weekly, monthly |
scheduled_time |
Time | When to run (e.g., 02:00) |
last_run_at |
DateTime | Last successful run (23hr block) |
next_run_at |
DateTime | Calculated next run time |
PublishingSettings
| Field | Type | Purpose |
|---|---|---|
auto_publish_enabled |
Boolean | Enable auto-scheduling |
scheduling_mode |
Choice | immediate, time_slots, stagger |
publish_days |
Array | ['mon', 'tue', ...] |
publish_time_slots |
Array | ['09:00', '14:00', '18:00'] |
daily_publish_limit |
Integer | Max posts/day |
6. Troubleshooting
Automation Didn't Run
| Check | Solution |
|---|---|
is_enabled = False |
Enable the automation |
| Time not in current window | Wait for next window or change time |
last_run_at within 23hrs |
Wait for 23hr block to expire |
| Another run in progress | Wait for current run to complete |
| Config updated after window | Schedule moved to next occurrence |
Content Not Publishing
| Check | Solution |
|---|---|
auto_publish_enabled = False |
Enable in PublishingSettings |
Content status != 'approved' |
Approve content first |
No wp_api_key on Site |
Configure WordPress integration |
| Daily/weekly limit reached | Wait for limit reset or increase |
7. Logs
Monitor these logs for scheduling issues:
# Automation scheduling
docker logs igny8_celery_worker 2>&1 | grep "AutomationTask"
# Publishing scheduling
docker logs igny8_celery_worker 2>&1 | grep "schedule_approved_content\|process_scheduled"