IxDF Developer Docs

Appearance

Maintenance Windows ​

Document state: draft

This document defines optimal maintenance windows for system operations that require minimal interference from scheduled cron jobs.

Overview ​

The IxDF platform runs a lot of scheduled cron jobs covering various domains. To perform system maintenance, or intensive operations safely, we need to identify periods with minimal cron job activity.

Maintenance Window Types ​

Daily (small: 1h): 01:00-01:59 UTC Weekly (big: 3h): Sunday 00:05-02:59 UTC

Characteristics:

  • Minimal cron job conflicts (only 1 job total on Sundays)
  • Minimum user activity (they are less active by weekends)

Considerations ​

Global User Base ​

  • The platform serves users across multiple timezones
  • "Quiet hours" vary by geographic region
  • Weekend patterns differ by culture and region

Critical Job Dependencies ​

  • Some cron jobs are time-sensitive (payment processing, notifications)
  • Maintenance windows should avoid critical business operations
  • Consider job dependencies and downstream effects

Timezone Coordination ​

  • All times should be specified in server timezone (likely UTC)
  • Document timezone clearly for operations team
  • Consider daylight saving time changes

Implementation Notes ​

When scheduling maintenance:

  1. Verify Current Jobs: Check active cron schedule before maintenance
  2. Monitor Dependencies: Identify any jobs that depend on systems being maintained
  3. Prepare Rollback: Ensure rollback procedures can complete within the window
  4. Communicate: Notify stakeholders of maintenance windows in advance
  5. Document: Log maintenance activities and any cron job impacts

Future Improvements ​

  • Automated monitoring of cron job execution times
  • Dynamic window adjustment based on actual job duration
  • Integration with deployment pipelines
  • Alerts for maintenance window conflicts

Activity Data Summary ​

Based on 4 years of course__quiz_answers.created_at data, the platform shows clear patterns:

Quietest Periods (Total Quiz Answers):

  1. Sunday 01:00 - 21,625 answers (absolute minimum)
  2. Saturday 01:00 - 22,059 answers
  3. Sunday 02:00 - 22,685 answers
  4. Saturday 02:00 - 22,931 answers
  5. Sunday 00:00 - 22,933 answers

Weekly Activity Pattern:

  • Sunday: Lowest activity day (peaks at ~70k vs 147k on weekdays)
  • Saturday: Second lowest (~62k peak)
  • Monday-Wednesday: Highest activity (~140-147k peak during business hours)
  • Thursday-Friday: Moderate to high activity

Daily Activity Pattern:

  • 00:00-05:00: Consistently lowest activity across all days
  • 13:00-15:00: Peak activity hours on weekdays
  • Weekend hours: Generally 50-70% lower activity than weekday equivalents

Last Updated: September 2025

Next Review: Annually or when significant user pattern changes are observed