Daily Maintenance Cron

This page describes the /api/cron/daily-maintenance pipeline and explains why each step is implemented this way.

Trigger and schedule

The job is triggered by Vercel Cron:

• Path: /api/cron/daily-maintenance
• Schedule: 0 3 * * * (once per day)

In production, the route requires CRON_SECRET (Bearer token). In non-production, it allows local/manual invocation without the secret.

Design goals

The cron pipeline optimizes for:

1. Correct core state first: key revocation must be durable and idempotent.
2. Best-effort side effects: denormalized counts and cache invalidation should not block the whole run.
3. Recoverability: failed side effects are persisted and retried later.
4. Low operational overhead: one daily cron can handle all periodic work on Vercel Free/Hobby.
5. Upgrade-friendly design: usage flush is exposed separately so higher-frequency schedules can be added later.

For low-frequency environments, operators can also trigger a protected manual usage flush from the dashboard without changing the scheduled architecture.

End-to-end pipeline

Step 1: Process retry queue first

Before running new maintenance work, the cron processes due entries in maintenance_retry_task.

Each retry task contains:

• taskType: what operation to retry
  • sync_project_api_key_count
  • invalidate_api_key_cache
• taskKey: target identifier (projectId or publicKey)
• attempts, maxAttempts
• nextRunAt, lastError

Why run retries first

• Prevents old failures from being starved by new work.
• Helps keep denormalized/cached state converged over time.
• Keeps behavior deterministic for operators: "old debt first, then fresh work."

Retry behavior

• Due tasks (nextRunAt <= now and attempts < maxAttempts) are processed in batches.
• Success removes the task from the queue.
• Failure increments attempts, stores lastError, and schedules next attempt with exponential backoff.
• Exhausted tasks (attempts >= maxAttempts) stay recorded and stop auto-running until re-enqueued by a new failure event.

Step 2A: Usage buffer flush

flushUsageBufferToDatabase() moves Redis minute buckets into usage_record.

Why this runs in daily cron

• Supports free-tier environments that can only schedule one cron job.
• Keeps authoritative usage metrics (usage_record) up to date without requiring a second scheduler.
• Still keeps the request path non-blocking by buffering writes in Redis first.

Step 2B: Request log cleanup

cleanupOldRequestLogs() deletes logs older than retention and returns deletedCount.

Why return deleted count

• Gives an immediate operational signal ("did cleanup do work this run?").
• Makes cron output more useful for monitoring and debugging.

Step 2C: Expired API key sweep

revokeExpiredApiKeys() executes:

1. Select active keys where expiresAt <= now and revokedAt IS NULL.
2. Bulk update them with revokedAt = now.
3. Recompute project.apiKeyCount for affected projects.
4. Invalidate cache entries for revoked public keys.

Why this ordering

• Revocation is the source-of-truth state transition, so it happens first.
• Count sync and cache invalidation are side effects; they can fail independently.
• Side-effect failures are captured into retry tasks instead of aborting all work.

Failure model

• Project count sync uses Promise.allSettled: one project failure does not short-circuit others.
• Cache invalidation also uses Promise.allSettled.
• Rejected items are:
  • Logged with target identifier and error
  • Enqueued into maintenance_retry_task for future retry

This provides eventual convergence without requiring external workers/queues.

Result contract

runDailyMaintenance() returns structured per-job status:

• retryQueue: processed/succeeded/failed/exhausted/remaining
• usageBufferFlush: scanned/processed keys, upserted rows, counters, and failures
• requestLogCleanup: ok + deletedCount
• expiredApiKeySweep: ok + expiredKeys + affectedProjects + queuedRetryTasks
• top-level ok and durationMs

This shape keeps the endpoint machine-readable for alerting and human-readable for incident triage.

Why this fits Vercel Free/Hobby

• Uses one scheduled cron route with Postgres and Redis (buffer/lock) — no extra queue infrastructure.
• Avoids single-point short-circuit failures on batch side effects.
• Supports eventual repair by retry queue + next cron execution.
• Maintains idempotent core updates (revokedAt IS NULL guard) for safe re-runs.

Trade-offs and future upgrades

Current trade-offs:

• Side effects are eventually consistent, not strongly consistent.
• Exhausted retries require operator visibility (monitoring is important).

Potential future upgrades:

• Dedicated reconciliation job (periodic full apiKeyCount rebuild).
• Separate dead-letter handling UI/reporting for exhausted tasks.
• Distributed lock to prevent overlapping cron executions under high latency.
• Re-enable a higher-frequency schedule for /api/cron/flush-usage-buffer when plan limits allow.

Weekly demo-data resets

GET /api/cron/weekly-reset is scheduled for Mondays at 04:00 UTC (Monday noon in Taiwan) and uses the same cron authorization as daily maintenance. It is disabled unless DEMO_RESET_ENABLED=true and DEMO_RESET_PROTECTED_USER_IDS contains valid Clerk user IDs. Those accounts are excluded from both cycle registration and deletion.

Other users receive a persistent 14-day cycle beginning at their first authenticated dashboard/onboarding visit or protected API call after activation. Existing users are backfilled from their oldest team's creation timestamp. Ordinary visits and API traffic do not extend an active cycle. If a user returns at or after the deadline, authenticated entry routes and protected dashboard API calls clear the expired workspace before continuing, then start a full new 14-day cycle. A saved team or project URL redirects to onboarding when its old workspace has been removed.

Weekly cleanup still removes expired data for users who are away, normally 14–21 days after a cycle starts. It sets nextResetAt to null: no further cycle runs until the next authenticated visit. Returning weeks or months later starts a fresh full 14-day window, rather than inheriting an offline cleanup deadline. Clerk accounts remain usable throughout. Both visit-time and scheduled resets lock and recheck the same cycle row, so old queued tasks skip a resumed cycle.

Resets retain Clerk accounts, but delete all selected users' teams (including personal teams), projects, API keys, request logs, usage records and pins. Users can log in again and complete onboarding to create fresh projects. Projects created during an ongoing cycle inherit that cycle's deadline. The reset-cycle record is retained to track the next reset.

Each weekly run persists all due users as reset_demo_user tasks before deleting anything, then handles up to 50 users immediately. Weekly and daily maintenance drain remaining tasks in batches of 200. The response reports failures, newly queued users and the remaining backlog. A backlog can delay resets beyond the normal 14–21 days. Queued resets recheck the enable flag, protected IDs and cycle deadline. Each user's deletion and cycle-state update commit in one transaction. Row locks serialize duplicate resets and protect the parent/child deletion snapshot. New teams created after that snapshot survive as fresh data.

Redis cleanup tasks are saved in the same database transaction before cascading deletions. They revoke cached API keys, invalidate project caches, and remove only the deleted projects' fields from shared usage buckets. The weekly endpoint attempts cleanup immediately; daily maintenance retries failures. Existing config caches have a 60-second TTL; requests already in flight may complete during reset. Late usage increments for deleted projects are discarded by the normal usage flush and expire from Redis within its existing two-week retention period. Rate-limit counters retain their normal TTLs. Image/CDN caches are not explicitly purged; authenticated service access relies on invalidating the project and keys.

The retry task types cleanup_deleted_project and revoke_deleted_api_key extend the existing maintenance queue. Project cleanup uses a JSON task key containing the project ID and slug. Inspect exhausted tasks if Redis remains unavailable; a failed cleanup does not roll back an already committed reset.

Demo notice

The dashboard and onboarding pages display a nonmodal notice only when demo resets are enabled and the protected-user configuration is valid. It explains the 14-day window, retained accounts, and self-hosting for lasting projects. The notice appears after entry without moving focus and remains visible until the user clicks “Got it”; it has no timer or swipe dismissal. Navigation does not automatically close it. Only an explicit dismissal is saved, so refreshing before acknowledging shows the reminder again. The footer’s “Demo notice” button (also available on onboarding) always reopens it, even after dismissal or signing back in within the same tab. A versioned sessionStorage key shows it once per login session and cycle in each tab; when storage is unavailable, an in-memory guard prevents repeated notices during the current page session. The active window's end date is shown in the browser's locale and time zone. Protected users see the general demo explanation without a personal reset deadline.

Deployments upgrading from the original reset implementation must allow nulls in user_reset_cycle.nextResetAt before deploying the new code. This is an additive, backwards-compatible schema change; it does not delete existing records.

For an existing deployment, the targeted schema update is:

ALTER TABLE public.user_reset_cycle
  ALTER COLUMN "nextResetAt" DROP NOT NULL;