Background Jobs

Iris processes memory extraction, conversation summarization, and memory consolidation in the background so your chat experience stays fast and responsive. Here's how the pieces fit together.

Why Background Jobs?

When you send a message, you want a quick response. But extracting memories and generating summaries involves LLM calls that can take several seconds. By running these as background jobs:

• Chat stays fast - Responses stream immediately
• Processing is reliable - Jobs retry on failure
• Load is distributed - Jobs run when capacity is available

Running Horizon

Iris uses Laravel Horizon for queue management:

# For development (included in composer dev)
php artisan horizon

Horizon provides:

• Dashboard at /horizon for monitoring
• Automatic process management
• Job metrics and failure tracking

IMPORTANT

Without Horizon running, chat messages won't process, memories won't be extracted, and conversations won't be summarized.

Chat Processing

Chat messages are processed asynchronously within the active thread, enabling reliable delivery even when connections are unstable.

How It Works

1. When you send a message, a job is queued for the active thread
2. The job builds thread-scoped context and processes your request, executing tools as needed
3. Response events broadcast to clients viewing that thread via WebSockets
4. If your connection drops, events are stored for replay

Reliability Features

Feature	Benefit
Automatic retries	Failed requests retry with exponential backoff
Event storage	Reconnecting clients catch up automatically
Graceful stops	Users can stop generation mid-stream

Automatic Jobs

These jobs dispatch automatically after Iris processes a response, based on configurable thresholds.

ExtractMemories

Pulls meaningful information from recent conversations and stores it as searchable memories with embeddings.

Setting	Default	Description
Trigger	Every 10 messages	Configurable via iris.extraction.threshold
Max per run	6 memories	Prevents over-extraction from a single batch
Timeout	120 seconds	Maximum LLM call duration

The job builds context from up to 50 recent messages, uses an LLM to identify what's worth remembering, then creates memory records with vector embeddings for semantic search.

See Memory Extraction for details on what gets extracted.

SummarizeConversation

Creates narrative summaries of older conversations within a thread, capturing emotional context and relationship dynamics. Each thread maintains its own independent summary chain.

Setting	Default	Description
Trigger	40+ unsummarized messages in thread	After keeping 35 recent messages as buffer
Timeout	120 seconds	Maximum LLM call duration

Summaries chain together via previous_summary_id within the thread, maintaining conversational continuity. Each summary includes emotional markers, resolved/unresolved threads, and relationship dynamics.

See Summarization for details on what summaries capture.

GenerateThreadBrief

Generates a thread brief — a 3-6 sentence digest of what a thread is about, including topic, key decisions, current status, and emotional context. Briefs power cross-thread awareness in chat and provide context for proactive message decisions.

Setting	Default	Description
Trigger	Every briefs.frequency assistant responses (default: 2)	After minimum briefs.threshold responses
Model	claude-haiku-4-5	Fast, inexpensive model for brief generation
Timeout	15 seconds	Maximum LLM call duration

The job loads the last ~10 messages, the existing brief (for continuity), and the latest conversation summary (if one exists) for structured context. Output is sanitized and stored on the thread model. The job implements ShouldBeUnique with "brief-{threadId}" to prevent concurrent generation for the same thread.

GenerateThreadName

Automatically generates a short, descriptive name for new threads based on conversation content.

Setting	Default	Description
Trigger	After threads.naming_threshold messages (default: 4)	Only for threads not manually renamed
Model	claude-haiku-4-5	Fast, inexpensive model for naming
Timeout	15 seconds	Maximum LLM call duration

The job validates LLM output (rejects multi-sentence prose, refusals, or overly long names) and broadcasts the new name to all connected clients via WebSocket.

Scheduled Jobs

Memory Consolidation

Consolidation merges semantically similar memories into denser, more useful representations. It runs on two schedules:

• Daily (3 AM) - Processes memories from the last 3 days
• Weekly (Sunday 4 AM) - Full sweep of all memories

Consolidation uses a two-phase job architecture:

1. ConsolidateUserMemories - Builds clusters of similar memories
2. ConsolidateMemoryCluster - Processes each cluster with LLM review

See Memory Consolidation for the full details on how it works, generation tracking, and command options.

Agent Daemon

Delegated sub-agent tasks are processed by the iris:agent daemon — a separate long-running process that operates independently of Horizon.

Why a Daemon?

Sub-agent tasks can run for several minutes and involve streaming LLM responses with real-time tool call broadcasting. This doesn't fit well into a traditional queue worker model. The daemon provides:

• Sequential processing — one task at a time, preventing resource exhaustion
• Inline rate limit handling — sleeps and retries when API limits are hit, rather than releasing back to a queue
• Graceful shutdown — responds to SIGINT/SIGTERM signals, finishing the current task before stopping
• Exponential backoff — when idle, polling frequency decreases to reduce overhead

Running the Daemon

# Included automatically in composer dev
php artisan iris:agent

The daemon polls for pending tasks, processes them using AgentTaskRunner, and broadcasts progress events via WebSockets. Task results are delivered as proactive messages in the originating thread.

IMPORTANT

The agent daemon is not managed by Horizon and won't appear in the Horizon dashboard. Monitor it through your process manager or terminal output.

See Task Delegation for feature details and configuration.

Monitoring Jobs

Horizon Dashboard

Access the Horizon dashboard at /horizon to monitor:

• Job throughput and processing times
• Failed job details and stack traces
• Queue lengths and wait times
• Memory and process health

Queue Commands

# Check failed jobs
php artisan queue:failed

# Retry failed jobs
php artisan queue:retry all

Logging

Jobs log their activity to Laravel's default log channel. Check storage/logs/laravel.log for:

• Job start/completion timestamps
• Extraction and summarization results
• Error details for failed jobs

Failure Handling

Automatic Retries

Jobs are configured with retry policies:

Job	Retries	Backoff
ProcessChatRequest	3	Exponential
ExtractMemories	3	Exponential
SummarizeConversation	3	Exponential
ConsolidateMemoryCluster	Time-based (2 hours)	Rate-limit aware

Rate Limit Handling

Consolidation jobs handle API rate limits specially:

• Jobs release back to queue when rate limited
• Automatic retry when the limit resets
• Uses Prism's resetsAt timing for precise retry scheduling

Manual Intervention

If jobs are persistently failing:

# View failed jobs
php artisan queue:failed

# Retry specific job
php artisan queue:retry <id>

# Clear failed jobs
php artisan queue:flush

Configuration

Key settings in config/iris.php:

'extraction' => [
    'threshold' => 10,     // Messages between extractions
    'max_memories' => 6,   // Max memories per run
    'timeout' => 120,      // API timeout in seconds
    'model' => 'claude-sonnet-4-5',
],

'summarization' => [
    'threshold' => 40,     // Unsummarized messages to trigger
    'buffer' => 35,        // Buffer before summarizing old messages
    'timeout' => 120,      // API timeout in seconds
    'model' => 'claude-sonnet-4-5',
],

'consolidation' => [
    'jobs_per_minute' => 10,  // Rate limit for LLM calls
    // ... other settings
],