AI agent background jobs in 2026: when to keep the request open, return a job ID, or use durable workflows

A B2B AI feature can look fine in a demo and still fail the first time somebody closes a laptop, refreshes a page, hits a rate limit, or asks it to wait for a human. The moment an AI task can outlive the request that started it, you have an execution-design decision—not just a model-selection decision.

Should the API keep the connection open? Should it return a job ID and let a worker continue? Should you use a provider's background mode? Is this really an offline batch job? Or do you need a durable workflow that can pause, retry one step, survive a deploy, and resume after somebody approves a change?

My short recommendation: keep the request open when the work is bounded and the user benefits from watching it finish. Return a job ID when the work can continue independently and a simple queue plus a durable job record can describe its state. Use a provider-managed background call when one long model operation is the problem and the provider's polling, retention, and model limits fit your product. Use durable orchestration when the process crosses multiple steps, external events, human decisions, retries, or deploy boundaries. Use batch for independent offline volume. If you cannot define the job, its states, its owner, and its completion condition, postpone the infrastructure choice and define the workflow first.

This is not a ranking of Temporal, LangGraph, Vercel Workflows, OpenAI, AWS, or any other platform. Those products solve different slices of the problem. The useful question is what must remain true when the user disconnects, the model call fails, the worker restarts, or the person who needs to approve the next step is asleep.

The short answer

There are five practical execution shapes. They can share code, but they are not interchangeable.

The names are less important than the boundaries. A background job can be a perfectly good product architecture. A durable workflow can be unnecessary ceremony. A provider background call can remove an annoying timeout while leaving your application with no useful notion of progress. Diagnose the job before choosing the label.

Start with the user experience, not the infrastructure

Teams often start this conversation with a timeout: “Our endpoint takes too long, so we need a queue.” That is a useful symptom, but it is not the decision. The first question is what the user believes they are asking for.

Consider three requests from a sales operations product:

• “Summarize this one call and give me three follow-up bullets.”
• “Review the last quarter of accounts and flag renewal risks.”
• “Draft the renewal plan, ask the account owner for missing details, and prepare an approval packet.”

The first request is probably interactive. The user is sitting there, the output is small, and a streamed response may make the experience feel alive. The second is a background job. It can acknowledge the request, process accounts in a worker, show progress, and let the user return to the results. The third is a workflow. It has a human pause, a state transition, an approval boundary, and likely several external systems.

None of those conclusions depends on whether the model is GPT, Claude, Gemini, an open-weight model, or a small classifier. The execution shape follows the job and its consequences.

Ask these questions before you look at infrastructure:

1. Is the user waiting for an answer or commissioning work? Those are different promises. “Answer me now” wants a request. “Start this investigation” wants a job.
2. Can the work continue if the user closes the browser? If yes, the browser cannot be the source of truth for progress.
3. Does the work have a natural finish line? “Complete when every account has a risk label and a source note” is a job contract. “Keep thinking until it feels good” is not.
4. Can a step be repeated safely? A failed read-only search is different from a failed request to create an invoice, send an email, or change an account.
5. Can a person need to intervene? If the answer is yes, waiting is part of the workflow, not an exceptional error.
6. What should the user see after a failure? A generic red toast is not enough if three of five actions already completed.

Public practitioner discussions keep circling this exact gap. One production write-up describes a multi-step agent losing its connection halfway through, partial tool calls finishing, and in-memory state disappearing after a restart. A newer discussion asks how teams handle async workers, persistent state, and telemetry after the proof of concept. These posts are not a survey and do not prove that every team has the same problem. They are useful signals because they name the boring product work that demos tend to hide.

“Async” is four different decisions

“Run it asynchronously” sounds precise until you ask where the state lives, who can query it, how the work resumes, and what completion means. In current platforms, at least four patterns hide behind the word.

1. A provider-managed background model call

OpenAI's current Background mode documentation describes starting a long-running Responses API task with a background flag and polling the response object later. AWS's AgentCore Runtime documentation describes a similar user experience: acknowledge a task that may take minutes or hours, continue in the background, and let the user check back.

This is useful when the long part of the work is one provider operation. Maybe a reasoning model is doing a deep analysis. Maybe a single agent loop is already implemented and the immediate problem is the HTTP connection. A provider-managed object can be a clean first step.

It does not automatically solve the rest of your product. You still need to answer:

• Where do you store the relationship between your user, your business object, and the provider response ID?
• What does your app show while the provider object is queued, running, failed, or expired?
• What happens if the provider returns a result but your database write fails?
• How do you reconcile a provider completion that your webhook or poller missed?
• What data is retained temporarily, and does that fit your privacy or deletion policy?
• What happens if the model or background feature is not available in the region, plan, or account you deploy to?

OpenAI's documentation is unusually clear about one boundary: background responses are temporarily stored to support polling, and the retention behavior depends on storage settings. That is not a footnote to bury in implementation. If your product promises a result later, you need a plan for the result and the source data when the provider's temporary object is no longer available.

2. An application-owned background job

A simple job usually looks like this:

1. The API validates the request and creates a job record.
2. The API returns a stable job ID immediately.
3. A worker claims the job and performs one step at a time.
4. Each step records its input reference, result reference, status, and error.
5. The user polls, subscribes, or receives a notification.
6. The app reconciles the final result into the business record.

This can be built with a queue, a worker, a database, and object storage. It does not need an agent framework. The queue moves work off the request path; the database makes the job visible; the worker does the work; and the product decides what status means.

The common mistake is to create only a queue message. A queue message says that something should happen. It is not the durable identity of the work, not the user's status page, and not an audit trail. Messages can be delivered more than once, delayed, retried, or dead-lettered. The job record is where your application says what the request means now.

A good simple job record might include a job ID, owner, tenant, input reference, workflow version, status, progress summary, attempt count, last error, result reference, cancellation request, created time, started time, completed time, and an idempotency key. Keep sensitive prompt and tool content in controlled storage rather than spraying it through logs or notification payloads.

3. Batch processing

Batch is for volume without immediate interaction. OpenAI's current Batch API guide describes submitting groups of independent requests, checking batch status, using custom IDs to reconcile results, and a completion window of up to 24 hours. It lists evaluation runs, classification, embedding repositories, and offline rendering as examples.

That is a different promise from “your account review is running; come back to this status page.” In a batch, one record can fail while the rest complete. The important product work is importing results correctly, handling a partial failure, and deciding whether a record is safe to retry. The user may never need to see each model call.

Use batch when the items are independent and a completion window is acceptable. Do not use it as a substitute for a workflow with dependencies. If step two needs the approved result of step one, or a person must answer a question before the next call, you have a workflow even if each model request is individually batchable.

4. Durable workflow orchestration

A durable workflow stores enough history or checkpoints that it can continue after a process crash, a deploy, a long wait, or a human decision. Temporal describes this through event history and replay. LangGraph describes checkpointers, thread IDs, interrupts, and resumption. Vercel describes workflows that survive failures and interruptions while running long-lived agent or backend work.

The important capability is not “the agent can run for a long time.” The important capability is “the system knows what has already happened and can continue without repeating unsafe side effects.” A workflow engine may make that easier, but the rule exists even if you build it yourself.

When a synchronous request is the right product

Async is not automatically more mature. For a narrow interactive feature, adding a job system can make the product slower to understand and harder to debug.

Keep the request open when the answer is useful only in the current interaction, the tool path is short, the result can be safely abandoned when the user leaves, and the application has a clear timeout and error response. A support copilot that drafts a reply from a known policy source may fit this shape. A form that extracts fields from one uploaded document may fit it too.

Streaming can make a synchronous call feel more responsive, but streaming is a delivery mechanism, not persistence. If the stream drops, decide whether the work is cancelled, continues, or is restarted. If it continues, you have a background job whether or not you call it one.

There is also a control advantage to staying synchronous. The user sees a direct response to a direct request. The code path can be small. The request can return an error while the user still has enough context to fix the input. This is particularly valuable while a workflow is changing quickly and the team is still learning what “done” means.

Use a synchronous baseline first when you are unsure. Measure the real workflow, not just model latency. Include tool time, retrieval, validation, retries, and the cost of explaining a partial result. If the baseline is fast and clear enough, leave it alone. If the work routinely outlives the interaction or creates stranded state, move the execution boundary deliberately.

When a job ID is enough

A job ID is the simplest honest answer to a long-running task. It tells the user: “Your request has become an object that exists independently of this page.” That is often all a B2B product needs.

Use a simple queued job when:

• The job has a small number of steps with known ordering.
• Failures can be retried at the step or job boundary.
• Most steps are independent of a human decision.
• The user needs status and a result, not a live interactive control surface.
• You can make side effects idempotent or put them behind an approval.
• The job can be cancelled, expired, or moved to a manual review queue without losing business context.

Imagine a local-services company that wants to turn a recorded customer call into a structured follow-up brief. The API can save the recording reference, create a job, enqueue transcription and extraction, then store a draft brief. If extraction fails, retry that step. If the account is missing, mark the job as needing input. If the brief is only a draft, there is no reason to build a workflow engine that can wait for a week-long approval.

The user-facing status can be simple:

1. Queued: accepted, waiting for a worker.
2. Running: the current step and a short progress description.
3. Needs input: a specific question or missing field.
4. Completed: a result is available and linked to the business record.
5. Failed: the next action is retry, edit input, or ask for help.
6. Cancelled: no new work will start; completed side effects remain visible.

Do not make status a cheerful sentence generated by the model. Status is application state. The model can summarize what happened, but code should own whether the job is queued, waiting, complete, or failed.

When durable workflow orchestration earns its complexity

Durability becomes worth paying for when a restart would otherwise make you repeat expensive or unsafe work, or when the process naturally waits for something outside the model.

Strong signals include:

• Human pauses: a customer, reviewer, manager, or account owner must respond before work continues.
• External events: a webhook, file arrival, payment result, deployment, or third-party callback determines the next step.
• Long gaps: the work can wait for minutes, hours, or days without holding a worker open.
• Many dependent steps: later steps depend on durable results from earlier steps.
• Partial completion matters: the system must know which side effects happened before a failure.
• Replays matter: an operator needs to inspect or repeat the reasoning path without repeating real-world actions.
• Deploys are routine: a running process must survive new application versions or at least migrate in a planned way.

Temporal's documentation makes the boundary concrete: workflow code is replayed from recorded event history, while external calls such as database queries, API calls, file I/O, and LLM invocations belong in activities whose results can be recorded. LangGraph's documentation reaches the same practical concern through checkpointers and interrupts: save the graph state, keep a stable thread ID, wait for external input, and resume with a command. Those are different implementations of a shared systems problem.

Durability is especially useful when the AI is a planner inside a larger process. The model can decide which investigation step to try next. The workflow still owns the state transition, retry budget, approval pause, and final reconciliation. That division makes the agent more adaptable without making it the only source of truth.

Be skeptical when a vendor says “durable” without explaining what is actually persisted. Ask:

• Are prompts and model outputs stored, or only step status?
• Can a run resume after a human pause with the same state?
• Are tool calls retried automatically, and how are side effects protected from duplicates?
• Can the workflow wait on a webhook without using a live worker?
• What happens when the workflow definition changes while an old run is paused?
• Can an operator inspect and cancel one run without stopping every run?
• How are large results stored without making event history or checkpoints unwieldy?

Vercel's current Workflows announcements are a useful market signal: application platforms are packaging durable execution, retries, external events, and reconnectable streams as first-class features. That may be the right fit for a TypeScript team already living on the platform. It is still a product claim, not proof that every workload needs that platform or that a managed feature removes the need for an explicit job contract.

Design the job contract before adding a framework

The most portable asset in a long-running AI system is not the queue or the workflow engine. It is the job contract. Write it down before you pick a tool.

A useful contract names:

• Input: what request starts the work, and which records or files are in scope?
• Owner: which user, team, tenant, or system can see and control the job?
• Output: what artifact or business state proves completion?
• Steps: what can the system do, and which steps are model-assisted?
• State: which values must survive a restart and which are disposable?
• Stop conditions: when should the system stop, abstain, escalate, or ask for input?
• Side effects: which actions change money, access, customer records, public content, or external messages?
• Time policy: how long may the job run, wait, retry, or remain unanswered?
• Version: which prompt, tool schema, workflow definition, and source policy produced the result?

This contract also exposes when the proposed AI is unnecessary. If the steps are known, the system can be a deterministic workflow with one model call. If the inputs are structured, a normal query or parser may be more reliable. If the job has no useful output beyond “the agent tried,” it may be a demo rather than a product.

LangChain's public async-deep-agents repository offers a small but telling example of this idea. It keeps async job metadata in a dedicated state channel with a job ID, agent name, thread ID, run ID, and status instead of relying on message history. The repository explicitly warns against immediate polling, stale status from conversation history, truncated IDs, and collecting a result before the job finishes. You do not need LangGraph to apply the lesson: job metadata belongs in durable application state, not in whatever context window happens to be visible.

Retries, idempotency, and partial completion

Retries are where an AI demo becomes a distributed system. A model request may time out after the provider accepted it. A webhook may arrive twice. A worker may crash after an external API succeeds but before your database records the success. A human may click approve twice because the UI looked frozen.

For every step, classify the action:

Idempotency means a repeated attempt does not create a second unwanted effect. A job ID is helpful, but it is not automatically an idempotency key for every downstream system. Put a deliberate key on the side effect, store the result of the first successful call, and make the retry path check whether the action already happened.

This is also why “just let the model retry” is not a complete failure policy. The model can help interpret an error or choose a safe next step. It should not be the only layer deciding whether a second payment, message, permission change, or public post is acceptable.

Make partial completion visible. If a five-step job completed three reads, produced a draft, and failed before sending a message, tell the operator exactly that. A final status of “failed” without the completed-step history creates the worst of both worlds: the business does not know what happened, and the retry may repeat work or side effects.

Human pauses, reconnects, and notifications

Human-in-the-loop is not a modal dialog you sprinkle on top of an agent. It is a pause in the job's state machine.

LangGraph's interrupt documentation describes saving graph state, waiting indefinitely, and resuming with the same thread ID. Temporal's AI cookbook describes human approvals through signals. The implementation details differ, but the product behavior is familiar:

1. The system reaches a decision that needs a person.
2. It saves the proposed action and the evidence that led to it.
3. It records who is allowed to decide and when the proposal expires.
4. It shows the person a specific approval, rejection, or edit choice.
5. It resumes from a known state and records the decision.

Do not perform an irreversible side effect before the pause and then hope the person confirms what already happened. If a pre-approval step must run, make it read-only or explicitly reversible. LangGraph calls out a related sharp edge: code before an interrupt can run again when the node restarts, so side effects there must be idempotent.

Reconnect behavior deserves the same attention. If the user opens the app after the job finishes, they should see the result. If the user opens it after the job failed, they should see the failed step and the next action. If the user opens it while the job is paused, they should see the approval request, not an old spinner.

Notifications are events, not truth. A notification may be delayed, duplicated, or missed. The job record and status API remain the source of truth. A current GitHub issue in the LangChain ecosystem makes this separation explicit by distinguishing a background-task completion event from the completion of the foreground run that launched it. That is a small detail with a large product consequence: the system that starts work and the system that finishes work may not share a live session.

Buy, build, or postpone?

The right comparison is not “which agent platform has the most features?” Compare the operating burden against the job you can prove exists.

Buy managed orchestration when the value is removing infrastructure work that your team does not want to own, and when the provider exposes the state, retries, logs, versioning, and controls you need. Build more of it when portability, tenant isolation, unusual execution policy, or deep integration is part of the product itself. Stay simple when a queue and a database are enough.

A provider's marketing page may show a durable agent running for hours. That does not answer whether you can migrate a paused run, redact its state, cancel one tenant's work, replay a failed step, or explain a duplicate side effect. Ask for those examples before buying a platform.

A practical 30-day pilot

You can learn the right execution shape without making a company-wide platform decision. Pick one repeated workflow and make the pilot small enough that a failure is informative rather than existential.

Days 1–5: define the job and build a baseline

Write one sentence for the job: “Given this input, produce this result, for this owner, within this policy.” List the steps, sources, tools, human decisions, and side effects. Collect real examples, including the ones that humans corrected or abandoned.

Implement the simplest baseline that can do the work. It may be a synchronous request, a normal database query, or a thin model call. Save the input, output, duration, validation result, and human correction. You need this baseline to know whether a queue or workflow improves the product or only relocates the confusion.

Days 6–12: make the request boundary honest

Test what happens when the browser closes, the network drops, a provider times out, a worker restarts, and the user opens the task from another device. If the work should stop when the request ends, say so. If it should continue, create the job record and return the job ID.

Add statuses that code owns. Add a result pointer rather than placing a giant generated answer directly in a status row. Store a redacted run receipt with model identity, prompt or workflow version, tool names, step status, and timestamps. Keep secrets and unnecessary sensitive content out of the record.

Days 13–19: test failure and approval

Force a timeout after a successful side effect. Deliver a webhook twice. Fail a read-only step. Fail the database write after a provider call. Pause for approval, wait, then deploy a code change. Ask whether the job resumes safely, repeats work, or becomes ambiguous.

Add an approval only where it changes the risk boundary. The approval should show the exact action, target, arguments, evidence, and expiration. Do not call a vague “continue” button a control.

Days 20–30: compare the operating cost

Compare the baseline with the queued or durable candidate on the whole job:

• Did the correct business result arrive?
• Could a user see and recover from a failure?
• Did retries create duplicates?
• Could an operator explain what happened?
• How much human review did the system create?
• What did the queue, workflow engine, provider calls, storage, and notifications cost?
• Could the team change the prompt or workflow without corrupting paused work?

Choose one outcome:

1. Keep it synchronous. The work is bounded and the simpler product is good enough.
2. Ship a simple job. The work outlives the request but has a manageable state model.
3. Use a provider background or batch primitive. The provider-owned lifecycle fits the job and the limits are acceptable.
4. Adopt durable orchestration. The workflow needs pause, resume, replay, or multi-day survival.
5. Postpone. The failure is really an undefined workflow, weak source data, or missing product decision.

Write the stop condition before expanding. Stop if the team cannot identify the source of truth, cannot prevent duplicate effects, cannot reconstruct a failed run, or sees the durable system create more operator work than it removes. A pilot succeeds when the next decision is clearer, not when the architecture diagram is larger.

What I would predict next

Inference: “background agent” will become less useful as a product category. Buyers will ask for durable runs, reconnectable results, approval states, and clear cancellation instead. The visible experience may still be a chat box, but the underlying contract will look like a job system or workflow.

Inference: provider background modes will remain useful for removing request timeouts, but application teams will keep their own job identity and reconciliation layer. Providers own a model call. The business owns whether the result belongs to the right customer, record, and workflow version.

Inference: the most portable design asset will be a task contract plus a replayable evaluation set. If you can describe the job, compare the result, and explain the side effects, you can move between a queue, a workflow engine, or a provider primitive without starting from a blank page.

Those are predictions, not promises. The durable fact today is simpler: an AI request that can outlive the request needs an owner, a state model, and a recovery policy.

FAQ

Should every AI agent run in the background?

No. Keep a narrow interactive task synchronous when the user benefits from an immediate answer and the work can be safely abandoned or cancelled. Move it to a job or workflow when the user is commissioning work that should continue after the interaction ends.

Is a provider background mode the same as a job queue?

No. A provider background mode can keep one model operation running and expose a status object. Your application still needs to associate that object with a user and business record, handle expiration and retention, reconcile completion, and decide what happens when the provider call succeeds but your own write fails.

When do I need Temporal, LangGraph, or another workflow engine?

Consider durable orchestration when the process needs checkpoints, human pauses, external events, replay, multi-step dependencies, or survival across failures and deploys. If a simple worker and database can express the job and its recovery policy, start there. The framework is not the goal; reliable state transitions are.

Should I use batch for user-facing work?

Usually not when a person is waiting for one result or when later steps depend on an approval. Batch is a better fit for many independent requests that can finish within a provider's completion window and be reconciled by stable IDs. A user-facing job may still use a queue or workflow around individual calls.

How do I stop duplicate emails, charges, or updates?

Put those actions behind an application authorization boundary and give each effect a deliberate idempotency key. Record the first successful outcome, look up that key before retrying, and make the job show whether the effect happened. A model instruction to “do not repeat yourself” is not an idempotency control.

What should the user see while a job runs?

Show the job identity, current status, last completed step, a short explanation of what the system is waiting for, and the next available action. Let the user leave and return. Treat notifications as helpful signals, not as the source of truth. The status API and durable job record should tell the same story after a reconnect.

The practical next step

Take one AI workflow that currently runs inside a request and write down what should happen if the user closes the browser at the halfway point. Then write down what happened to each completed tool call, who owns the next step, and how the user finds the result later.

If those answers are clear, you can choose the smallest execution shape that preserves them. If the answers are not clear, do not begin with a workflow engine. Start with the job contract, a small evaluation set, and one failure you can reproduce.

BishopTech can help turn that messy workflow into a bounded build decision: a synchronous feature, a queue-backed job, a provider background call, an offline batch, or a durable workflow with approvals and recovery. See the custom software approach or explore automation systems if the next step is implementation. For a focused recommendation tied to your actual workflow, start a scoped BishopTech consultation.

Return to the My Mind research collection, or read the related guides to what agentic AI can actually do, why agent security is an authorization problem, and where persistent agent memory belongs.

Sources and further reading

The links below are dated references used for the distinctions and framework in this article. Official documentation describes product or framework behavior; vendor announcements include vendor claims; public discussions and repositories are included as qualitative practitioner signals, not universal benchmarks, survey results, or demand data.

1. OpenAI Developers — Background mode ↗ — current API documentation accessed August 23, 2026.
2. OpenAI Developers — Batch API ↗ — current API documentation accessed August 23, 2026.
3. Amazon Web Services — Handle asynchronous and long running agents with Amazon Bedrock AgentCore Runtime ↗ — current documentation accessed August 23, 2026.
4. Temporal — Temporal Workflow ↗ — current documentation accessed August 23, 2026.
5. Temporal — AI Cookbook ↗ — current documentation accessed August 23, 2026.
6. LangChain LangGraph — Persistence ↗ — current documentation accessed August 23, 2026.
7. LangChain LangGraph — Interrupts ↗ — current documentation accessed August 23, 2026.
8. Vercel — A new programming model for durable execution ↗ — April 16, 2026.
9. Vercel — The Agent Stack ↗ — June 17, 2026.
10. r/AI_Agents — AI agents work great until you deploy them and everything falls apart ↗ — October 6, 2025, public practitioner signal.
11. r/AI_Agents — You build your agent aaaand then what? ↗ — current discussion displayed August 23, 2026, public practitioner signal.
12. LangChain — async-deep-agents ↗ — public implementation and job-state signal accessed August 23, 2026.
13. LangChain deepagents issue #4656 — Notify when a background async task finishes ↗ — current public issue accessed August 23, 2026.