Skip to content
BishopTechBishopTech
Back to My Mind
AI architectureCode executionSandboxesRuntime boundaries28 min read

B2B AI code execution in 2026: hosted tool, sandbox, or no code?

Choose the smallest execution surface that can do the job. Keep authority, policy, and durable business truth outside the generated script.

Four lanes for B2B AI code execution from deterministic functions to self-hosted runtimes

As runtime control rises, so do the data, policy, and operations you must own.

Giving an AI system a code tool can feel like a tiny product decision. Add the tool, tell the model to use it when helpful, and suddenly a chat can calculate a cohort, transform a spreadsheet, inspect a repository, or produce a chart. The demo feels more capable because it is more capable.

Production is where the tiny decision becomes an architecture question. Where does the code run? What can it read? Can it reach the internet? Which packages are installed? Does the workspace survive the next request? What happens when the command waits for input? Who owns the files it creates? Can a model-generated script call an internal API or spend money? How do you know whether a timeout means the script failed, the runtime failed, or the tool never started?

My short answer is this: use the smallest execution surface that can complete the job. Keep known business operations as deterministic application functions. Use a provider-hosted code execution tool for bounded, ephemeral analysis when its language, data path, runtime, and retention behavior fit. Use a managed sandbox when the product genuinely needs arbitrary packages, multiple languages, persistent workspace state, or broader agent iteration. Self-host the runtime only when the data boundary, network controls, residency, or operational requirements justify owning much more of the system.

Decision rule: a sandbox limits the blast radius of code. It does not decide whether the code is authorized, whether the input is trustworthy, whether the result is correct, or whether a write should happen. Keep identity, policy, approval, and business truth outside the model and outside the generated script.

The short answer: choose the lane before the tool

There are four honest lanes for most B2B products. They are not four brands or four interchangeable infrastructure products. They are four different product commitments.

Lane Use it when What you accept
Deterministic function The operation is known, repeatable, and expressible as a reviewed application function or typed tool. Less open-ended flexibility, in exchange for clearer behavior, tests, permissions, and operations.
Provider-hosted code tool The user needs short-lived analysis, calculations, file transformation, or charts inside a provider-defined environment. Provider-specific language, packages, data path, timeouts, state, artifact lifecycle, and pricing.
Managed sandbox The product needs a controllable runtime with custom images, packages, files, multiple languages, or iterative state. A new runtime lifecycle, resource budget, egress policy, secret strategy, isolation review, and vendor dependency.
Self-hosted or VPC runtime The organization must own the data path, network boundary, patching schedule, execution policy, or regional placement. The organization also owns orchestration, image supply chain, capacity, recovery, observability, and incident response.

The most common mistake is starting with the second or third lane because the model is impressive when it can write code. Start with the job. If the job is “calculate the renewal date from these fields and apply a rule,” a reviewed function is usually a better product than a model writing a date-calculation script. If the job is “explore an unfamiliar CSV, try several transformations, and return a chart,” open-ended execution may be exactly the right capability.

That distinction is more important than whether the interface calls the feature an agent, a copilot, a code interpreter, a sandbox, or a terminal. Names describe the experience. The execution boundary describes the risk.

Why code execution is a product boundary, not a model feature

A normal model response is usually text or a structured value. The application can validate it, show it, and decide what to do next. Code execution adds another loop:

  1. The model proposes code or a command.
  2. The runtime interprets it and allocates resources.
  3. The code reads inputs, imports packages, and may create files.
  4. The runtime returns stdout, stderr, files, images, or an error.
  5. The model observes the result and may revise the code.
  6. The application decides whether the result is useful, safe, and authorized to leave the runtime.

Each step adds a separate failure mode. The code can be syntactically valid but semantically wrong. A file can be in the workspace but not be the file the user meant. A package can be available today and missing tomorrow. A command can hang while waiting for input. A result can be numerically correct but based on a stale export. A sandbox can isolate the process but still have broad network egress. A generated artifact can contain a customer identifier that the chat UI was not supposed to reveal.

OpenAI, Anthropic, and Google all document provider-hosted code execution, but their boundaries are not the same. OpenAI documents Python execution in a sandboxed container with uploaded or generated files, selectable memory tiers, and an expiration model. Anthropic documents Python and Bash execution, pre-installed libraries, no internet access in the described container, and newer modes for persistent interpreter state or programmatic tool calling. Google documents Python-only execution, iterative feedback, a maximum code-environment runtime, and retries after errors. Those are product differences, not implementation trivia.

The useful question is not “which provider has a code interpreter?” It is “what exact runtime contract does this workflow need, and which part of that contract am I willing to outsource?”

The four execution lanes

Lane one: a deterministic function or typed tool

Use this lane when you already know the operation. The model may still help interpret the request, select a function, fill typed arguments, or explain the result. It does not need to invent a program every time.

Examples include:

  • Converting a quoted price between supported currencies using a current rate source.
  • Looking up an account's current plan and calculating a permitted renewal window.
  • Filtering a reporting view by a validated tenant, date range, and status enum.
  • Generating a signed download link for a specific file the user is already allowed to see.
  • Applying a known eligibility rule and returning the rule version with the result.

A deterministic function is not anti-AI. It is often the way to make an AI feature more dependable. The model can remain flexible at the language boundary while the application keeps the operation narrow. The function can have tests, typed inputs, explicit authorization, idempotency, and a clear audit event. If it fails, you can usually name the failure without reconstructing a one-off program.

Do not force code execution into a known operation merely because it sounds more agentic. A model that writes a SQL query for a fixed dashboard filter may introduce syntax, scope, and data-exfiltration risk that a parameterized query never needed. A model that writes a refund script is not more capable in the product sense; it is harder to govern.

Use the function lane when you can write the function name in one sentence. If the job has a stable input contract and a stable set of allowed side effects, start with a typed tool or ordinary application code. Add open-ended execution only when a real class of tasks cannot be expressed that way.

Lane two: a provider-hosted code tool

A provider-hosted tool is attractive because it removes a large amount of infrastructure from the first version. The provider provisions an execution environment, handles the tool loop, captures results, and may already support files or charts. That is a real advantage for a product that needs analysis now and does not yet need a general-purpose runtime platform.

This lane fits jobs such as:

  • Letting a user upload a CSV and ask for a one-time summary.
  • Checking a spreadsheet for duplicates, missing values, or simple anomalies.
  • Generating a chart or a transformed file from an uploaded dataset.
  • Running a bounded calculation whose intermediate steps benefit from actual execution.
  • Turning a document or image into a derived artifact when the provider's tool supports the needed file path.

The trade is that the provider owns important parts of the runtime contract. The language may be fixed. The package set may be curated. The network may be disabled or restricted. The container may be ephemeral. The artifact may need to be copied to your storage before the environment expires. The provider may bill the execution session separately from model tokens. A tool can be excellent for a user-facing analysis and still be the wrong place to store a durable customer record.

Provider-hosted code is also not automatically a private computation boundary just because it is called a sandbox. You still need to understand what leaves your system, what the provider retains, which region or plan applies, how files are deleted, and whether logs or tool traces contain sensitive values. “Hosted” simplifies operations. It does not remove procurement or data-governance questions.

Lane three: a managed sandbox

A managed sandbox is a better fit when runtime control is part of the product experience. You may need custom dependencies, multiple languages, a repository checkout, a persistent workspace, long-lived processes, or a controlled path for an agent to test and revise code. The sandbox provider becomes an execution substrate rather than merely a model tool.

Current managed sandbox documentation shows why this lane is different. Modal describes runtime-created sandboxes for untrusted user or agent code, custom images, arbitrary commands, readiness probes, timeouts, and lifecycle management. Daytona documents stateless and stateful code execution across several languages, shell commands, sessions, and long-running processes. E2B describes per-sandbox Firecracker microVM isolation, timeout destruction, pause/resume state, and a bring-your-own-cloud option. These are broader controls than a single provider-defined Python call.

That flexibility is the point—and the danger. A managed sandbox can give a model more room to work, but your application now has to define what “more room” means. Which image is allowed? Which package sources are trusted? Can the code install packages? Does it have outbound network access? Can it reach an internal service? How are secrets injected? Can two tasks share a workspace? How are files scanned before they return to a user? What happens after a timeout? How do you terminate a runaway process?

Do not buy a sandbox to solve a prompt problem. Buy or build one when the job itself needs an execution environment.

Lane four: self-hosted or VPC execution

Self-hosting is a control decision, not a badge of seriousness. It may be justified by a strict data boundary, a private network dependency, regional placement, custom hardware, a requirement to inspect every runtime event, or a need to integrate with existing security controls. It may also be the wrong move for a small team that has not yet proven the workload.

At this level, you own the boundary. That includes images, patching, host hardening, scheduling, capacity, secrets, network policy, logging, incident response, snapshot behavior, and recovery. A microVM primitive such as Firecracker can provide a stronger isolation boundary than a traditional process or container, and managed platforms may build on that kind of primitive. But adopting the primitive directly does not give you a finished product. It gives you a powerful component that still needs an operating model.

The question is not whether self-hosting is safer in the abstract. The question is whether your team can operate the chosen boundary more reliably than the available managed option, and whether the resulting control is worth the cost and delay.

Compare the runtime contract, not the marketing label

Before choosing a tool, write down the runtime contract the workflow actually needs. This makes hidden assumptions visible.

Dimension Questions to answer Why it changes the lane
Language and packages Is Python enough? Are native libraries, browser binaries, JavaScript, or system tools required? A provider tool may be enough for a fixed library set. Custom runtimes point toward a managed or owned sandbox.
Data path What data enters the runtime, where is it processed, and what must remain in a private network? A strict data boundary may rule out a default hosted tool or require a VPC deployment.
Network egress Does the code need no network, a domain allowlist, or access to a private service? Network access is an authority and exfiltration question, not merely a convenience setting.
State Is each run independent, or must files, variables, and processes survive between steps? Persistent state adds lifecycle, tenant isolation, cleanup, and stale-data concerns.
Side effects Does the code only return an artifact, or can it change a system of record? Side effects need application authorization, idempotency, review, and a way to reconcile unknown outcomes.
Time Must the user wait seconds, or can the work run as a durable job? Interactive tools have hard limits. Longer work needs explicit job state, cancellation, and recovery.
Evidence What code, inputs, outputs, versions, and logs must a reviewer be able to inspect? Audit and evaluation requirements can make a narrow application-owned path preferable to a black box.

The table is deliberately boring. Boring is useful here. Teams get into trouble when they choose a tool from a feature list and only later discover that the important requirements were persistence, private data, or a reversible side effect.

Six questions that usually choose the lane

1. Is the job open-ended, or merely described in natural language?

Natural language does not automatically imply open-ended computation. “Find the unpaid invoices older than thirty days” is a natural-language request, but it may map cleanly to a filtered, authorized query. “Explore this unfamiliar export and tell me which segments behave differently” is more open-ended. It may benefit from actual code because the user does not know the analysis path in advance.

Write two versions of the job:

  • Known operation: “Given a tenant, date range, and status, return the matching records and totals.”
  • Exploratory operation: “Given a dataset and a question, inspect, transform, test, and explain a useful analysis.”

The first usually wants a function. The second may want code execution. If you cannot tell which one you are building, do not begin with arbitrary execution. The uncertainty is a product-definition problem.

2. What data must cross the boundary?

List the input classes, not just the database name. A workflow may use public reference data, user-uploaded files, tenant records, credentials, customer messages, or regulated information. These have different handling requirements.

Then draw the path:

  1. User or system input enters the model context.
  2. The model chooses whether to invoke code.
  3. Data is sent to the execution runtime.
  4. The runtime reads files, packages, environment variables, or network responses.
  5. Results return to the model and application.
  6. The application stores, displays, or acts on the result.

That diagram often reveals that the code runtime is not the only third party. The model provider, file store, logging system, tracing platform, package registry, and notification path may all see pieces of the job. A hosted tool can be the right answer, but only after the data path is explicit.

Do not use a local model as a shortcut around an unsafe execution design. A model running in your network can still generate a dangerous command, read a secret, or follow malicious instructions in a repository. Model location and execution authority are related decisions, not substitutes for each other. The earlier B2B AI hosting guide covers the model-side choice; this article is about what happens after the model proposes work.

3. Do you need a runtime, or do you need a calculation?

This is the question that saves the most unnecessary infrastructure.

A calculation needs an input, an algorithm, and an output. A runtime is a place where arbitrary programs can load files, call libraries, create processes, and make decisions about what to do next. If your product needs only the first shape, build the calculation. If it needs the second, decide how much of the runtime is safe to expose.

For example, a sales-forecast feature may appear to need Python because a model can write a forecasting script. But if the product supports three known methods with an approved dataset and a versioned result, a deterministic service can be easier to test and explain. Conversely, a data analyst uploading an unfamiliar workbook may reasonably need a flexible environment to inspect sheets, normalize columns, and try several visualizations.

Open-ended code is valuable when the user cannot know the exact computation before seeing the data. It is unnecessary when the product already knows the computation.

4. Does state belong to the user, the job, or the runtime?

State is one of the quiet differences between a demo and a product. A code tool may preserve a conversation, a container, a Python interpreter, a filesystem, or nothing at all. Those are different kinds of state.

Define what must survive:

  • Conversation state: what the model needs to understand the next request.
  • Job state: what the application needs to resume, retry, cancel, or reconcile work.
  • Workspace state: files, variables, processes, and artifacts inside the runtime.
  • Business state: the durable record that belongs in your system of record.

Do not confuse workspace state with business state. A container or sandbox can be a useful scratchpad, but it should not be the only copy of a contract, customer decision, report, or approval. Copy the durable artifact to storage you control, attach provenance, and record the runtime and code version that produced it.

Persistent workspaces also create tenant questions. Can one customer ever observe another customer's files? Does a paused workspace retain sensitive information? Who can reopen it? What happens when the user leaves the organization? Can an old package or file influence a new run? State makes a sandbox more useful, but also makes cleanup and access control part of the feature.

5. What can the code change?

Separate computation from authority. A script that creates a chart has a different risk profile from a script that sends an email, updates a CRM, changes a price, or deploys a release. The model may generate both, but the application should not treat them as the same class of tool.

A practical progression is:

  1. Read-only scratch: use copied or scoped data, no credentials, and no external writes.
  2. Constrained reads: expose approved datasets or views through a broker rather than a broad database credential.
  3. Proposed change: return a diff, report, or typed action proposal for application validation.
  4. Approved write: let a separate application function perform the side effect after identity, policy, and approval checks.

Do not put a production token into an environment variable simply because the sandbox supports environment variables. Secret injection is an authority decision. Prefer short-lived, downscoped credentials behind a broker, and do not expose secrets to the model context or arbitrary subprocesses when the application can perform the authorized operation itself.

The AI agent identity guide goes deeper on delegated user access, workload identity, agent principals, and task grants. The relevant rule here is simple: the code runtime is where work happens; it is not automatically the principal allowed to do the work.

6. What happens when execution fails halfway through?

Every execution design needs a failure story before it needs a success demo. Test at least these cases:

  • The code has a syntax error.
  • The code runs out of time.
  • The command waits for interactive input.
  • A package is missing or incompatible.
  • The runtime loses network access mid-run.
  • The process creates an artifact and then crashes.
  • The application times out but the runtime keeps working.
  • A write may have happened before the client disconnected.

Retrying is not recovery by itself. A retry can duplicate a side effect, reuse stale state, or hide a configuration problem. Record a job identifier, runtime identifier, code or tool version, input references, output references, exit state, and cancellation state. If a result is unknown, make “unknown” a real state that a person or reconciliation process can resolve.

The background-jobs guide covers durable job state, idempotency, and reconnects. The connection is important: code execution may be one step inside a job, but it is not a job system just because it can run for a while.

Security: “sandboxed” is only one line in the design

Current security guidance is refreshingly direct about this. The NIST preliminary AI cybersecurity profile says arbitrary code execution by AI agents should, in most applications, be curtailed, sandboxed, subject to approval and monitoring, or completely disallowed. Singapore's 2026 addendum on securing agentic AI emphasizes that risk rises when systems can formulate plans and act through tools and data. NIST's agent-security work describes how untrusted emails, websites, and repositories can steer agents toward downloading or running malicious code.

Those recommendations do not mean that code execution is unusable. They mean that “we put it in a sandbox” is not the end of the threat model.

The sandbox boundary

Ask what the runtime is actually isolated from. Is it a process, a container, a gVisor-style boundary, a microVM, a separate host, or a customer-owned cloud account? What is shared: kernel, filesystem, memory, network, image cache, logs, or control plane? How are workloads separated across tenants?

Isolation strength is not a moral ranking of technologies. It is a fit question. A short-lived read-only transformation over non-sensitive data may not justify a custom microVM platform. An agent that runs arbitrary code with elevated privileges or handles confidential repositories may need a stronger boundary. The more sensitive the data and the more powerful the runtime, the less acceptable it is to leave the boundary vague.

Egress and package supply chain

Network access changes the threat model. Without egress restrictions, code can send data to an arbitrary host, download a new package, fetch instructions, or call an API that the product never intended to expose. With no network, some useful workflows stop working. The right answer is usually a deliberate allowlist, a broker, or a split between an untrusted execution environment and a trusted service that performs narrowly defined requests.

Package installation deserves the same attention. A model may decide that a library would help, but automatically installing code from an untrusted source turns the package manager into another execution surface. Pin dependencies, use vetted images, scan artifacts, restrict package sources, and make installation a policy decision. If the job does not need package installation, do not enable it because it makes the demo more flexible.

Filesystem, secrets, and identity

Use a disposable workspace by default. Mount only the files needed for the job. Avoid host mounts, shared writable directories, and broad home directories. Separate customer workspaces and delete them on a defined schedule. Make the application own the durable artifact rather than leaving it in a runtime that may expire or be reused.

Keep secrets out of the model's context. Better still, keep them out of arbitrary generated code. If a sandbox needs to request data, route the request through a narrowly scoped service that checks the user, tenant, task, fields, rate, and policy. The service can return a filtered result without exposing the credential or the entire underlying system.

Identity needs to follow the action. “The user asked” is not enough to authorize a script to query every account. “The agent has a service account” is not enough to authorize every operation that account can perform. Tie each read or write to a tenant, user or workload identity, task, resource, operation, and policy result. Log the decision alongside the code and output references.

Approval and observability

Approval should happen before the consequential action, not after the system has already run the dangerous code. Show the reviewer the exact command or action, the target, the identity, the input evidence, the expected side effect, the policy result, and the rollback or recovery path. A generic “approve AI action” button is a weak boundary because the reviewer cannot tell what will happen.

Keep a redacted execution receipt. It should answer: what request started the job, which model or tool version proposed the code, which runtime and image ran it, what inputs were mounted, what network policy applied, which commands ran, which files were created, what exit state occurred, and whether the result was accepted, rejected, or sent to review.

This is where code execution meets evaluation. The B2B evals guide argues for testing the whole job rather than only the final answer. For code execution, the whole job includes the chosen tool, code path, resource use, blocked actions, artifacts, and recovery behavior.

Useful distinction: isolation limits what a process can touch. Authorization decides whether the proposed operation is allowed. Validation decides whether the result is meaningful. Approval decides whether a human accepts a consequential step. Logs make the decision reconstructable. You need all four when the job can affect a real system.

Which lane fits common B2B jobs?

A one-time CSV analysis

A user uploads a sales export and asks which segments changed month over month. This is a strong candidate for a provider-hosted tool if the provider can process the file under the user's data requirements, the needed libraries are available, the runtime is long enough, and the output is copied into your own storage. The model can inspect columns, write a small analysis, iterate after an error, and return a chart or a report.

Keep the final report separate from the scratch runtime. Store the input reference, transformation steps, generated artifact, and caveats. If the report will be used in a board packet or compensation decision, add a review state. A successful chart is not proof that the columns were interpreted correctly.

A customer-facing formula

A customer asks, “What will my invoice be if I add three seats next month?” This is usually a deterministic function, not code execution. The price book, billing period, discounts, tax rules, and account state are too important to leave to an improvised script. Let the model translate the question into typed inputs; let application code calculate and explain the result.

If customers can define their own formulas, you may need a constrained expression language rather than arbitrary Python or shell. A small language with a fixed set of operations is easier to validate, version, and evaluate than general code. The goal is not to make the feature less powerful. It is to make the power legible.

A document transformation pipeline

A team wants to turn a folder of proposals into normalized summaries and a comparison workbook. If the document families and output schema are stable, a document pipeline with explicit parsers and application validation may be a better first version. If the input formats are still changing and analysts need to explore the files, a sandbox can be useful during the pilot.

Do not allow the model-generated code to decide which customer documents it may read. Resolve that scope in the application first. Pass a manifest of allowed files or a filtered copy into the runtime. Preserve page, row, or region evidence so that a reviewer can find the source. The document AI guide covers why extraction structure is not the same thing as provenance.

Testing a repository or generated patch

An AI coding workflow may need a real workspace, package installation, a test runner, and several iterations. That is a legitimate managed-sandbox or self-hosted-runtime use case. The runtime should receive a disposable checkout or worktree, not a developer's home directory or production credentials. Network access should be limited to what dependency installation and tests require. The output should be a diff, test receipt, and artifact—not an unreviewed deploy.

In this job, state can be useful because the agent needs to iterate. It also makes cleanup, branch isolation, and cancellation important. A runtime that can modify files is not automatically authorized to merge or release them. Those actions belong behind the normal review and deployment controls.

An internal data question

An operations user asks, “Which accounts have open support cases and no successful payment in the last thirty days?” The first version should probably expose a curated read-only view or typed query tool. The model can select filters and explain the answer, while the service enforces tenant scope, row-level rules, and field redaction.

Arbitrary code may become useful for an analyst who needs to explore an unfamiliar dataset, but it should receive a purpose-built extract or query result, not unrestricted database access. The runtime should not become a back door into every table simply because SQL or Python is convenient.

A production write or external action

Sending an email, changing a subscription, issuing a credit, creating a user, or deploying a release is not a good first use of arbitrary code execution. The model can propose the action. A typed application operation should validate the target, permissions, state, idempotency key, and review requirement. The execution runtime can help draft or simulate the change, but the authoritative write should happen in a controlled service.

If you eventually support a write from an agent, treat it as a separate lane with its own tests and approval policy. Do not infer that because a read-only analysis worked, the same sandbox is suitable for writes.

Build, buy, or postpone?

Choice Choose it when Do not choose it when
Use a provider-hosted tool The job is bounded, mostly ephemeral, and fits the provider's language, file, network, state, and data contract. You need private-network access, custom runtimes, durable workspaces, or a portable execution contract the provider does not expose.
Buy a managed sandbox Runtime control is part of the user value, and your team wants to outsource isolation and provisioning while keeping application policy. You have not measured the job, do not know the required boundary, or expect a vendor to solve authorization and governance for you.
Build or self-host Data locality, private networking, custom isolation, or existing platform controls justify owning the runtime. Your main reason is avoiding a small provider bill, or your team cannot operate patching, scheduling, logging, and incident response.
Postpone arbitrary execution The job can be solved by a typed function, the source data is unclear, or the side effect is too consequential to govern yet. The product's real value depends on open-ended analysis that a fixed function cannot express.

“Postpone” is a useful product outcome. It means the team has identified the missing contract rather than hiding it behind a bigger model or a more impressive runtime. You can return to the decision after you have a real workload, representative data, and a clear stop condition.

If you buy a managed sandbox, ask the vendor questions that belong to your own threat model:

  • What is the isolation boundary and what is shared?
  • Where do control-plane requests, logs, snapshots, and artifacts travel?
  • Can you deny network access by default and allow only specific domains or services?
  • How are images and packages built, signed, scanned, and updated?
  • Can each customer, task, or trust boundary receive a separate workspace?
  • How are secrets injected, scoped, rotated, and prevented from reaching model output?
  • What are the timeout, cancellation, retry, pause, resume, and cleanup semantics?
  • Can you export an execution receipt for incident review and customer support?
  • Which data-retention, regional, contractual, and compliance claims are documented rather than implied?

Vendor documentation is useful evidence, but it is still a vendor description. Ask for the contract, the current security materials, and a test environment. Do not convert a product page into a guarantee.

A 30-day pilot that can settle the decision

You do not need to build a general-purpose agent platform to decide whether code execution belongs in your product. You need a narrow workload and a testable boundary.

Days 1–5: define the job and baseline

Choose one job with a clear owner and a real source of inputs. Write the contract:

Given this input, produce this artifact for this owner, within this data and authority boundary, with this evidence and this failure state.

Collect representative examples, including malformed files, missing columns, large inputs, ambiguous requests, and cases a human corrected. Implement the simplest non-executing baseline. If a deterministic function can solve the known subset, build it first. This gives you something to compare against.

Days 6–12: test the smallest execution lane

If the baseline cannot express the job, test a provider-hosted tool with a narrow file manifest and no external credentials. Record runtime, model, input size, output size, execution duration, package assumptions, error state, and whether the result can be reproduced. Save the input and output references outside the runtime.

Do not optimize for the happiest demo. Include a job that should abstain, a job that should fail cleanly, and a job where the source data is intentionally misleading. You are testing whether the system knows when it should not run or trust the code.

Days 13–19: add the control boundary

Test the controls that make the feature acceptable:

  • Read-only workspace with no host mounts.
  • Network disabled or limited to an explicit allowlist.
  • No production credentials in the model context or runtime by default.
  • Per-task or per-tenant workspace separation.
  • Maximum runtime, CPU, memory, disk, process count, and output size.
  • Command cancellation and hard termination.
  • Artifact scanning and redaction before display or storage.
  • Execution receipt with code, inputs, outputs, versions, and policy result.

If the job truly needs custom packages, multiple languages, persistence, or controlled network access, compare a managed sandbox with the hosted tool. Keep the application contract constant so that you are comparing execution lanes rather than changing the product at the same time.

Days 20–26: test failure and human review

Force timeouts, missing dependencies, interactive commands, broken files, malformed output, network denial, package installation attempts, and a simulated secret. Confirm that the user sees a useful failure rather than an empty spinner. Confirm that the application does not retry a possible write as if it were a harmless syntax error.

Add a review step for any output that will affect a customer, financial record, access decision, public communication, or production system. Review should show the source evidence, proposed action, target, identity, policy result, and exact artifact. If a reviewer cannot tell what will happen, the review is not ready.

Days 27–30: make the go, no-go, or narrow decision

Choose one of three outcomes:

  • Go: the lane solves a real job, the boundary is documented, failure is visible, and the operational cost is acceptable.
  • Narrow: execution is useful for a lower-risk subset, while known sources, private data, or side effects remain on deterministic paths.
  • No-go for now: the source, authority, runtime, recovery, or data contract is still too unclear.

Measure more than answer quality. Track completion, useful artifact rate, timeout and cancellation behavior, blocked-action tests, human correction, data exposure, runtime cost, and the percentage of jobs that should have used the deterministic baseline. If the feature looks good only when you ignore the jobs that fail, the pilot is telling you to narrow the scope.

What is likely to change next

This is an inference, not a settled fact: code execution is becoming a composable layer in agent products. Provider-hosted interpreters make short analysis easy. Managed sandboxes make custom runtimes and persistent workspaces easier to provision. Security and infrastructure teams are building stronger boundaries around the execution surface. The likely result is not one universal “agent runtime,” but a stack of execution lanes selected per job.

That direction will make the boundary more important, not less. As tools become easier to invoke, teams may be tempted to let the model choose an environment, package source, network path, or credential simply because it can. A mature product will expose those choices as application policy. It will know which tasks are allowed to use a scratch interpreter, which require a managed sandbox, which require a private runtime, and which should stay ordinary software.

The durable advantage will not be that a product can execute code. Many products will be able to do that. The advantage will be knowing when to execute, what to expose, how to recover, and when to say no.

FAQ

Is a provider-hosted code interpreter safe enough for production?

It can be appropriate for a bounded, low-side-effect workflow when its data path, language, runtime, package set, network behavior, retention, and cost fit your requirements. “Safe enough” is not a property of the label. Keep the execution environment separate from your authority layer, validate outputs, and avoid making the provider workspace your only durable record.

Should I use a sandbox or a deterministic function?

Use a deterministic function when the operation is known and has a stable contract. Use a sandbox when the user genuinely needs open-ended computation, custom dependencies, multiple languages, or iterative workspace behavior. If you are unsure, implement the deterministic subset first and let real failures show whether a runtime is necessary.

Is a Docker container enough to sandbox AI-generated code?

It depends on the threat model, configuration, workload, and trust boundary. A container may be appropriate for a constrained, ephemeral task with no sensitive mounts or network access, but “container” is not a universal security guarantee. Consider the kernel boundary, filesystem, privileges, egress, credentials, image supply chain, tenant separation, and monitoring. Higher-risk arbitrary code may justify a stronger isolation primitive or a managed service with a documented boundary.

Can my code-executing agent access internal APIs?

Do not give arbitrary generated code broad internal access by default. If access is necessary, use a narrow broker or typed tool that enforces identity, tenant, resource, fields, rate, and policy. Prefer read-only filtered views for exploration. Keep production writes in a controlled application service with idempotency and explicit approval where the consequence is high.

How do I handle long-running code?

Make it a job with an owner, status, timeout, cancellation path, artifact references, and recovery state. A provider tool or sandbox can be one step in that job, but the runtime should not be the only source of truth. See the background-jobs guide for the broader job-state pattern.

Do I need to self-host the sandbox for compliance?

Not automatically. Start with the required data path, regions, retention, contract, control-plane behavior, network boundary, and audit evidence. A managed service may offer a suitable private deployment or customer-cloud option; self-hosting may give more control but also makes your team responsible for more controls. Ask what must be owned, not what sounds most private.

What should I log from an execution run?

Record a redacted request and job ID, model or tool version, runtime and image version, allowed inputs, code or command references, resource and network policy, outputs and artifacts, exit state, cancellation or retry events, authorization result, and human review result. Avoid retaining secrets or unnecessary customer content in logs.

The practical next step

If you are deciding whether to add code execution, bring one real workflow—not a broad wish list. The useful starting materials are the input examples, the intended artifact, the data path, the side effects, the failure cases, and the boundary you are willing to defend. That is enough to choose a first lane without pretending you need a universal agent runtime.

Start a scoped BishopTech consultation

Sources and further reading

  1. OpenAI — Code Interpreter — current API documentation accessed September 23, 2026; hosted Python execution, files, containers, memory tiers, and expiration behavior.
  2. OpenAI — Pricing — current pricing documentation accessed September 23, 2026; hosted shell and Code Interpreter container billing separate from model tokens.
  3. Anthropic — Code execution tool — current platform documentation accessed September 23, 2026; Python and Bash, tool versions, state persistence, pre-installed libraries, no-internet container behavior, and limits.
  4. Anthropic — Self-hosted sandboxes security — current security documentation accessed September 23, 2026; egress restrictions and separation of trust-boundary workspaces.
  5. Google AI for Developers — Code execution — current API documentation accessed September 23, 2026; Python execution, iterative feedback, runtime and file limits, retries, and billing behavior.
  6. Modal — Sandboxes — current platform documentation accessed September 23, 2026; runtime-created sandboxes, arbitrary commands, custom images, timeouts, readiness, and lifecycle.
  7. Modal — Networking and security — current platform security documentation accessed September 23, 2026; gVisor isolation, default resource separation, and network controls.
  8. E2B — Security and compliance — current first-party security page accessed September 23, 2026; stated Firecracker microVM isolation, lifecycle, state preservation, and BYOC data-path details.
  9. Daytona — Process and Code Execution — current platform documentation accessed September 23, 2026; stateless and stateful execution, multiple languages, shells, sessions, and long-running processes.
  10. Firecracker — Secure and fast microVMs — current project page accessed September 23, 2026; VM-level isolation and jailer defense-in-depth.
  11. NIST — Cybersecurity Framework Profile for Artificial Intelligence — December 2025 initial preliminary draft; curtail, sandbox, approve, monitor, or disallow arbitrary agent code execution.
  12. Cyber Security Agency of Singapore — Securing Agentic AI addendum — June 17, 2026; agentic risk from planning and tool/data access, with controls across autonomy levels.
  13. NIST CAISI — Insights into AI Agent Security from a Large-Scale Red-Teaming Competition — March 23, 2026; agent hijacking through untrusted data and harmful code execution paths.
  14. NIST CAISI — Lessons Learned from the Consortium: Tool Use in Agent Systems — August 2025; read-only, constrained-write, and write tool patterns across trusted and untrusted environments.
  15. Reddit r/aiagents — How are you guys handling enterprise InfoSec and sandboxing for agents in production? — May 26, 2026 public practitioner thread; qualitative signal about governance, microVMs, read-only APIs, isolation, and state.
  16. n8n issue #30358 — Code node hangs with AI Agent HTTP Request Tool present — opened May 13, 2026; qualitative implementation signal about runtime initialization and timeout behavior.
  17. Google Gemini CLI issue #24707 — run_shell_command hangs for interactive or slow commands — opened April 5, 2026; qualitative implementation signal about interactive input, hard timeouts, and cancellation.
  18. Reddit r/AI_Agents — I ran AI agents on 5 sandbox setups for 6 weeks. Firecracker won. — May 7, 2026 public practitioner thread; qualitative, self-reported signal about stronger isolation versus operational complexity.