LangGraph ToolNode Ignores max_concurrency in Async Execution: Reproduction and Workaround

LangGraph ToolNode Ignores max_concurrency in Async Execution: Reproduction and Workaround

The result is specific but important: with langgraph==1.2.10, direct multi-call ToolNode execution respects RunnableConfig.max_concurrency through graph.invoke(), but graph.ainvoke() can ignore the same ceiling. I sent four tool calls to one minimal ToolNode without a model or API key. At max_concurrency=1, the sync path reached one active tool while the async path reached four. At a limit of two, sync reached two and async still reached four.

The fastest temporary protection is not to convert every tool to sync. Put a shared asyncio.Semaphore around tools that consume the same bounded resource. That can protect a rate-limited API, connection pool, browser session, GPU slot, or non-concurrency-safe dependency. It does not repair ToolNode scheduling, so it should remain a tested application boundary until a released upstream fix passes your regression matrix.

The behavior was reported in LangGraph issue #8517 on August 3, 2026. As of August 4, the issue was still open. External contributors had suggested an async parity test and a bounded gather direction, but there was no maintainer-confirmed release fix to validate.

This page answers one search task:

Why do several asynchronous tools in the same ToolNode start together even though I call graph.ainvoke(..., config={"max_concurrency": 1})?

Scope: do not generalize this to every LangGraph agent path

The experiment has one custom StateGraph, one ToolNode, and one input message containing four tool calls. The finding covers the async batch inside that direct ToolNode path.

It is not evidence that every asynchronous LangGraph agent ignores the setting. The issue author described a separate control involving a create_agent Send-based path, but XBSTACK did not include it in this experiment. Before applying the workaround, identify whether your overlap occurs inside one ToolNode batch, across graph tasks, through Send, between subgraphs, or across several nodes.

For timeouts, retries, repeated side effects, and recovery policy, use LangGraph failure recovery for Tool Error and Timeout. To record when each tool starts and ends, combine this test with LangGraph observability and decision tracing.

Verified offline environment

The run completed on August 4, 2026:

Component	Version
OS	macOS 26.5.2 arm64
Python	3.11.15
langgraph	1.2.10
langgraph-prebuilt	1.1.0
langchain-core	1.5.3
pytest	9.1.1
Model / API key	Not required

The experiment contains the reproduction, workaround, raw output, version matrix, and tests:

experiments/langgraph-toolnode-max-concurrency/
├── README.md
├── requirements.txt
├── repro.py
├── workaround.py
├── tests/
│   └── test_max_concurrency.py
└── results/
    ├── repro-results.json
    ├── repro-results.txt
    └── version-matrix.md

Standalone repository:

langgraph-toolnode-max-concurrency-repro

Minimal reproduction with four calls in one batch

Each tool waits for the same interval and updates a shared active counter. The complete repository also writes trace and JSON evidence; this reduced example shows the core behavior:

import asyncio

from langchain_core.messages import AIMessage
from langchain_core.tools import StructuredTool
from langgraph.graph import END, START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode


class AsyncProbe:
    def __init__(self):
        self.active = 0
        self.max_active = 0

    async def run(self, delay: float = 0.08) -> str:
        self.active += 1
        self.max_active = max(self.max_active, self.active)
        try:
            await asyncio.sleep(delay)
            return "ok"
        finally:
            self.active -= 1


async def main():
    probe = AsyncProbe()
    names = [f"tool_{index}" for index in range(4)]
    tools = []

    for name in names:
        async def tool_coroutine() -> str:
            return await probe.run()

        tools.append(
            StructuredTool.from_function(
                coroutine=tool_coroutine,
                name=name,
                description=f"Probe {name}",
            )
        )

    builder = StateGraph(MessagesState)
    builder.add_node("tools", ToolNode(tools))
    builder.add_edge(START, "tools")
    builder.add_edge("tools", END)
    graph = builder.compile()

    message = AIMessage(
        content="",
        tool_calls=[
            {"id": f"call-{index}", "name": name, "args": {}}
            for index, name in enumerate(names)
        ],
    )

    await graph.ainvoke(
        {"messages": [message]},
        config={"max_concurrency": 1},
    )
    print(probe.max_active)


asyncio.run(main())

On the tested versions, the output is 4, not the expected 1.

Matrix: sync is bounded, async ignores lower limits

Four tools each waited for roughly 80 milliseconds:

Path	Configured limit	Measured maximum active	Respected
invoke()	1	1	yes
ainvoke()	1	4	no
invoke()	2	2	yes
ainvoke()	2	4	no
invoke()	4	4	yes
ainvoke()	4	4	superficially, because the limit equals the call count

Raw timing:

sync  max_concurrency=1 max_active=1 elapsed=0.3461s
async max_concurrency=1 max_active=4 elapsed=0.0877s
sync  max_concurrency=2 max_active=2 elapsed=0.1727s
async max_concurrency=2 max_active=4 elapsed=0.0845s
sync  max_concurrency=4 max_active=4 elapsed=0.0913s
async max_concurrency=4 max_active=4 elapsed=0.0953s

Timing depends on the machine and should not be the only evidence. The active counter and trace are stronger: under a limit of one, the sync trace alternates start/end one tool at a time; the async trace records all four starts before the first completion.

Root cause: configured executor versus unconditional gather

The installed package and current ToolNode source show the same split.

The sync _func() uses the configured executor:

with get_executor_for_config(config) as executor:
    outputs = list(
        executor.map(self._run_one, tool_calls, input_types, tool_runtimes)
    )

get_executor_for_config(config) reads the concurrency setting, which is why the sync matrix produced maxima of 1, 2, and 4.

The async _afunc() collects every coroutine and sends the batch to asyncio.gather():

coros = []
for call, tool_runtime in zip(tool_calls, tool_runtimes, strict=False):
    coros.append(self._arun_one(call, input_type, tool_runtime))

outputs = await asyncio.gather(*coros)

That path does not use config.get("max_concurrency") to bound starts. asyncio.gather() waits for the full batch, but it does not impose a concurrency ceiling. The RunnableConfig is still propagated into each tool runtime; it simply does not control this batch scheduler.

LangChain Core already exposes gather_with_concurrency(n, *coros). A maintainer-level change could use that helper or an equivalent bounded implementation, followed by parity tests. The final design must still account for cancellation, error propagation, Command output, and compatibility with existing ToolNode behavior.

Why the mismatch matters in production

An extra four-way overlap may look harmless for local pure functions. Production tools usually touch bounded resources:

• provider APIs with per-account or per-tenant concurrency limits;
• PostgreSQL, Redis, HTTP, or browser connection pools;
• one file, device, GPU, or session that is not safe for concurrent use;
• paid quotas shared across several tool names;
• write operations that are not idempotent under overlap.

A team can test the sync path, see max_concurrency=1 work, then move production to ainvoke() and assume the resource remains protected. A single model response containing several calls can instead hit the provider simultaneously, causing 429 responses, pool exhaustion, out-of-order side effects, or retry amplification. Pair the concurrency test with the failure recovery and retry boundary guide rather than adding retries blindly.

Temporary workaround: one shared Semaphore per bounded resource

Wrap every tool that consumes the same resource with the same Semaphore:

import asyncio

api_semaphore = asyncio.Semaphore(1)


def limit_tool(coroutine):
    async def guarded(*args, **kwargs):
        async with api_semaphore:
            return await coroutine(*args, **kwargs)
    return guarded


@limit_tool
async def search_customers(query: str) -> str:
    return await customer_api.search(query)


@limit_tool
async def search_orders(query: str) -> str:
    return await order_api.search(query)

The important property is sharing. If two tools consume the same provider quota but each creates its own Semaphore(1), the combined system can still run two calls at once.

The local suite covers:

• limits 1, 2, and 4;
• measured maxima 1, 2, and 4 with the wrapper;
• one tool raising RuntimeError;
• active returning to zero after all calls;
• the Semaphore not remaining locked;
• 11 passed overall.

async with semaphore releases the permit when the coroutine returns or raises. The tool still needs its own timeout, idempotency, and error policy. A Semaphore is not a replacement for those controls.

Workarounds that do not solve this exact path

Locking the outer ainvoke() call

An outer lock can limit how many whole graph runs overlap. It does not limit four tool calls inside one graph run, which is the behavior reproduced here.

Converting all tools to sync

The sync path respected the limit in this version, but forcing network I/O into sync execution can block workers and reduce useful throughput. It is a diagnostic control, not a default production design.

Relying on output ordering

asyncio.gather() returns results in input order, but ordered output does not mean ordered execution. Every tool can start together while its final result is placed back into the original slot. Side-effect safety depends on the overlap, not only the returned list.

Looking only at average duration

Network variance, caching, and connection reuse distort timing. Record active count, peak count, start/end timestamps, request IDs, and resource keys at the tool boundary. LangGraph observability provides the broader trace context.

What a released fix should prove

A regression matrix should cover at least:

1. max_concurrency=None retaining the intended default behavior;
2. sync and async direct ToolNode batches matching under limits 1, 2, and 4;
3. exception and cancellation behavior remaining defined;
4. ToolMessage, Command, and multi-output combination remaining correct;
5. each tool continuing to receive the correct ToolRuntime config;
6. no accidental change to graph-level Send or other schedulers;
7. the old reproduction turning into a passing anti-regression test.

Do not remove the resource-level guard only because an issue is closed. Upgrade the released package, rerun the matrix against your actual tool set, and verify the external quota or pool directly.

Upstream status and recommended action

As of August 4, 2026:

• issue #8517 was open and labeled as a bug;
• the report included a self-contained reproduction;
• contributors had proposed async parity coverage and bounded gather;
• no maintainer-confirmed released version was available for this article to verify.

A practical sequence is:

1. run repro.py on your installed version;
2. share one Semaphore across tools using the same bounded resource;
3. assert max_active in tests instead of only asserting successful output;
4. monitor 429s, pool exhaustion, and duplicate writes;
5. rerun the matrix after an official release before removing the workaround.

For worker handoffs, continue with LangGraph Supervisor/Worker Handoff. For state that must survive interrupted or concurrent runs, use LangGraph Checkpointer with MemorySaver, SQLite, and Redis. The LangGraph hub connects the rest of the production series.

Reproduce and verify

cd experiments/langgraph-toolnode-max-concurrency
python3.11 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python repro.py \
  --json-output results/repro-results.json \
  --text-output results/repro-results.txt
.venv/bin/python -m pytest -q

Current result:

11 passed in 1.11s

After an upstream fix, change the tests that describe the current defect so async limit one expects a measured maximum of one. The reproduction should become a permanent regression gate, not a test that preserves the old bug forever.

Topic path / LangGraph

Continue through the production LangGraph learning path

The LangGraph hub organizes state isolation, checkpointing, human approval, retries, observability, supervisors, subgraphs and memory into one reviewable path.

Open LangGraph hub → Start with state isolation Subscribe for updates