cancel() 之后,任务就消失了吗

取消是一个请求。任务获得执行机会时,会在相应位置收到 CancelledError;清理逻辑仍需要运行。等待任务退出,才能知道收尾走完。

import asyncio

async def generate(rollout_id):
    try:
        await asyncio.sleep(60)
        return rollout_id
    finally:
        print("cleanup", rollout_id)

async def exercise():
    task = asyncio.create_task(generate(10))
    await asyncio.sleep(0)
    task.cancel()
    try:
        await task
    except asyncio.CancelledError:
        print("cancelled")
    try:
        async with asyncio.timeout(0.01):
            await asyncio.sleep(60)
    except TimeoutError:
        print("timeout")

if __name__ == "__main__":
    asyncio.run(exercise())

下载并运行本节 example.py。只使用 Python 标准库;推荐 Python 3.12,最低 Python 3.11。

运行结果:

cleanup 10
cancelled
timeout

示例先让任务进入等待,再请求取消。finally 中的 cleanup 先出现,等待者随后捕获取消。第二段的 asyncio.timeout 为一个代码块设定超时,在块外捕获 TimeoutError。本实验要求 Python 3.11 及以上。

Slime 为什么取消之后还 gather

_GenerationActor._next() 同时等待取数任务和生产者。无论结果、异常或取消从哪里返回,finally 都取消不再需要的取数任务,再等待它收尾,避免它以后取走不属于本次请求的数据。

展开 Slime 源码:_GenerationActor._next

_GenerationActor._next
slime/rollout/fully_async_distributed.py · L305–L317
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码

    async def _next(self):
        take = asyncio.create_task(self.outputs.get())
        try:
            await asyncio.wait((take, self.producer), return_when=asyncio.FIRST_COMPLETED)
            if take.done():
                return take.result()
            self.producer.result()  # Propagate custom generation errors.
            if not self.outputs.empty():
                return await take
            return None
        finally:
            take.cancel()
            await asyncio.gather(take, return_exceptions=True)
展开 Slime 源码:generate

generate
examples/coding_agent_rl/generate.py · L201–L211
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码

    try:
        async with asyncio.timeout(CONFIG.rollout_guard_sec):
            async with boot_agent_sandbox(md["image"], instance_id) as sb:
                await swe.prepare_workspace(sb, md["workdir"], md)
                agent_exit_code = await HARNESS_CLS().run(
                    sb,
                    workdir=md["workdir"],
                    session_id=session_id,
                    adapter_url=state.adapter_url,
                    time_budget_sec=CONFIG.agent_time_budget_sec,
                    prompt=swe.SWE_PROMPT,

三种“等到时间”的区别

写法到时发生什么
wait(tasks, timeout=t)返回 done/pending,pending 不会自动取消
wait_for(awaitable, timeout=t)通常取消等待对象,并等待取消处理;实际耗时可超过 t
async with asyncio.timeout(t)用当前任务的取消机制限制一个代码块,在作用域外表现为 TimeoutError

清理后一般继续传播 CancelledError。把它吞掉并返回正常结果,会让上层误判任务成功。shield 保护等待对象免受这一层取消传播,但外层等待者仍然可以被取消;它不是“全程不可能取消”。

shield 保护谁,取消后谁来收尾

把 await pending 换成 await asyncio.shield(pending),改变的是这一层等待的取消传播:调用方被取消时,仍会收到 CancelledError,但这次取消不会通过 shield 顺带取消里面的 pending。pending 仍可能被其他代码直接取消,所以这不代表它永远不会停止。

Slime 的 run_rollout_io() 保留 pending 的引用。调用方取消之后,它继续等已经接受的工作完成,再取结果、释放许可,最后重新抛出取消。这里取消的是“我在等待”,不是“已接受的工作已经安全回收”。线程里究竟做什么,第 8 节再展开。

阶段调用方pending
正常等待在 await shield(pending) 处挂起可以继续推进
调用方收到取消进入 except CancelledError不被这一层取消顺带取消
收尾继续等待并检查结果完成或报告自己的错误
返回上层传播取消或实际工作错误不留下无人负责的已接受工作
展开 Slime 源码:取消之后继续等待已接受的工作

run_rollout_io
slime/data/transport.py · L368–L389
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码



async def run_rollout_io(args, function, *values, **kwargs):
    """Bound the executor backlog before submitting encoding/sync work."""
    loop = asyncio.get_running_loop()
    key = (os.getpid(), loop)
    if key not in _async_limits:
        _async_limits[key] = asyncio.Semaphore(getattr(args, "rollout_io_concurrency", 4))
    async with _async_limits[key]:
        pending = asyncio.create_task(asyncio.to_thread(function, *values, **kwargs))
        try:
            return await asyncio.shield(pending)
        except asyncio.CancelledError:
            # Repeated cancellation still cannot stop a filesystem commit.
            # Keep capacity and input ownership until the accepted write finishes.
            while not pending.done():
                try:
                    await asyncio.shield(pending)
                except asyncio.CancelledError:
                    pass
            pending.result()
            raise

asyncio.TimeoutError 在 Python 3.11 以后是内置 TimeoutError 的别名,源码两种拼法表达同一异常类型;本教程使用 Python 3.11 及以上。wait(timeout=...)、wait_for(...) 和 timeout(...) 的差别仍要看上表,不能只看异常名字。

复述检查

cancel() 发出请求;await task 或清理用的 gather 等待请求被处理。超时预算还可能包含清理时间,不是强行停止线程或外部进程的按钮。

语义核对:任务取消、超时。

本节对应的全部源码位置

下面把同类用法在 Slime 中的每个扫描位置接回本节讲解;点击文件名可查看完整源码快照。重复出现的 API 共用解释,具体输入与清理由所在函数决定。

展开本节 29 条源码记录
记录 / 讲解源码位置原代码
occ-000021examples/coding_agent_rl/generate.py:202
generate
async with asyncio.timeout(CONFIG.rollout_guard_sec):
occ-000028examples/coding_agent_rl/generate.py:270
generate
except asyncio.TimeoutError:
occ-000245examples/search-r1/google_search_server.py:59
fetch
except (aiohttp.ClientError, asyncio.TimeoutError):
occ-000390slime/agent/adapters/common.py:364
BaseAdapter._run_turn
except (ConnectionResetError, asyncio.CancelledError) as e:
occ-000391slime/agent/adapters/common.py:372
BaseAdapter._run_turn
if isinstance(e, asyncio.CancelledError):
occ-000409slime/agent/adapters/common.py:502
call_sglang_generate
except (asyncio.CancelledError, aiohttp.ClientError, asyncio.TimeoutError) as e:
occ-000410slime/agent/adapters/common.py:502
call_sglang_generate
except (asyncio.CancelledError, aiohttp.ClientError, asyncio.TimeoutError) as e:
occ-000616slime/backends/megatron_utils/server/megatron_server.py:471
_build_http_app.update_from_disk
result = await asyncio.shield(update_future)
occ-000636slime/backends/megatron_utils/server/megatron_server.py:585
_build_http_app.generate
except asyncio.CancelledError:
occ-000696slime/backends/sglang_utils/server_control.py:153
_abort_server_once
await asyncio.wait_for(
occ-000699slime/backends/sglang_utils/server_control.py:160
_get_server_load
return await asyncio.wait_for(
occ-000842slime/data/transport.py:379
run_rollout_io
return await asyncio.shield(pending)
occ-000843slime/data/transport.py:380
run_rollout_io
except asyncio.CancelledError:
occ-000846slime/data/transport.py:385
run_rollout_io
await asyncio.shield(pending)
occ-000847slime/data/transport.py:386
run_rollout_io
except asyncio.CancelledError:
occ-000997slime/rollout/fully_async_distributed.py:405
_GenerationActor._rebuffer
await asyncio.shield(future)
occ-001059slime/rollout/fully_async_rollout.py:245
AsyncRolloutWorker._make_done_cb._cb
except asyncio.CancelledError:
occ-001127slime/rollout/sglang_rollout.py:268
_run_request_abortable_generate
except asyncio.CancelledError:
occ-001546tests/test_agent/test_agent_rollout_cpu.py:62
<module>
asyncio.timeout = _timeout_shim
occ-001730tests/test_distributed_rollout.py:1009
test_worker_refills_locally_and_slow_group_does_not_block_results.exercise
first = await asyncio.wait_for(worker._next(), timeout=5)
occ-001732tests/test_distributed_rollout.py:1010
test_worker_refills_locally_and_slow_group_does_not_block_results.exercise
second = await asyncio.wait_for(worker._next(), timeout=5)
occ-001735tests/test_distributed_rollout.py:1017
test_worker_refills_locally_and_slow_group_does_not_block_results.exercise
output = await asyncio.wait_for(worker._next(), timeout=5)
occ-001743tests/test_distributed_rollout.py:1029
test_worker_refills_locally_and_slow_group_does_not_block_results.exercise
while (output := await asyncio.wait_for(worker._next(), timeout=5)) is not None:
occ-001964tests/test_rollout_straw_transport.py:409
test_synchronous_rollout_stores_during_generation_and_preserves_legacy_hook.exercise
return await asyncio.wait_for(sr.generate_rollout_async(args, 0, lambda count: [None] * count), timeout=20)
occ-002114tests/test_straw_r3.py:175
test_cancelled_straw_publication_drains_before_releasing_capacity.run
assert isinstance(results[0], asyncio.CancelledError)
occ-002175tests/test_streaming_rollout.py:438
test_unrelated_stream_cancellation_propagates.exercise
with pytest.raises(asyncio.CancelledError):
occ-002200tests/test_streaming_rollout.py:573
test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise
await asyncio.wait_for(first_batch.wait(), 1)
occ-002204tests/test_streaming_rollout.py:578
test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise
with pytest.raises(asyncio.CancelledError):
occ-002207tests/test_streaming_rollout.py:581
test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise
await asyncio.wait_for(all_started.wait(), 1)

专题首页 · 上一节 · 下一节:06 · Semaphore:限制同时进入的工作数量 · 源码与教程对照 · 全部用法