cancel() 之后,任务就消失了吗
取消是一个请求。任务获得执行机会时,会在相应位置收到 CancelledError;清理逻辑仍需要运行。等待任务退出,才能知道收尾走完。
import asyncio
async def generate(rollout_id):
try:
await asyncio.sleep(60)
return rollout_id
finally:
print("cleanup", rollout_id)
async def exercise():
task = asyncio.create_task(generate(10))
await asyncio.sleep(0)
task.cancel()
try:
await task
except asyncio.CancelledError:
print("cancelled")
try:
async with asyncio.timeout(0.01):
await asyncio.sleep(60)
except TimeoutError:
print("timeout")
if __name__ == "__main__":
asyncio.run(exercise())
下载并运行本节 example.py。只使用 Python 标准库;推荐 Python 3.12,最低 Python 3.11。
运行结果:
cleanup 10
cancelled
timeout
示例先让任务进入等待,再请求取消。finally 中的 cleanup 先出现,等待者随后捕获取消。第二段的 asyncio.timeout 为一个代码块设定超时,在块外捕获 TimeoutError。本实验要求 Python 3.11 及以上。
Slime 为什么取消之后还 gather
_GenerationActor._next() 同时等待取数任务和生产者。无论结果、异常或取消从哪里返回,finally 都取消不再需要的取数任务,再等待它收尾,避免它以后取走不属于本次请求的数据。
展开 Slime 源码:_GenerationActor._next
_GenerationActor._nextslime/rollout/fully_async_distributed.py · L305–L317
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码
async def _next(self):
take = asyncio.create_task(self.outputs.get())
try:
await asyncio.wait((take, self.producer), return_when=asyncio.FIRST_COMPLETED)
if take.done():
return take.result()
self.producer.result() # Propagate custom generation errors.
if not self.outputs.empty():
return await take
return None
finally:
take.cancel()
await asyncio.gather(take, return_exceptions=True)展开 Slime 源码:generate
generateexamples/coding_agent_rl/generate.py · L201–L211
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码
try:
async with asyncio.timeout(CONFIG.rollout_guard_sec):
async with boot_agent_sandbox(md["image"], instance_id) as sb:
await swe.prepare_workspace(sb, md["workdir"], md)
agent_exit_code = await HARNESS_CLS().run(
sb,
workdir=md["workdir"],
session_id=session_id,
adapter_url=state.adapter_url,
time_budget_sec=CONFIG.agent_time_budget_sec,
prompt=swe.SWE_PROMPT,三种“等到时间”的区别
| 写法 | 到时发生什么 |
|---|---|
wait(tasks, timeout=t) | 返回 done/pending,pending 不会自动取消 |
wait_for(awaitable, timeout=t) | 通常取消等待对象,并等待取消处理;实际耗时可超过 t |
async with asyncio.timeout(t) | 用当前任务的取消机制限制一个代码块,在作用域外表现为 TimeoutError |
清理后一般继续传播 CancelledError。把它吞掉并返回正常结果,会让上层误判任务成功。shield 保护等待对象免受这一层取消传播,但外层等待者仍然可以被取消;它不是“全程不可能取消”。
shield 保护谁,取消后谁来收尾
把 await pending 换成 await asyncio.shield(pending),改变的是这一层等待的取消传播:调用方被取消时,仍会收到 CancelledError,但这次取消不会通过 shield 顺带取消里面的 pending。pending 仍可能被其他代码直接取消,所以这不代表它永远不会停止。
Slime 的 run_rollout_io() 保留 pending 的引用。调用方取消之后,它继续等已经接受的工作完成,再取结果、释放许可,最后重新抛出取消。这里取消的是“我在等待”,不是“已接受的工作已经安全回收”。线程里究竟做什么,第 8 节再展开。
| 阶段 | 调用方 | pending |
|---|---|---|
| 正常等待 | 在 await shield(pending) 处挂起 | 可以继续推进 |
| 调用方收到取消 | 进入 except CancelledError | 不被这一层取消顺带取消 |
| 收尾 | 继续等待并检查结果 | 完成或报告自己的错误 |
| 返回上层 | 传播取消或实际工作错误 | 不留下无人负责的已接受工作 |
展开 Slime 源码:取消之后继续等待已接受的工作
run_rollout_ioslime/data/transport.py · L368–L389
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码
async def run_rollout_io(args, function, *values, **kwargs):
"""Bound the executor backlog before submitting encoding/sync work."""
loop = asyncio.get_running_loop()
key = (os.getpid(), loop)
if key not in _async_limits:
_async_limits[key] = asyncio.Semaphore(getattr(args, "rollout_io_concurrency", 4))
async with _async_limits[key]:
pending = asyncio.create_task(asyncio.to_thread(function, *values, **kwargs))
try:
return await asyncio.shield(pending)
except asyncio.CancelledError:
# Repeated cancellation still cannot stop a filesystem commit.
# Keep capacity and input ownership until the accepted write finishes.
while not pending.done():
try:
await asyncio.shield(pending)
except asyncio.CancelledError:
pass
pending.result()
raiseasyncio.TimeoutError 在 Python 3.11 以后是内置 TimeoutError 的别名,源码两种拼法表达同一异常类型;本教程使用 Python 3.11 及以上。wait(timeout=...)、wait_for(...) 和 timeout(...) 的差别仍要看上表,不能只看异常名字。
复述检查
cancel() 发出请求;await task 或清理用的 gather 等待请求被处理。超时预算还可能包含清理时间,不是强行停止线程或外部进程的按钮。
本节对应的全部源码位置
下面把同类用法在 Slime 中的每个扫描位置接回本节讲解;点击文件名可查看完整源码快照。重复出现的 API 共用解释,具体输入与清理由所在函数决定。
展开本节 29 条源码记录
| 记录 / 讲解 | 源码位置 | 原代码 |
|---|---|---|
| occ-000021 | examples/coding_agent_rl/generate.py:202 generate | async with asyncio.timeout(CONFIG.rollout_guard_sec): |
| occ-000028 | examples/coding_agent_rl/generate.py:270 generate | except asyncio.TimeoutError: |
| occ-000245 | examples/search-r1/google_search_server.py:59 fetch | except (aiohttp.ClientError, asyncio.TimeoutError): |
| occ-000390 | slime/agent/adapters/common.py:364 BaseAdapter._run_turn | except (ConnectionResetError, asyncio.CancelledError) as e: |
| occ-000391 | slime/agent/adapters/common.py:372 BaseAdapter._run_turn | if isinstance(e, asyncio.CancelledError): |
| occ-000409 | slime/agent/adapters/common.py:502 call_sglang_generate | except (asyncio.CancelledError, aiohttp.ClientError, asyncio.TimeoutError) as e: |
| occ-000410 | slime/agent/adapters/common.py:502 call_sglang_generate | except (asyncio.CancelledError, aiohttp.ClientError, asyncio.TimeoutError) as e: |
| occ-000616 | slime/backends/megatron_utils/server/megatron_server.py:471 _build_http_app.update_from_disk | result = await asyncio.shield(update_future) |
| occ-000636 | slime/backends/megatron_utils/server/megatron_server.py:585 _build_http_app.generate | except asyncio.CancelledError: |
| occ-000696 | slime/backends/sglang_utils/server_control.py:153 _abort_server_once | await asyncio.wait_for( |
| occ-000699 | slime/backends/sglang_utils/server_control.py:160 _get_server_load | return await asyncio.wait_for( |
| occ-000842 | slime/data/transport.py:379 run_rollout_io | return await asyncio.shield(pending) |
| occ-000843 | slime/data/transport.py:380 run_rollout_io | except asyncio.CancelledError: |
| occ-000846 | slime/data/transport.py:385 run_rollout_io | await asyncio.shield(pending) |
| occ-000847 | slime/data/transport.py:386 run_rollout_io | except asyncio.CancelledError: |
| occ-000997 | slime/rollout/fully_async_distributed.py:405 _GenerationActor._rebuffer | await asyncio.shield(future) |
| occ-001059 | slime/rollout/fully_async_rollout.py:245 AsyncRolloutWorker._make_done_cb._cb | except asyncio.CancelledError: |
| occ-001127 | slime/rollout/sglang_rollout.py:268 _run_request_abortable_generate | except asyncio.CancelledError: |
| occ-001546 | tests/test_agent/test_agent_rollout_cpu.py:62 <module> | asyncio.timeout = _timeout_shim |
| occ-001730 | tests/test_distributed_rollout.py:1009 test_worker_refills_locally_and_slow_group_does_not_block_results.exercise | first = await asyncio.wait_for(worker._next(), timeout=5) |
| occ-001732 | tests/test_distributed_rollout.py:1010 test_worker_refills_locally_and_slow_group_does_not_block_results.exercise | second = await asyncio.wait_for(worker._next(), timeout=5) |
| occ-001735 | tests/test_distributed_rollout.py:1017 test_worker_refills_locally_and_slow_group_does_not_block_results.exercise | output = await asyncio.wait_for(worker._next(), timeout=5) |
| occ-001743 | tests/test_distributed_rollout.py:1029 test_worker_refills_locally_and_slow_group_does_not_block_results.exercise | while (output := await asyncio.wait_for(worker._next(), timeout=5)) is not None: |
| occ-001964 | tests/test_rollout_straw_transport.py:409 test_synchronous_rollout_stores_during_generation_and_preserves_legacy_hook.exercise | return await asyncio.wait_for(sr.generate_rollout_async(args, 0, lambda count: [None] * count), timeout=20) |
| occ-002114 | tests/test_straw_r3.py:175 test_cancelled_straw_publication_drains_before_releasing_capacity.run | assert isinstance(results[0], asyncio.CancelledError) |
| occ-002175 | tests/test_streaming_rollout.py:438 test_unrelated_stream_cancellation_propagates.exercise | with pytest.raises(asyncio.CancelledError): |
| occ-002200 | tests/test_streaming_rollout.py:573 test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise | await asyncio.wait_for(first_batch.wait(), 1) |
| occ-002204 | tests/test_streaming_rollout.py:578 test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise | with pytest.raises(asyncio.CancelledError): |
| occ-002207 | tests/test_streaming_rollout.py:581 test_generation_pacing_preserves_concurrency_cancellation_and_reuse.exercise | await asyncio.wait_for(all_started.wait(), 1) |
专题首页 · 上一节 · 下一节:06 · Semaphore:限制同时进入的工作数量 · 源码与教程对照 · 全部用法