工作移到线程后,多了什么责任
线程共享同一个进程中的对象。一个操作由“读、计算、写”几步组成时,多个线程交错执行可能破坏它的整体含义。锁保护的是这段需要一起完成的操作。
import threading
lock = threading.Lock()
ready = threading.Event()
state = {"count": 0}
def worker():
ready.wait()
for _ in range(1000):
with lock:
state["count"] += 1
if __name__ == "__main__":
threads = [threading.Thread(target=worker) for _ in range(2)]
for thread in threads:
thread.start()
ready.set()
for thread in threads:
thread.join()
print("count", state["count"])
下载并运行本节 example.py。只使用 Python 标准库;推荐 Python 3.12,最低 Python 3.11。
运行结果:
count 2000
实验把累加操作放在 with lock 中;Event 负责通知两个线程开始;join() 负责等它们结束。三者分别解决互斥、通知、等待退出,不能互相替代。
回到 Slime 的健康检查线程
RolloutHealthMonitor 使用 Event 表示停止、暂停,使用 Lock 协调检查中的工作。stop() 发出停止信号,调用带超时的 join(),再用 is_alive() 检查是否真的退出。
展开 Slime 源码:RolloutHealthMonitor.stop
RolloutHealthMonitor.stopslime/utils/health_monitor.py · L63–L78
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码
"""Stop the health monitor thread completely. Called during dispose."""
if not self._thread:
return
logger.info("Stopping RolloutHealthMonitor...")
self._stop_event.set()
if timeout is None:
timeout = self._check_timeout + 5
self._thread.join(timeout=timeout)
if self._thread.is_alive():
raise TimeoutError(f"Rollout health monitor did not terminate within {timeout:.1f}s")
else:
logger.info("RolloutHealthMonitor stopped.")
self._thread = None
展开 Slime 源码:QueueReader._heartbeat_loop
QueueReader._heartbeat_loopslime/data/queue_data_source.py · L1047–L1054
快照 2f2318653f6f794dddd321eff7c9d4b7b174643f· 查看固定版本源码
def _heartbeat_loop(self):
interval = max(0.1, getattr(self.args, "rollout_queue_lease_seconds", 300) / 3)
while not self._stop.wait(interval):
with self._lock:
leases = list(self._leases.values())
try:
for offset in range(0, len(leases), 128):
batch = leases[offset : offset + 128]单独执行 join(timeout=...),只能说明等待时间结束;线程可能仍然活着。把线程设成 daemon,也不能替代正常关闭流程。
停止通知怎样保存在 Event 中
| 方法 | 这里表达的意思 |
|---|---|
event.set() | 把标记设为真,通知等待这个标记的线程 |
event.clear() | 把标记恢复为假,供下一轮使用 |
event.is_set() | 只查看标记,不等待,也不消耗通知 |
event.wait(timeout) | 等标记变真;返回值说明标记是否在等待结束时为真 |
例如 _stop.wait(interval) 既可等待检查间隔,也可在停止标记设置后提前返回。Event 保存一个布尔标记,多次 set() 不会积累成多份消息;需要逐项保存结果时仍用 Queue。
threading.Thread(target=...).start() 启动线程,直接调用目标函数仍在当前线程执行。threading.get_ident() 取得当前线程标识,可用于 Slime 判断是否在同一线程重复进入某个区域;它不是进程编号,线程退出后标识也可能被复用。
要给共享状态加锁,可使用 with lock 自动配对获取和释放;显式 acquire(timeout=...) 则要检查返回值,只有获取成功后才可 release()。后面的第 16 节再展开 RLock、Condition 与 Timer。
两种 Lock 分别属于哪里
threading.Lock协调 OS 线程;等待会阻塞调用它的线程。asyncio.Lock协调一个事件循环中的任务;等待通过异步方式进行,不用于跨线程互斥。RLock允许同一线程重入同一把锁,但不会自动防止多把锁互相等待。
复述检查
停止 Event 是通知;线程自身必须检查这个通知并返回。join 等待它退出,is_alive 才能检查超时后是否仍在运行。
语义核对:threading。
本节对应的全部源码位置
下面把同类用法在 Slime 中的每个扫描位置接回本节讲解;点击文件名可查看完整源码快照。重复出现的 API 共用解释,具体输入与清理由所在函数决定。
展开本节 120 条源码记录
| 记录 / 讲解 | 源码位置 | 原代码 |
|---|---|---|
| occ-000468 | slime/agent/aiohttp_threaded.py:6 <module> | import threading |
| occ-000469 | slime/agent/aiohttp_threaded.py:29 AppHandle | thread: threading.Thread |
| occ-000477 | slime/agent/aiohttp_threaded.py:61 run_app_in_thread | started = threading.Event() |
| occ-000483 有接收者线索 | slime/agent/aiohttp_threaded.py:80 run_app_in_thread._run | started.set() |
| occ-000485 有接收者线索 | slime/agent/aiohttp_threaded.py:84 run_app_in_thread._run | started.set() |
| occ-000486 | slime/agent/aiohttp_threaded.py:87 run_app_in_thread | thread = threading.Thread(target=_run, name=thread_name, daemon=True) |
| occ-000487 有接收者线索 | slime/agent/aiohttp_threaded.py:88 run_app_in_thread | thread.start() |
| occ-000488 有接收者线索 | slime/agent/aiohttp_threaded.py:89 run_app_in_thread | started.wait(timeout=start_timeout_sec) |
| occ-000569 | slime/backends/megatron_utils/server/megatron_server.py:3 <module> | import threading |
| occ-000638 | slime/backends/megatron_utils/server/megatron_server.py:616 _start_http_server | stats_thread = threading.Thread( |
| occ-000639 有接收者线索 | slime/backends/megatron_utils/server/megatron_server.py:622 _start_http_server | stats_thread.start() |
| occ-000734 | slime/data/archive.py:56 RolloutArchive.save | import threading |
| occ-000747 | slime/data/queue_data_source.py:16 <module> | import threading |
| occ-000754 | slime/data/queue_data_source.py:147 RolloutQueueController.__init__ | self._producer_lock = threading.Lock() |
| occ-000756 | slime/data/queue_data_source.py:154 RolloutQueueController.__init__ | self._gc_stop = threading.Event() |
| occ-000757 | slime/data/queue_data_source.py:184 RolloutQueueController._start_gc | self._gc_thread = threading.Thread(target=self._gc_loop, name="straw-gc", daemon=True) |
| occ-000758 有接收者线索 | slime/data/queue_data_source.py:185 RolloutQueueController._start_gc | self._gc_thread.start() |
| occ-000759 有接收者线索 | slime/data/queue_data_source.py:200 RolloutQueueController._gc_loop | while not self._gc_stop.wait(60): |
| occ-000760 有接收者线索 | slime/data/queue_data_source.py:205 RolloutQueueController._gc_loop | self._gc_stop.set() |
| occ-000795 有接收者线索 | slime/data/queue_data_source.py:970 RolloutQueueController.close | self._gc_stop.set() |
| occ-000796 有接收者线索 | slime/data/queue_data_source.py:972 RolloutQueueController.close | self._gc_thread.join() |
| occ-000801 | slime/data/queue_data_source.py:1036 QueueReader.__init__ | self._stop = threading.Event() |
| occ-000802 | slime/data/queue_data_source.py:1044 QueueReader.__init__ | self._heartbeats = threading.Thread(target=self._heartbeat_loop, name="queue-leases", daemon=True) |
| occ-000803 有接收者线索 | slime/data/queue_data_source.py:1045 QueueReader.__init__ | self._heartbeats.start() |
| occ-000804 有接收者线索 | slime/data/queue_data_source.py:1049 QueueReader._heartbeat_loop | while not self._stop.wait(interval): |
| occ-000815 有接收者线索 | slime/data/queue_data_source.py:1288 QueueReader.close | self._stop.set() |
| occ-000816 有接收者线索 | slime/data/queue_data_source.py:1289 QueueReader.close | self._heartbeats.join(timeout=5) |
| occ-000824 | slime/data/transport.py:8 <module> | import threading |
| occ-000825 | slime/data/transport.py:25 <module> | _writers_lock = threading.Lock() |
| occ-000911 | slime/rollout/fully_async_distributed.py:6 <module> | import threading |
| occ-000924 | slime/rollout/fully_async_distributed.py:177 RolloutScheduler.generate | self.thread = threading.Thread(target=self._run, name="rollout-collect", daemon=True) |
| occ-000925 有接收者线索 | slime/rollout/fully_async_distributed.py:178 RolloutScheduler.generate | self.thread.start() |
| occ-000934 有接收者线索 | slime/rollout/fully_async_distributed.py:250 RolloutScheduler.close | self.thread.join() |
| occ-001024 | slime/rollout/fully_async_rollout.py:34 <module> | import threading |
| occ-001030 | slime/rollout/fully_async_rollout.py:99 AsyncRolloutWorker.__init__ | self.worker_thread: threading.Thread | None = None |
| occ-001032 | slime/rollout/fully_async_rollout.py:106 AsyncRolloutWorker.start | self.worker_thread = threading.Thread(target=self._thread_main, name="fully-async-rollout", daemon=True) |
| occ-001033 有接收者线索 | slime/rollout/fully_async_rollout.py:107 AsyncRolloutWorker.start | self.worker_thread.start() |
| occ-001036 有接收者线索 | slime/rollout/fully_async_rollout.py:138 AsyncRolloutWorker.close | self.worker_thread.join(timeout=35) |
| occ-001037 有接收者线索 | slime/rollout/fully_async_rollout.py:139 AsyncRolloutWorker.close | if self.worker_thread.is_alive(): |
| occ-001191 | slime/utils/accelerator/__init__.py:14 <module> | import threading |
| occ-001197 | slime/utils/async_utils.py:2 <module> | import threading |
| occ-001208 | slime/utils/async_utils.py:49 AsyncLoopThread.__init__ | self._thread = threading.Thread(target=self._start_loop, daemon=True) |
| occ-001209 有接收者线索 | slime/utils/async_utils.py:50 AsyncLoopThread.__init__ | self._thread.start() |
| occ-001218 | slime/utils/health_monitor.py:2 <module> | import threading |
| occ-001219 | slime/utils/health_monitor.py:27 RolloutHealthMonitor.__init__ | self._stop_event = threading.Event() |
| occ-001220 | slime/utils/health_monitor.py:28 RolloutHealthMonitor.__init__ | self._pause_event = threading.Event() |
| occ-001221 有接收者线索 | slime/utils/health_monitor.py:29 RolloutHealthMonitor.__init__ | self._pause_event.set() # Engines may be offloaded before the first rollout. |
| occ-001222 | slime/utils/health_monitor.py:34 RolloutHealthMonitor.__init__ | self._check_lock = threading.Lock() |
| occ-001223 有接收者线索 | slime/utils/health_monitor.py:50 RolloutHealthMonitor.start | self._stop_event.clear() |
| occ-001224 有接收者线索 | slime/utils/health_monitor.py:51 RolloutHealthMonitor.start | self._pause_event.set() |
| occ-001225 | slime/utils/health_monitor.py:53 RolloutHealthMonitor.start | self._thread = threading.Thread( |
| occ-001226 有接收者线索 | slime/utils/health_monitor.py:58 RolloutHealthMonitor.start | self._thread.start() |
| occ-001227 有接收者线索 | slime/utils/health_monitor.py:68 RolloutHealthMonitor.stop | self._stop_event.set() |
| occ-001228 有接收者线索 | slime/utils/health_monitor.py:71 RolloutHealthMonitor.stop | self._thread.join(timeout=timeout) |
| occ-001229 有接收者线索 | slime/utils/health_monitor.py:72 RolloutHealthMonitor.stop | if self._thread.is_alive(): |
| occ-001230 有接收者线索 | slime/utils/health_monitor.py:82 RolloutHealthMonitor.pause | self._pause_event.set() |
| occ-001231 有接收者线索 | slime/utils/health_monitor.py:84 RolloutHealthMonitor.pause | if not self._check_lock.acquire(timeout=self._check_timeout if timeout is None else timeout): |
| occ-001232 有接收者线索 | slime/utils/health_monitor.py:86 RolloutHealthMonitor.pause | self._check_lock.release() |
| occ-001233 有接收者线索 | slime/utils/health_monitor.py:92 RolloutHealthMonitor.resume | self._pause_event.clear() |
| occ-001234 有接收者线索 | slime/utils/health_monitor.py:95 RolloutHealthMonitor._health_monitor_loop | while not self._stop_event.is_set(): |
| occ-001235 有接收者线索 | slime/utils/health_monitor.py:97 RolloutHealthMonitor._health_monitor_loop | while self._pause_event.is_set() and not self._stop_event.is_set(): |
| occ-001236 有接收者线索 | slime/utils/health_monitor.py:97 RolloutHealthMonitor._health_monitor_loop | while self._pause_event.is_set() and not self._stop_event.is_set(): |
| occ-001237 有接收者线索 | slime/utils/health_monitor.py:98 RolloutHealthMonitor._health_monitor_loop | self._stop_event.wait(timeout=0.5) |
| occ-001238 有接收者线索 | slime/utils/health_monitor.py:100 RolloutHealthMonitor._health_monitor_loop | if self._stop_event.is_set(): |
| occ-001239 有接收者线索 | slime/utils/health_monitor.py:106 RolloutHealthMonitor._health_monitor_loop | if self._stop_event.wait(self._check_first_wait): |
| occ-001240 有接收者线索 | slime/utils/health_monitor.py:109 RolloutHealthMonitor._health_monitor_loop | if self._pause_event.is_set(): |
| occ-001241 有接收者线索 | slime/utils/health_monitor.py:116 RolloutHealthMonitor._health_monitor_loop | if not self._pause_event.is_set() and not self._stop_event.is_set(): |
| occ-001242 有接收者线索 | slime/utils/health_monitor.py:116 RolloutHealthMonitor._health_monitor_loop | if not self._pause_event.is_set() and not self._stop_event.is_set(): |
| occ-001244 有接收者线索 | slime/utils/health_monitor.py:123 RolloutHealthMonitor._health_monitor_loop | if self._stop_event.wait(self._check_interval): |
| occ-001245 有接收者线索 | slime/utils/health_monitor.py:134 RolloutHealthMonitor._run_health_checks | if self._stop_event.is_set(): |
| occ-001246 有接收者线索 | slime/utils/health_monitor.py:136 RolloutHealthMonitor._run_health_checks | if not force and self._pause_event.is_set(): |
| occ-001296 | slime/utils/routed_experts.py:4 <module> | import threading |
| occ-001305 | slime/utils/routed_experts.py:213 RoutedExpertsMicrobatch.__init__ | self._lock = threading.Lock() |
| occ-001311 | slime_plugins/rollout_buffer/buffer.py:6 <module> | import threading |
| occ-001404 | tests/pipeline_rl_test_helpers.py:5 <module> | import threading |
| occ-001408 | tests/pipeline_rl_test_helpers.py:49 GenerationProbe.__init__ | self.thread = threading.Thread(target=self.run, daemon=True) |
| occ-001414 有接收者线索 | tests/pipeline_rl_test_helpers.py:110 GenerationProbe.verify | self.thread.join(timeout=30) |
| occ-001415 有接收者线索 | tests/pipeline_rl_test_helpers.py:111 GenerationProbe.verify | assert not self.thread.is_alive() |
| occ-001417 有接收者线索 | tests/pipeline_rl_test_helpers.py:116 GenerationProbe.verify_flush | self.thread.join(timeout=30) |
| occ-001418 有接收者线索 | tests/pipeline_rl_test_helpers.py:117 GenerationProbe.verify_flush | assert not self.thread.is_alive() |
| occ-001632 | tests/test_distributed_rollout.py:9 <module> | import threading |
| occ-001656 | tests/test_distributed_rollout.py:602 test_slow_task_acquire_does_not_block_async_generation | entered, release = threading.Event(), threading.Event() |
| occ-001657 | tests/test_distributed_rollout.py:602 test_slow_task_acquire_does_not_block_async_generation | entered, release = threading.Event(), threading.Event() |
| occ-001676 | tests/test_distributed_rollout.py:672 scheduler_factory.create | release = threading.Event() |
| occ-001678 有接收者线索 | tests/test_distributed_rollout.py:687 scheduler_factory.create.make_generator.execute | while not release.is_set(): |
| occ-001697 | tests/test_distributed_rollout.py:811 test_slow_worker_control_does_not_block_collection_or_close | entered = threading.Event() |
| occ-001698 有接收者线索 | tests/test_distributed_rollout.py:815 test_slow_worker_control_does_not_block_collection_or_close.stall | entered.set() |
| occ-001702 有接收者线索 | tests/test_distributed_rollout.py:829 test_slow_worker_control_does_not_block_collection_or_close | assert entered.wait(5) |
| occ-001787 | tests/test_distributed_rollout.py:2010 test_background_gc_failure_stops_queue_work_and_fails_close | controller._gc_thread = threading.Thread(target=controller._gc_loop) |
| occ-001788 有接收者线索 | tests/test_distributed_rollout.py:2011 test_background_gc_failure_stops_queue_work_and_fails_close | controller._gc_thread.start() |
| occ-001789 有接收者线索 | tests/test_distributed_rollout.py:2012 test_background_gc_failure_stops_queue_work_and_fails_close | controller._gc_thread.join(5) |
| occ-001790 有接收者线索 | tests/test_distributed_rollout.py:2013 test_background_gc_failure_stops_queue_work_and_fails_close | assert not controller._gc_thread.is_alive() |
| occ-001824 | tests/test_fully_async_rollout.py:23 <module> | import threading |
| occ-001833 | tests/test_fully_async_rollout.py:218 test_done_callback_never_blocks_event_loop_thread | pusher = threading.Thread(target=_push_all, daemon=True) |
| occ-001834 有接收者线索 | tests/test_fully_async_rollout.py:219 test_done_callback_never_blocks_event_loop_thread | pusher.start() |
| occ-001835 有接收者线索 | tests/test_fully_async_rollout.py:220 test_done_callback_never_blocks_event_loop_thread | pusher.join(timeout=30) |
| occ-001836 有接收者线索 | tests/test_fully_async_rollout.py:222 test_done_callback_never_blocks_event_loop_thread | assert not pusher.is_alive(), "done-callback blocked on a full output queue" |
| occ-001863 | tests/test_published_payload_retry.py:5 <module> | import threading |
| occ-001984 | tests/test_score_centering_binary.py:3 <module> | import threading |
| occ-001985 | tests/test_score_centering_binary.py:96 test_http_worker_decodes_off_event_loop | main_thread = threading.get_ident() |
| occ-001986 | tests/test_score_centering_binary.py:101 test_http_worker_decodes_off_event_loop.decode | calls.append(threading.get_ident()) |
| occ-002060 | tests/test_straw_fully_async_recovery.py:15 <module> | import threading |
| occ-002073 | tests/test_straw_fully_async_recovery.py:132 _run_job | thread = threading.Thread(target=generate, daemon=True) |
| occ-002074 有接收者线索 | tests/test_straw_fully_async_recovery.py:133 _run_job | thread.start() |
| occ-002075 | tests/test_straw_fully_async_recovery.py:158 _run_job | threading.Event().wait() |
| occ-002076 有接收者线索 | tests/test_straw_fully_async_recovery.py:158 _run_job | threading.Event().wait() |
| occ-002092 | tests/test_straw_r3.py:2 <module> | import threading |
| occ-002095 | tests/test_straw_r3.py:143 test_cancelled_straw_publication_drains_before_releasing_capacity | entered, release = threading.Event(), threading.Event() |
| occ-002096 | tests/test_straw_r3.py:143 test_cancelled_straw_publication_drains_before_releasing_capacity | entered, release = threading.Event(), threading.Event() |
| occ-002230 | tests/test_training_recovery.py:292 test_recovery_e2e_contacts_dashboard_without_proxy | import threading |
| occ-002233 | tests/test_training_recovery.py:339 test_recovery_e2e_contacts_dashboard_without_proxy | thread = threading.Thread(target=server.serve_forever, daemon=True) |
| occ-002234 有接收者线索 | tests/test_training_recovery.py:340 test_recovery_e2e_contacts_dashboard_without_proxy | thread.start() |
| occ-002236 有接收者线索 | tests/test_training_recovery.py:346 test_recovery_e2e_contacts_dashboard_without_proxy | thread.join(timeout=5) |
| occ-002249 | tools/convert_hf_to_fp8.py:22 <module> | import threading |
| occ-002251 | tools/convert_hf_to_fp8.py:105 ConversionResult.__init__ | self.lock = threading.Lock() |
| occ-002255 | tools/convert_hf_to_int4_direct.py:16 <module> | import threading |
| occ-002257 | tools/convert_hf_to_int4_direct.py:158 ConversionResult.__init__ | self.lock = threading.Lock() |
| occ-002262 | tools/convert_torch_dist_to_hf_parallel.py:8 <module> | import threading |
| occ-002265 | tools/convert_torch_dist_to_hf_parallel.py:43 WrappedStorageReader.__init__ | self._cache_lock = threading.Lock() if max_workers and max_workers > 1 else None |
| occ-002272 | tools/convert_torch_dist_to_hf_parallel.py:284 save_tensors | lock = threading.Lock() |
专题首页 · 上一节 · 下一节:10 · async with、async for 与异步生成器 · 源码与教程对照 · 全部用法