ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Home Page:https://ray.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Core] Buffered logs lost sometimes

robertnishihara opened this issue · comments

What happened + What you expected to happen

When running in IPython, some print statements are sometimes suppressed. We buffer them under the hood and deduplicate them (ignoring numerical values), but then they never seem to get printed.

In [1]: import time

In [2]: import ray

In [3]: @ray.remote
   ...: def f(i):
   ...:     print(i)
   ...:

In [4]: [f.remote(i) for i in range(10)]
2024-05-10 18:01:35,772	INFO worker.py:1740 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8266
Out[4]:
[ObjectRef(c8ef45ccd0112571ffffffffffffffffffffffff0100000001000000),
 ObjectRef(16310a0f0a45af5cffffffffffffffffffffffff0100000001000000),
 ObjectRef(c2668a65bda616c1ffffffffffffffffffffffff0100000001000000),
 ObjectRef(32d950ec0ccf9d2affffffffffffffffffffffff0100000001000000),
 ObjectRef(e0dc174c83599034ffffffffffffffffffffffff0100000001000000),
 ObjectRef(f4402ec78d3a2607ffffffffffffffffffffffff0100000001000000),
 ObjectRef(f91b78d7db9a6593ffffffffffffffffffffffff0100000001000000),
 ObjectRef(82891771158d68c1ffffffffffffffffffffffff0100000001000000),
 ObjectRef(8849b62d89cb30f9ffffffffffffffffffffffff0100000001000000),
 ObjectRef(80e22aed7718a125ffffffffffffffffffffffff0100000001000000)]

(f pid=31071) 7
In [5]: time.sleep(1)

In [6]: time.sleep(5)

In [7]: ray.put(1)
Out[7]: ObjectRef(00ffffffffffffffffffffffffffffffffffffff0100000001e1f505)

In [8]:

In [8]: f.remote(0)
Out[8]: ObjectRef(359ec6ce30d3ca2dffffffffffffffffffffffff0100000001000000)

(f pid=31068) 0
In [9]:

In [9]:
Do you really want to exit ([y]/n)?

Versions / Dependencies

Python 3.11.4
ray, version 2.21.0
macOS 13.5

Reproduction script

See above

Issue Severity

None