I tried to use ray to speed up my program. But here is some question. Here is the example code,it is just used to demonstrate the question.
with open(complete_json_data_path, "r", encoding="utf-8") as comf:
complete_datas = json.load(comf)
@ray.remote
def test(result):
print(id(result))
print("start one process")
for i in tqdm(range(100)):
time.sleep(1)
print("begin first")
time.sleep(4)
print("begin second")
time.sleep(4)
print("end")
return [1],[2]
cpu_count = 4
ray.init(num_cpus=cpu_count)
list_of_incomplete_datas = split_list_into_n_parts(incomplete_datas, cpu_count)
result_ids = [test.remote(complete_datas) for _ in range(4)]
result = []
not_find_result = []
for result_id in result_ids:
result_, not_find_result_ = ray.get(result_id)
result.extend(result_)
not_find_result.extend(not_find_result_)
In this demo,when i run it four processes will execute sequentially in series, with the same PID why?
I want to know what caused this result. If I put the code to open the file into the remote function, the program will run in parallel normally.