Some fonts may not be processed by convert_ttf_to_sfd_mp.py

[ssotobayashi-m]

Hi, I'm encountering the problem in the title.

For example, if mp.cpu_count()=8 and you have 8 fonts under font_ttfs/test, one font will not be processed.
In this case, process_nums=6 and font_num_per_process=1.

Processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)].

will launch 7 processes, but they will each process one font, so only 7 fonts will be processed in total.

It's not a pretty solution, but it seems to work by having the process with the largest pid handle all the remaining fonts, as shown below:

--- a/data_utils/convert_ttf_to_sfd_mp.py
+++ b/data_utils/convert_ttf_to_sfd_mp.py
@@ -26,8 +26,8 @@ def convert_mp(opts):
     else:
         font_num_per_process = font_num // process_nums
 
-    def process(process_id, font_num_p_process):
-        for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process):
+    def process(process_id, font_num_p_process, last=False):
+        for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process if not last else font_num):
             if i >= font_num:
                 break
 
@@ -72,7 +72,7 @@ def convert_mp(opts):
 
             cur_font.close()
 
-    processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)]
+    processes = [mp.Process(target=process, args=(pid, font_num_per_process, pid==process_nums)) for pid in range(process_nums + 1)]
 
     for p in processes:
         p.start()

[Yizhi Wang]

Hi, thank you for your feedback!
Yes, the partition only works when font_num_per_process > process_nums, i.e., the number of fonts is very big.
It seems to be correct to use the following code:

Processes = [mp.Process(target=process, args=(pid, font_num_per_process + 1)) for pid in range(process_nums)].

I will update it when I have tested it.