several test failures under ruby3.1

[Antonio Terceiro]

Hi,

When running the tests from the current git master branch, I get several test failures under ruby3.1:1

root@ruby3.1:/packages/upstream/ruby-prof# ruby3.1 -S rake test
/usr/bin/ruby3.1 -w -I"lib:lib:ext:test" /usr/lib/ruby/gems/3.1.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/abstract_printer_test.rb" "test/alias_test.rb" "test/basic_test.rb" "test/call_tree_visitor_test.rb" "test/call_trees_test.rb" "test/duplicate_names_test.rb" "test/dynamic_method_test.rb" "test/enumerable_test.rb" "test/exceptions_test.rb" "test/exclude_methods_test.rb" "test/exclude_threads_test.rb" "test/fiber_test.rb" "test/gc_test.rb" "test/inverse_call_tree_test.rb" "test/line_number_test.rb" "test/marshal_test.rb" "test/measure_allocations_test.rb" "test/measure_allocations_trace_test.rb" "test/measure_memory_trace_test.rb" "test/measure_process_time_test.rb" "test/measure_wall_time_test.rb" "test/multi_printer_test.rb" "test/no_method_class_test.rb" "test/pause_resume_test.rb" "test/printer_call_stack_test.rb" "test/printer_call_tree_test.rb" "test/printer_flat_test.rb" "test/printer_graph_html_test.rb" "test/printer_graph_test.rb" "test/printers_test.rb" "test/printing_recursive_graph_test.rb" "test/profile_test.rb" "test/rack_test.rb" "test/recursive_test.rb" "test/singleton_test.rb" "test/stack_printer_test.rb" "test/start_stop_test.rb" "test/thread_test.rb" "test/unique_call_path_test.rb" "test/yarv_test.rb" 
Run options: --seed 29674

# Running:

.F..............F..F.....FFF.F........F.......................F........FF..FFF....F.F............................

Finished in 79.942766s, 1.4135 runs/s, 18.6383 assertions/s.

  1) Failure:
MeasureAllocationsTest#test_allocations [/packages/upstream/ruby-prof/test/measure_allocations_test.rb:24]:
Expected |20 - 36.0| (16.0) to be <= 1.

  2) Failure:
UniqueCallPathTest#test_children_of [/packages/upstream/ruby-prof/test/unique_call_path_test.rb:82]:
Expected: 1
  Actual: 2

  3) Failure:
UniqueCallPathTest#test_unique_path [/packages/upstream/ruby-prof/test/unique_call_path_test.rb:109]:
Expected: 1
  Actual: 2

  4) Failure:
RecursiveTest#test_cycle [/packages/upstream/ruby-prof/test/recursive_test.rb:115]:
Expected: 5
  Actual: 6

  5) Failure:
RecursiveTest#test_simple [/packages/upstream/ruby-prof/test/recursive_test.rb:49]:
Expected: 3
  Actual: 5

  6) Failure:
MeasureMemoryTraceTest#test_memory [/packages/upstream/ruby-prof/test/measure_memory_trace_test.rb:1109]:
Expected |1680 - 1120.0| (560.0) to be <= 1.

  7) Failure:
BugsTest#test_array_push_optimized [/packages/upstream/ruby-prof/test/yarv_test.rb:28]:
--- expected
+++ actual
@@ -1 +1 @@
-["BugsTest#test_array_push_optimized", "BugsTest#array_push_optimized", "Array#push"]
+["BugsTest#test_array_push_optimized", "BugsTest#array_push_optimized", "Array#<<", "Array#push"]


  8) Failure:
MeasureAllocationsTraceTest#test_allocations [/packages/upstream/ruby-prof/test/measure_allocations_trace_test.rb:24]:
Expected |20 - 28.0| (8.0) to be <= 1.

  9) Failure:
DynamicMethodTest#test_dynamic_method [/packages/upstream/ruby-prof/test/dynamic_method_test.rb:51]:
--- expected
+++ actual
@@ -6,4 +6,4 @@
 DynamicMethodTest::FruitMedley#banana
 DynamicMethodTest::FruitMedley#orange
 DynamicMethodTest::FruitMedley#apple
-Symbol#to_s"
+Integer#=="


 10) Failure:
MeasureProcessTimeTest#test_class_methods_busy [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:138]:
Expected: 3
  Actual: 5

 11) Failure:
MeasureProcessTimeTest#test_class_methods_busy_threaded [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:215]:
Expected: 3
  Actual: 5

 12) Failure:
MeasureProcessTimeTest#test_instance_methods_busy [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:440]:
Expected: 5
  Actual: 7

 13) Failure:
MeasureProcessTimeTest#test_instance_methods_busy_block [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:485]:
Expected: 6
  Actual: 8

 14) Failure:
MeasureProcessTimeTest#test_instance_methods_busy_threaded [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:583]:
Expected: 5
  Actual: 7

 15) Failure:
MeasureProcessTimeTest#test_module_instance_methods_busy [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:747]:
Expected: 5
  Actual: 7

 16) Failure:
MeasureProcessTimeTest#test_module_methods_busy [/packages/upstream/ruby-prof/test/measure_process_time_test.rb:665]:
Expected: 3
  Actual: 5

113 runs, 1490 assertions, 16 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1): [ruby -w -I"lib:lib:ext:test" /usr/lib/ruby/gems/3.1.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/abstract_printer_test.rb" "test/alias_test.rb" "test/basic_test.rb" "test/call_tree_visitor_test.rb" "test/call_trees_test.rb" "test/duplicate_names_test.rb" "test/dynamic_method_test.rb" "test/enumerable_test.rb" "test/exceptions_test.rb" "test/exclude_methods_test.rb" "test/exclude_threads_test.rb" "test/fiber_test.rb" "test/gc_test.rb" "test/inverse_call_tree_test.rb" "test/line_number_test.rb" "test/marshal_test.rb" "test/measure_allocations_test.rb" "test/measure_allocations_trace_test.rb" "test/measure_memory_trace_test.rb" "test/measure_process_time_test.rb" "test/measure_wall_time_test.rb" "test/multi_printer_test.rb" "test/no_method_class_test.rb" "test/pause_resume_test.rb" "test/printer_call_stack_test.rb" "test/printer_call_tree_test.rb" "test/printer_flat_test.rb" "test/printer_graph_html_test.rb" "test/printer_graph_test.rb" "test/printers_test.rb" "test/printing_recursive_graph_test.rb" "test/profile_test.rb" "test/rack_test.rb" "test/recursive_test.rb" "test/singleton_test.rb" "test/stack_printer_test.rb" "test/start_stop_test.rb" "test/thread_test.rb" "test/unique_call_path_test.rb" "test/yarv_test.rb" ]

Tasks: TOP => test
(See full trace by running task with --trace)
root@ruby3.1:/packages/upstream/ruby-prof# ruby3.1 -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux-gnu]

[Charlie Savage]

Yup...looks like Ruby 3.1 sends trace events for more methods like Integer#==.

Added 3.1 to the test suite. 5300eb6

[Charlie Savage]

Test now mostly pass on 3.1 and 3.2 except for MacOS whose timings are significantly off (I assume due to the way github runs MacOS). So going to close this.