ubsan IsAccessibleMemoryRange use of pipe causes tsan false positives
karen-arutyunov opened this issue · comments
When we run our build2 program built on Linux with GCC 9.1.0 with UndefinedBehaviorSanitizer and ThreadSanitizer enabled (both -fsanitize=thread -fsanitize=undefined options are passed to the compile/link commands), then the following diagnostics is produced, that suggests that ThreadSanitizer complains on itself:
==================
WARNING: ThreadSanitizer: data race (pid=16658)
Write of size 8 at 0x7b0400000030 by thread T1:
#0 pipe ../../../../libsanitizer/tsan/tsan_interceptors.cc:1674 (libtsan.so.0+0x300da)
#1 __sanitizer::IsAccessibleMemoryRange(unsigned long, unsigned long) ../../../../libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cc:282 (libubsan.so.1+0x1b15f)
#2 match_impl /home/karen/work/build2/build2/build2/algorithm.cxx:485 (b+0xa713f2)
#3 operator() /home/karen/work/build2/build2/build2/algorithm.cxx:598 (b+0xa71c8b)
#4 thunk<0, 1, 2, 3> /home/karen/work/build2/build2/build2/scheduler.hxx:375 (b+0xa866ef)
#5 task_thunk<build2::match(build2::action, const build2::target&, std::size_t, build2::atomic_count*, bool)::<lambda(const build2::diag_frame*, const build2::target_lock*, build2::target&, std::size_t)>, const build2::diag_frame*&, const build2::target_lock*&, std::reference_wrapper<build2::target>, long unsigned int&> /home/karen/work/build2/build2/build2/scheduler.txx:132 (b+0xa8534e)
#6 build2::scheduler::execute(std::unique_lock<std::mutex>&, build2::scheduler::task_data&) /home/karen/work/build2/build2/build2/scheduler.hxx:650 (b+0x82fa6f)
#7 build2::scheduler::pop_back(build2::scheduler::task_queue&, std::unique_lock<std::mutex>&) /home/karen/work/build2/build2/build2/scheduler.hxx:615 (b+0x82f6e5)
#8 build2::scheduler::wait(unsigned long, std::atomic<unsigned long> const&, build2::scheduler::work_queue) /home/karen/work/build2/build2/build2/scheduler.cxx:54 (b+0x826d92)
#9 build2::wait_guard::wait() /home/karen/work/build2/build2/build2/context.ixx:57 (b+0x917f3a)
#10 match_prerequisite_range<build2::group_prerequisites, std::function<build2::prerequisite_target(build2::action, const build2::target&, const build2::prerequisite&, build2::include_type)> > /home/karen/work/build2/build2/build2/algorithm.cxx:738 (b+0xa824ed)
#11 build2::match_prerequisites(build2::action, build2::target&, std::function<build2::prerequisite_target (build2::action, build2::target const&, build2::prerequisite const&, build2::include_type)> const&, build2::scope const*) /home/karen/work/build2/build2/build2/algorithm.cxx:754 (b+0xa72b30)
#12 build2::match_prerequisites(build2::action, build2::target&, std::function<build2::prerequisite_target (build2::action, build2::target const&, build2::prerequisite const&, build2::include_type)> const&) /home/karen/work/build2/build2/build2/algorithm.ixx:554 (b+0x8bfe7e)
#13 build2::alias_rule::apply(build2::action, build2::target&) const /home/karen/work/build2/build2/build2/rule.cxx:153 (b+0x8bdd2f)
#14 build2::apply_impl(build2::action, build2::target&, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::reference_wrapper<build2::rule const> > const&) /home/karen/work/build2/build2/build2/algorithm.cxx:400 (b+0xa70788)
#15 match_impl /home/karen/work/build2/build2/build2/algorithm.cxx:509 (b+0xa716db)
#16 operator() /home/karen/work/build2/build2/build2/algorithm.cxx:598 (b+0xa71c8b)
#17 thunk<0, 1, 2, 3> /home/karen/work/build2/build2/build2/scheduler.hxx:375 (b+0xa866ef)
#18 task_thunk<build2::match(build2::action, const build2::target&, std::size_t, build2::atomic_count*, bool)::<lambda(const build2::diag_frame*, const build2::target_lock*, build2::target&, std::size_t)>, const build2::diag_frame*&, const build2::target_lock*&, std::reference_wrapper<build2::target>, long unsigned int&> /home/karen/work/build2/build2/build2/scheduler.txx:132 (b+0xa8534e)
#19 build2::scheduler::execute(std::unique_lock<std::mutex>&, build2::scheduler::task_data&) /home/karen/work/build2/build2/build2/scheduler.hxx:650 (b+0x82fa6f)
#20 build2::scheduler::pop_front(build2::scheduler::task_queue&, std::unique_lock<std::mutex>&) /home/karen/work/build2/build2/build2/scheduler.hxx:584 (b+0x82efa6)
#21 build2::scheduler::helper(void*) /home/karen/work/build2/build2/build2/scheduler.cxx:750 (b+0x82cd11)
Previous write of size 8 at 0x7b0400000030 by main thread:
#0 pipe ../../../../libsanitizer/tsan/tsan_interceptors.cc:1674 (libtsan.so.0+0x300da)
#1 __sanitizer::IsAccessibleMemoryRange(unsigned long, unsigned long) ../../../../libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cc:282 (libubsan.so.1+0x1b15f)
#2 build2::file_search(build2::target const&, build2::prerequisite_key const&) /home/karen/work/build2/build2/build2/target.cxx:679 (b+0xa1a393)
#3 build2::search(build2::target const&, build2::prerequisite_key const&) /home/karen/work/build2/build2/build2/algorithm.cxx:33 (b+0xa6ca2a)
#4 build2::search(build2::target const&, build2::prerequisite const&) /home/karen/work/build2/build2/build2/algorithm.ixx:18 (b+0x7d704c)
#5 match_prerequisite_range<build2::group_prerequisites, std::function<build2::prerequisite_target(build2::action, const build2::target&, const build2::prerequisite&, build2::include_type)> > /home/karen/work/build2/build2/build2/algorithm.cxx:729 (b+0xa8214c)
#6 build2::match_prerequisites(build2::action, build2::target&, std::function<build2::prerequisite_target (build2::action, build2::target const&, build2::prerequisite const&, build2::include_type)> const&, build2::scope const*) /home/karen/work/build2/build2/build2/algorithm.cxx:754 (b+0xa72b30)
#7 build2::match_prerequisites(build2::action, build2::target&, std::function<build2::prerequisite_target (build2::action, build2::target const&, build2::prerequisite const&, build2::include_type)> const&) /home/karen/work/build2/build2/build2/algorithm.ixx:554 (b+0x8bfe7e)
#8 build2::alias_rule::apply(build2::action, build2::target&) const /home/karen/work/build2/build2/build2/rule.cxx:153 (b+0x8bdd2f)
#9 build2::apply_impl(build2::action, build2::target&, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::reference_wrapper<build2::rule const> > const&) /home/karen/work/build2/build2/build2/algorithm.cxx:400 (b+0xa70788)
#10 match_impl /home/karen/work/build2/build2/build2/algorithm.cxx:509 (b+0xa716db)
#11 operator() /home/karen/work/build2/build2/build2/algorithm.cxx:598 (b+0xa71c8b)
#12 thunk<0, 1, 2, 3> /home/karen/work/build2/build2/build2/scheduler.hxx:375 (b+0xa866ef)
#13 task_thunk<build2::match(build2::action, const build2::target&, std::size_t, build2::atomic_count*, bool)::<lambda(const build2::diag_frame*, const build2::target_lock*, build2::target&, std::size_t)>, const build2::diag_frame*&, const build2::target_lock*&, std::reference_wrapper<build2::target>, long unsigned int&> /home/karen/work/build2/build2/build2/scheduler.txx:132 (b+0xa8534e)
#14 build2::scheduler::execute(std::unique_lock<std::mutex>&, build2::scheduler::task_data&) /home/karen/work/build2/build2/build2/scheduler.hxx:650 (b+0x82fa6f)
#15 build2::scheduler::pop_back(build2::scheduler::task_queue&, std::unique_lock<std::mutex>&) /home/karen/work/build2/build2/build2/scheduler.hxx:615 (b+0x82f6e5)
#16 build2::scheduler::wait(unsigned long, std::atomic<unsigned long> const&, build2::scheduler::work_queue) /home/karen/work/build2/build2/build2/scheduler.cxx:54 (b+0x826d92)
#17 build2::wait_guard::wait() /home/karen/work/build2/build2/build2/context.ixx:57 (b+0x917f3a)
#18 build2::match(butl::small_vector<build2::value, 1ul> const&, build2::action, build2::action_targets&, unsigned short, bool) /home/karen/work/build2/build2/build2/operation.cxx:186 (b+0x910893)
#19 build2::main(int, char**) /home/karen/work/build2/build2/build2/b.cxx:1494 (b+0x7405f6)
#20 main /home/karen/work/build2/build2/build2/b.cxx:1596 (b+0x74331e)
Thread T1 (tid=16695, running) created by main thread at:
#0 pthread_create ../../../../libsanitizer/tsan/tsan_interceptors.cc:964 (libtsan.so.0+0x2cd1b)
#1 build2::scheduler::create_helper(std::unique_lock<std::mutex>&) /home/karen/work/build2/build2/build2/scheduler.cxx:698 (b+0x82c5b2)
#2 build2::scheduler::activate_helper(std::unique_lock<std::mutex>&) /home/karen/work/build2/build2/build2/scheduler.cxx:540 (b+0x82bb17)
#3 async<build2::match(build2::action, const build2::target&, std::size_t, build2::atomic_count*, bool)::<lambda(const build2::diag_frame*, const build2::target_lock*, build2::target&, std::size_t)>, const build2::diag_frame*&, const build2::target_lock*&, std::reference_wrapper<build2::target>, long unsigned int&> /home/karen/work/build2/build2/build2/scheduler.txx:115 (b+0xa81bf4)
#4 build2::match(build2::action, build2::target const&, unsigned long, std::atomic<unsigned long>*, bool) /home/karen/work/build2/build2/build2/algorithm.cxx:582 (b+0xa7202a)
#5 build2::match_async(build2::action, build2::target const&, unsigned long, std::atomic<unsigned long>&, bool) /home/karen/work/build2/build2/build2/algorithm.ixx:388 (b+0x918580)
#6 build2::match(butl::small_vector<build2::value, 1ul> const&, build2::action, build2::action_targets&, unsigned short, bool) /home/karen/work/build2/build2/build2/operation.cxx:174 (b+0x910839)
#7 build2::main(int, char**) /home/karen/work/build2/build2/build2/b.cxx:1494 (b+0x7405f6)
#8 main /home/karen/work/build2/build2/build2/b.cxx:1596 (b+0x74331e)
SUMMARY: ThreadSanitizer: data race ../../../../libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cc:282 in __sanitizer::IsAccessibleMemoryRange(unsigned long, unsigned long)
==================
Note that if we build our program with only one sanitizer enabled (either -fsanitize=thread or -fsanitize=undefined option is used), then no diagnostics is issued.
Hi Karen,
Thanks for the report.
It seems that the problem is in ubsan, or more precisely in IsAccessibleMemoryRange. It should not use pipe call as is, but instead use internal_pipe just like it uses internal_write/internal_close/etc. Pipe is getting intercepted by tsan can causes false positives because the rest of ubsan is not instrumented.
internal_pipe does not exist yet, so one would need to add it first to fix this.