Deadlock appeared again
lhsoft opened this issue · comments
We have used the tcmalloc of master branch recently and got the following stacks：
#1 0x0000000000b05f36 in AbslInternalSpinLockDelay (w=w@entry=0x22ec830 <tcmalloc::tcmalloc_internal::pageheap_lock>, value=value@entry=9) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock_linux.inc:63 #2 0x0000000000493231 in absl::base_internal::SpinLockDelay (scheduling_mode=absl::base_internal::SCHEDULE_KERNEL_ONLY, loop=1, value=9, w=0x22ec830 <tcmalloc::tcmalloc_internal::pageheap_lock>) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock_wait.h:91 #3 absl::base_internal::SpinLock::SlowLock (this=0x22ec830 <tcmalloc::tcmalloc_internal::pageheap_lock>) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.cc:162 #4 0x0000000000ad87cc in absl::base_internal::SpinLock::Lock (this=<optimized out>) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.h:75 #5 absl::base_internal::SpinLockHolder::SpinLockHolder (l=<optimized out>, this=<synthetic pointer>) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.h:196 #6 tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::LockAndAlloc ( from_released=0x7f6d64298f6f, objects_per_span=1, n=..., this=0x2181288 <tcmalloc::tcmalloc_internal::Static::page_allocator_+695848>) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:541 #7 tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::New ( this=0x2181288 <tcmalloc::tcmalloc_internal::Static::page_allocator_+695848>, n=..., objects_per_span=1) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:526 #8 0x0000000000ab181d in tcmalloc::tcmalloc_internal::PageAllocator::New (tag=tcmalloc::tcmalloc_internal::MemoryTag::kSampled, objects_per_span=1, n=..., this=<optimized out>) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/page_allocator.h:177 #9 tcmalloc::sized_ptr_t tcmalloc::tcmalloc_internal::SampleifyAllocation<tcmalloc::tcmalloc_internal::Static, tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::NullOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy, tcmalloc::tcmalloc_internal::AllocationAccessHotPolicy, tcmalloc::tcmalloc_internal::InvokeHooksPolicy, tcmalloc::tcmalloc_internal::NonSizeReturningPolicy, tcmalloc::tcmalloc_internal::LocalNumaPartitionPolicy> >(tcmalloc::tcmalloc_internal::Static&, tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::NullOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy, tcmalloc::tcmalloc_internal::AllocationAccessHotPolicy, tcmalloc::tcmalloc_internal::InvokeHooksPolicy, tcmalloc::tcmalloc_internal::NonSizeReturningPolicy, tcmalloc::tcmalloc_internal::LocalNumaPartitionPolicy>, unsigned long, unsigned long, unsigned long, void*, tcmalloc::tcmalloc_internal::Span*) [clone .isra.0] () at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/allocation_sampling.h:354 #10 0x0000000000ab2628 in tcmalloc::tcmalloc_internal::SampleSmallAllocation<tcmalloc::tcmalloc_internal::Static, tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (state=..., size_class=4, weight=<optimized out>, requested_size=40, policy=..., res=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/sampler.h:197 #11 tcmalloc::tcmalloc_internal::(anonymous namespace)::AllocSmall<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (size=40, size_class=4, policy=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:658 #12 slow_alloc<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (size=40, policy=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:987 #13 0x00000000012881d6 in fast_alloc<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > ( size=size@entry=40, policy=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/sizemap.h:149 #14 TCMallocInternalNew (size=size@entry=40) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:1135 #15 0x0000000000cda6fa in google::protobuf::Arena::InternalHelper<brpc::policy::RpcResponseMeta>::New () at /usr/include/c++/9/new:174 #16 google::protobuf::Arena::CreateMessageInternal<brpc::policy::RpcResponseMeta> (arena=<optimized out>) at external/protobuf~3.19.6/src/google/protobuf/arena.h:559 #17 google::protobuf::Arena::CreateMaybeMessage<brpc::policy::RpcResponseMeta> (arena=<optimized out>)
The #111 had fixed the deadlock, but it seems that the deadlock appears again
gdb.txt
@ckennelly any idea for this?
It looks like the deadlock happens in SampleifyAllocation,
@lhsoft Which thread holds the lock?
SampelifyAllocation seems to be spinning/waiting for the lock, but it doesn't hold pageheap_lock when it invokes state.page_allocator().New().
@ckennelly Is there any way to find the thread which holds the lock? It looks like that the spinlock doesn't have the info of owner.
@ckennelly The root cause is that we have a thread to dump the memory stats periodically by calling PrintStats
Thread 20 (Thread 0x7f01a6068700 (LWP 36400)):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x0000000000ea3763 in AbslInternalSpinLockDelay (w=0x29fff80 <tcmalloc::tcmalloc_internal::pageheap_lock>, value=9)
at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock_linux.inc:63
#2 0x0000000000ea34e8 in absl::base_internal::SpinLockDelay (w=0x29fff80 <tcmalloc::tcmalloc_internal::pageheap_lock>, value=9, loop=1,
scheduling_mode=absl::base_internal::SCHEDULE_KERNEL_ONLY) at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock_wait.h:92
#3 0x0000000000ea306f in absl::base_internal::SpinLock::SlowLock (this=0x29fff80 <tcmalloc::tcmalloc_internal::pageheap_lock>)
at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.cc:162
#4 0x0000000000d5f0b7 in absl::base_internal::SpinLock::Lock (this=0x29fff80 <tcmalloc::tcmalloc_internal::pageheap_lock>)
at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.h:75
#5 0x0000000000d5f1d7 in absl::base_internal::SpinLockHolder::SpinLockHolder (this=0x7f01a6060b20, l=0x29fff80 <tcmalloc::tcmalloc_internal::pageheap_lock>)
at external/_main~data_deps_ext~com_google_absl/absl/base/internal/spinlock.h:196
#6 0x0000000000dc8fdf in tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::LockAndAlloc (this=0x2ac16e8 <tcmalloc::tcmalloc_internal::Static::page_allocator_+695848>, n=..., objects_per_span=1,
from_released=0x7f01a6060b70) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:541
#7 0x0000000000dc7a2e in tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::New (this=0x2ac16e8 <tcmalloc::tcmalloc_internal::Static::page_allocator_+695848>, n=..., objects_per_span=1)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:526
#8 0x0000000000d613aa in tcmalloc::tcmalloc_internal::PageAllocator::New (this=0x2a178c0 <tcmalloc::tcmalloc_internal::Static::page_allocator_>, n=...,
objects_per_span=1, tag=tcmalloc::tcmalloc_internal::MemoryTag::kSampled)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/page_allocator.h:177
#9 0x0000000000d4f237 in tcmalloc::tcmalloc_internal::SampleifyAllocation<tcmalloc::tcmalloc_internal::Static, tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (state=..., policy=..., requested_size=513, weight=2097256,
size_class=30, obj=0x4cf3bdef680, span=0x0) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/allocation_sampling.h:354
#10 0x0000000000d4cc21 in tcmalloc::tcmalloc_internal::SampleSmallAllocation<tcmalloc::tcmalloc_internal::Static, tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (state=..., policy=..., requested_size=513, weight=2097256,
size_class=30, res=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/allocation_sampling.h:494
#11 0x0000000000d39aa1 in tcmalloc::tcmalloc_internal::(anonymous namespace)::AllocSmall<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (size=513, size_class=30, policy=...)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:658
#12 slow_alloc<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (
policy=..., size=513) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:987
#13 0x0000000001b0ce57 in fast_alloc<tcmalloc::tcmalloc_internal::TCMallocPolicy<tcmalloc::tcmalloc_internal::CppOomPolicy, tcmalloc::tcmalloc_internal::DefaultAlignPolicy> > (size=513, policy=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:1037
#14 TCMallocInternalNew (size=513) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:1135
#15 0x0000000001ada86c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct (this=0x7f01a6061460,
__n=<optimized out>, __c=<optimized out>) at /mnt/ssd01/liuhu/workspace/src/gcc-9.4.0/x86_64-redhat-linux/libstdc++-v3/include/bits/char_traits.h:300
#16 0x0000000000460736 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> > (
this=0x7f01a6061460, __n=512, __c=0 '\000', __a=...) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/bits/basic_string.h:546
#17 0x0000000000e8aff9 in absl::str_format_internal::(anonymous namespace)::FallbackToSnprintf<double> (v=4.8301623795663097e-05, conv=...,
sink=0x7f01a6061900) at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/float_conversion.cc:979
#18 0x0000000000e89e0e in absl::str_format_internal::(anonymous namespace)::FloatToSink<double> (v=4.8301623795663097e-05, conv=..., sink=0x7f01a6061900)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/float_conversion.cc:1386
#19 0x0000000000e88b6d in absl::str_format_internal::ConvertFloatImpl (v=4.8301623795663097e-05, conv=..., sink=0x7f01a6061900)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/float_conversion.cc:1452
#20 0x0000000000e7e800 in absl::str_format_internal::(anonymous namespace)::ConvertFloatArg<double> (v=4.8301623795663097e-05, conv=..., sink=0x7f01a6061900)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/arg.cc:288
#21 0x0000000000e7e35e in absl::str_format_internal::FormatConvertImpl (v=4.8301623795663097e-05, conv=..., sink=0x7f01a6061900)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/arg.cc:451
#22 0x0000000000e81be3 in absl::str_format_internal::FormatArgImpl::Dispatch<double> (arg=..., spec=..., out=0x7f01a6061900)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/arg.h:574
#23 0x0000000000e85934 in absl::str_format_internal::FormatArgImplFriend::Convert<absl::str_format_internal::FormatArgImpl> (arg=..., conv=...,
out=0x7f01a6061900) at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/arg.h:388
#24 0x0000000000e836b3 in absl::str_format_internal::(anonymous namespace)::DefaultConverter::ConvertOne (this=0x7f01a6061830, bound=...)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/bind.cc:143
#25 0x0000000000e84e6a in absl::str_format_internal::(anonymous namespace)::ConverterConsumer<absl::str_format_internal::(anonymous namespace)::DefaultConverter>::ConvertOne (this=0x7f01a6061830, conv=..., conv_string=...) at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/bind.cc:116
#26 0x0000000000e84cd7 in absl::str_format_internal::ParseFormatString<absl::str_format_internal::(anonymous namespace)::ConverterConsumer<absl::str_format_internal::(anonymous namespace)::DefaultConverter> > (src=..., consumer=...)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/parser.h:99
#27 0x0000000000e84312 in absl::str_format_internal::(anonymous namespace)::ConvertAll<absl::str_format_internal::(anonymous namespace)::DefaultConverter> (
format=..., args=..., converter=...) at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/bind.cc:131
#28 0x0000000000e83d16 in absl::str_format_internal::FormatUntyped (raw_sink=..., format=..., args=...)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/bind.cc:201
#29 0x0000000000e8410e in absl::str_format_internal::SnprintF (output=0x30a4ffc2a842 "", size=87997, format=..., args=...)
at external/_main~data_deps_ext~com_google_absl/absl/strings/internal/str_format/bind.cc:248
#30 0x0000000000de6673 in absl::SNPrintF<unsigned long, unsigned long, unsigned long, double, double, double>(char*, unsigned long, absl::str_format_internal::FormatSpecTemplate<(ArgumentToConv<unsigned long>)(), (ArgumentToConv<unsigned long>)(), (ArgumentToConv<unsigned long>)(), (ArgumentToConv<double>)(), (ArgumentToConv<double>)(), (ArgumentToConv<double>)()> const&, unsigned long const&, unsigned long const&, unsigned long const&, double const&, double const&, double const&) (output=0x30a4ffc2a842 "", size=87997, format=...) at external/_main~data_deps_ext~com_google_absl/absl/strings/str_format.h:463
#31 0x0000000000de5f33 in tcmalloc::tcmalloc_internal::Printer::printf<unsigned long, unsigned long, unsigned long, double, double, double>(absl::str_format_internal::FormatSpecTemplate<(ArgumentToConv<unsigned long>)(), (ArgumentToConv<unsigned long>)(), (ArgumentToConv<unsigned long>)(), (ArgumentToConv<double>)(), (ArgumentToConv<double>)(), (ArgumentToConv<double>)()> const&, unsigned long const&, unsigned long const&, unsigned long const&, double const&, double const&, double const&) (this=0x7f01a60673d0, format=...) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/internal/logging.h:192
#32 0x0000000000de469b in tcmalloc::tcmalloc_internal::PageAllocInfo::<lambda(const tcmalloc::tcmalloc_internal::PageAllocInfo::Counts&, tcmalloc::tcmalloc_internal::Length, tcmalloc::tcmalloc_internal::Length)>::operator()(const tcmalloc::tcmalloc_internal::PageAllocInfo::Counts &, tcmalloc::tcmalloc_internal::Length, tcmalloc::tcmalloc_internal::Length) const (__closure=0x7f01a6062010, c=..., nmin=..., nmax=...)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/stats.cc:364
#33 0x0000000000de48c7 in tcmalloc::tcmalloc_internal::PageAllocInfo::Print (this=0x2a178c8 <tcmalloc::tcmalloc_internal::Static::page_allocator_+8>,
out=0x7f01a60673d0) at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/stats.cc:372
#34 0x0000000000dca95c in tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::Print (this=0x2a178c0 <tcmalloc::tcmalloc_internal::Static::page_allocator_>, out=0x7f01a60673d0, everything=true)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:877
#35 0x0000000000dc8a24 in tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::Print (this=0x2a178c0 <tcmalloc::tcmalloc_internal::Static::page_allocator_>, out=0x7f01a60673d0)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/huge_page_aware_allocator.h:821
#36 0x0000000000d9497c in tcmalloc::tcmalloc_internal::PageAllocator::Print (this=0x2a178c0 <tcmalloc::tcmalloc_internal::Static::page_allocator_>,
out=0x7f01a60673d0, tag=tcmalloc::tcmalloc_internal::MemoryTag::kNormalP0)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/page_allocator.h:260
#37 0x0000000000d90c67 in tcmalloc::tcmalloc_internal::DumpStats (out=0x7f01a60673d0, level=2)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/global_stats.cc:398
#38 0x0000000000d2956d in tcmalloc::tcmalloc_internal::TCMalloc_Internal_GetStats (
buffer=0x30a4ffc00000 "See https://github.com/google/tcmalloc/tree/master/docs/stats.md for an explanation of this page\n", '-' <repeats 48 times>, "\nMALLOC: 599367736 ( 571.6 MiB) Bytes in use by a"..., buffer_length=262143)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:211
#39 0x0000000000d294db in tcmalloc::tcmalloc_internal::MallocExtension_Internal_GetStats (ret=0x7f01a6067490)
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/tcmalloc.cc:196
#40 0x0000000000e09b8e in tcmalloc::MallocExtension::GetStats[abi:cxx11]() ()
at external/_main~data_deps_ext~com_github_google_tcmalloc/tcmalloc/malloc_extensio.cc:206
#41 0x00000000005bc047 in taishan::node::RunStatCollector::dump_memory_stats (this=0x4cebfc94ec0) at node/src/run_stat_collector.cpp:253
#42 0x00000000005c1ad2 in std::__invoke_impl<void, void (taishan::node::RunStatCollector::*)(), taishan::node::RunStatCollector*> (
__f=@0x4cebfcf24f0: (void (taishan::node::RunStatCollector::*)(class taishan::node::RunStatCollector * const)) 0x5bbea2 <taishan::node::RunStatCollector::dump_memory_stats()>, __t=@0x4cebfcf24e8: 0x4cebfc94ec0) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/bits/invoke.h:73
#43 0x00000000005c1a14 in std::__invoke<void (taishan::node::RunStatCollector::*)(), taishan::node::RunStatCollector*> (
__fn=@0x4cebfcf24f0: (void (taishan::node::RunStatCollector::*)(class taishan::node::RunStatCollector * const)) 0x5bbea2 <taishan::node::RunStatCollector::dump_memory_stats()>) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/bits/invoke.h:95
#44 0x00000000005c1983 in std::thread::_Invoker<std::tuple<void (taishan::node::RunStatCollector::*)(), taishan::node::RunStatCollector*> >::_M_invoke<0ul, 1ul> (this=0x4cebfcf24e8) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/thread:244
#45 0x00000000005c193e in std::thread::_Invoker<std::tuple<void (taishan::node::RunStatCollector::*)(), taishan::node::RunStatCollector*> >::operator() (
this=0x4cebfcf24e8) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/thread:251
#46 0x00000000005c1922 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (taishan::node::RunStatCollector::*)(), taishan::node::RunStatCollector*> > >::_M_run (this=0x4cebfcf24e0) at /usr/local/lib/gcc/x86_64-redhat-linux/9/../../../../include/c++/9/thread:195
#47 0x0000000001add3d0 in std::execute_native_thread_routine (__p=0x4cebfcf24e0) at ../../../.././libstdc++-v3/src/c++11/thread.cc:80
#48 0x00007f01b0096fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#49 0x00007f01afc1106f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
The func tcmalloc::tcmalloc_internal::huge_page_allocator_internal::HugePageAwareAllocator<tcmalloc::tcmalloc_internal::huge_page_allocator_internal::StaticForwarder>::Print holds the pageheap_lock before return
inline void HugePageAwareAllocator<Forwarder>::Print(Printer* out,
bool everything) {
SmallSpanStats small;
LargeSpanStats large;
BackingStats bstats;
PageAgeHistograms ages(absl::base_internal::CycleClock::Now());
absl::base_internal::SpinLockHolder h(&pageheap_lock);
if (everything) {
regions_.Print(out);
out->printf("\n");
cache_.Print(out);
alloc_.Print(out);
out->printf("\n");
// Use statistics
info_.Print(out);
// and age tracking.
ages.Print("HugePageAware", out);
}
If any of print func allocate memory and call SampleifyAllocation will cause deadlock
Thanks for that very helpful stack trace!
I'll ask one of the Abseil StrFormat maintainers, but this is somewhat surprising. We rely on this not allocating internally too.