Possible memory leak with `flag::register`

Question

Possible memory leak with `flag::register`

vjancik opened this issue 3 years ago · comments

Environment:
rustc: 1.48.0
target: x86_64-unknown-linux-gnu

Valgrind reports a memory leak after a normal (graceful) return from fn main.

Code (lines main.rs: 202..=208):

{ // valgrind reports memory leak
    let term_sig = sync::Arc::new(atomic::AtomicBool::new(false));
    for sig in TERM_SIGNALS {
        flag::register_conditional_shutdown(*sig, 1, sync::Arc::clone(&term_sig))?;
        flag::register(*sig, sync::Arc::clone(&term_sig))?;
    }
}

Valgrind output:

==30965== Memcheck, a memory error detector
==30965== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==30965== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==30965== Command: ./target/debug/tun_playground --tun tunclient1 --virtual 10.0.0.2 client --server 192.168.0.117:5555
==30965== 
Received a signal Origin { signal: 2, process: None, cause: Kernel }
Terminating
==30965== 
==30965== HEAP SUMMARY:
==30965==     in use at exit: 2,396 bytes in 15 blocks
==30965==   total heap usage: 317 allocs, 302 frees, 166,090 bytes allocated
==30965== 
==30965== 24 bytes in 1 blocks are possibly lost in loss record 1 of 9
==30965==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==30965==    by 0x144CDB: alloc::alloc::alloc (alloc.rs:74)
==30965==    by 0x144D99: alloc::alloc::Global::alloc_impl (alloc.rs:153)
==30965==    by 0x144F39: <alloc::alloc::Global as core::alloc::AllocRef>::alloc (alloc.rs:212)
==30965==    by 0x144C3C: alloc::alloc::exchange_malloc (alloc.rs:302)
==30965==    by 0x146329: alloc::sync::Arc<T>::new (sync.rs:314)
==30965==    by 0x14E66C: tun_playground::main (main.rs:203)
==30965==    by 0x1525AA: core::ops::function::FnOnce::call_once (function.rs:227)
==30965==    by 0x141C0D: std::sys_common::backtrace::__rust_begin_short_backtrace (backtrace.rs:137)
==30965==    by 0x13DB30: std::rt::lang_start::{{closure}} (rt.rs:66)
==30965==    by 0x307346: call_once<(),Fn<()>> (function.rs:259)
==30965==    by 0x307346: do_call<&Fn<()>,i32> (panicking.rs:381)
==30965==    by 0x307346: try<i32,&Fn<()>> (panicking.rs:345)
==30965==    by 0x307346: catch_unwind<&Fn<()>,i32> (panic.rs:382)
==30965==    by 0x307346: std::rt::lang_start_internal (rt.rs:51)
==30965==    by 0x13DB06: std::rt::lang_start (rt.rs:65)
==30965== 
==30965== 72 bytes in 3 blocks are possibly lost in loss record 5 of 9
==30965==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==30965==    by 0x16A1CB: alloc::alloc::alloc (alloc.rs:74)
==30965==    by 0x16A289: alloc::alloc::Global::alloc_impl (alloc.rs:153)
==30965==    by 0x16ACD9: <alloc::alloc::Global as core::alloc::AllocRef>::alloc (alloc.rs:212)
==30965==    by 0x16A12C: alloc::alloc::exchange_malloc (alloc.rs:302)
==30965==    by 0x15B091: alloc::sync::Arc<T>::new (sync.rs:314)
==30965==    by 0x15BDFE: <alloc::sync::Arc<T> as core::convert::From<T>>::from (sync.rs:2158)
==30965==    by 0x162CF2: signal_hook_registry::register_unchecked_impl (lib.rs:572)
==30965==    by 0x161F88: signal_hook_registry::register_sigaction_impl (lib.rs:527)
==30965==    by 0x1636B4: signal_hook_registry::register (lib.rs:498)
==30965==    by 0x161E04: signal_hook::flag::register (flag.rs:154)
==30965==    by 0x14E8CC: tun_playground::main (main.rs:206)
==30965== 
==30965== 96 bytes in 3 blocks are possibly lost in loss record 6 of 9
==30965==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==30965==    by 0x16A1CB: alloc::alloc::alloc (alloc.rs:74)
==30965==    by 0x16A289: alloc::alloc::Global::alloc_impl (alloc.rs:153)
==30965==    by 0x16ACD9: <alloc::alloc::Global as core::alloc::AllocRef>::alloc (alloc.rs:212)
==30965==    by 0x16A12C: alloc::alloc::exchange_malloc (alloc.rs:302)
==30965==    by 0x15AF38: alloc::sync::Arc<T>::new (sync.rs:314)
==30965==    by 0x15BDE2: <alloc::sync::Arc<T> as core::convert::From<T>>::from (sync.rs:2158)
==30965==    by 0x1622A0: signal_hook_registry::register_unchecked_impl (lib.rs:572)
==30965==    by 0x162107: signal_hook_registry::register_sigaction_impl (lib.rs:527)
==30965==    by 0x1636F0: signal_hook_registry::register (lib.rs:498)
==30965==    by 0x161E6F: signal_hook::flag::register_conditional_shutdown (flag.rs:188)
==30965==    by 0x14E7A9: tun_playground::main (main.rs:205)
==30965== 
==30965== 788 bytes in 1 blocks are possibly lost in loss record 8 of 9
==30965==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==30965==    by 0x17ACAB: alloc::alloc::alloc (alloc.rs:74)
==30965==    by 0x1796F6: hashbrown::raw::RawTable<T>::new_uninitialized (mod.rs:411)
==30965==    by 0x17833D: <hashbrown::raw::RawTable<T> as core::clone::Clone>::clone (mod.rs:1147)
==30965==    by 0x18A967: <hashbrown::map::HashMap<K,V,S> as core::clone::Clone>::clone (map.rs:197)
==30965==    by 0x17C198: <std::collections::hash::map::HashMap<K,V,S> as core::clone::Clone>::clone (map.rs:202)
==30965==    by 0x183BF2: <signal_hook_registry::SignalData as core::clone::Clone>::clone (lib.rs:186)
==30965==    by 0x18387C: signal_hook_registry::unregister (lib.rs:636)
==30965==    by 0x1690C6: <signal_hook::iterator::backend::DeliveryState as core::ops::drop::Drop>::drop (backend.rs:68)
==30965==    by 0x152AE5: core::ptr::drop_in_place (mod.rs:175)
==30965==    by 0x146D11: alloc::sync::Arc<T>::drop_slow (sync.rs:934)
==30965==    by 0x153A74: <alloc::sync::Arc<T> as core::ops::drop::Drop>::drop (sync.rs:1454)
==30965== 
==30965== 1,104 bytes in 3 blocks are possibly lost in loss record 9 of 9
==30965==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==30965==    by 0x17ACAB: alloc::alloc::alloc (alloc.rs:74)
==30965==    by 0x17AD69: alloc::alloc::Global::alloc_impl (alloc.rs:153)
==30965==    by 0x17B1C9: <alloc::alloc::Global as core::alloc::AllocRef>::alloc (alloc.rs:212)
==30965==    by 0x17AC0C: alloc::alloc::exchange_malloc (alloc.rs:302)
==30965==    by 0x173DD0: new<alloc::collections::btree::node::LeafNode<signal_hook_registry::ActionId, alloc::sync::Arc<Fn<(&libc::unix::linux_like::linux::gnu::b64::x86_64::siginfo_t)>>>> (boxed.rs:175)
==30965==    by 0x173DD0: alloc::collections::btree::node::Root<K,V>::new_leaf (node.rs:163)
==30965==    by 0x186563: <alloc::collections::btree::map::BTreeMap<K,V> as core::clone::Clone>::clone::clone_subtree (map.rs:151)
==30965==    by 0x187081: <alloc::collections::btree::map::BTreeMap<K,V> as core::clone::Clone>::clone (map.rs:211)
==30965==    by 0x183B4C: <signal_hook_registry::Slot as core::clone::Clone>::clone (lib.rs:141)
==30965==    by 0x189527: core::clone::Clone::clone (clone.rs:119)
==30965==    by 0x179335: hashbrown::raw::RawTable<T>::clone_from_impl (mod.rs:1258)
==30965==    by 0x1786B2: <hashbrown::raw::RawTable<T> as hashbrown::raw::RawTableClone>::clone_from_spec (mod.rs:1208)
==30965== 
==30965== LEAK SUMMARY:
==30965==    definitely lost: 0 bytes in 0 blocks
==30965==    indirectly lost: 0 bytes in 0 blocks
==30965==      possibly lost: 2,084 bytes in 11 blocks
==30965==    still reachable: 312 bytes in 4 blocks
==30965==         suppressed: 0 bytes in 0 blocks
==30965== Reachable blocks (those to which a pointer was found) are not shown.
==30965== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30965== 
==30965== For lists of detected and suppressed errors, rerun with: -s
==30965== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)

I isolated the leaking code in it's own block so I'm certain there's no interaction with the rest of the code base.
It appears the Arc isn't getting Dropped correctly?

Michal 'vorner' Vaner · Answer 1 · Thu Jan 07 2021 03:28:04 GMT+0800 (China Standard Time)

I might try digging into it a bit more, but I believe this is to be expected. Rust doesn't run destructors for statics at the end of the program and the action, containing the arc, is installed past the end of main. Freeing it before the end of the program would indeed be problematic if another signal came after that.

My system's valgrind is currently broken, so I can't test, but can you try something like this?

static mut G: Option<String> = None;

fn main() {
    unsafe {
        G = Some("Hello world".to_owned());
    }
}

Also, if you ask signal-hook to remove the actions at the end of the program (with the IDs you get back from it), the leak should go away. Can you also try that?

Thanks

Michal 'vorner' Vaner · Answer 2 · Thu Jan 07 2021 03:47:27 GMT+0800 (China Standard Time)

OK, I've found a system with working valgrind. Some experiments I've run:

If I unregister the actions again before exiting, there's far fewer blocks in possibly lost. They are a hashmap of actions that's left in there as a global variable and as Rust doesn't run destructors of globals + doesn't shrink containers when things are removed from them, it kind of makes sense.
I do get some possibly lost blocks with the fragment I've shown.

I'll try some more torturing (making sure that when actions are repeatedly added and removed, it doesn't grow), but I think this is more a limitation of valgrind than a true leak.

Michal 'vorner' Vaner · Answer 3 · Sat Jan 09 2021 22:48:28 GMT+0800 (China Standard Time)

I've tried this program that repeatedly registers and removes the above actions:

use std::sync::Arc;
use std::sync::atomic::AtomicBool;

use signal_hook::{flag, low_level};
use signal_hook::consts::SIGTERM;

type Error = Box<dyn std::error::Error + Send + Sync>;

fn main() -> Result<(), Error> {
    let mut old = Vec::new();
    loop {
        let cond = Arc::new(AtomicBool::new(false));
        let sh = flag::register_conditional_shutdown(SIGTERM, 1, Arc::clone(&cond))?;
        let set = flag::register(SIGTERM, Arc::clone(&cond))?;
        for act in old.drain(..) {
            assert!(low_level::unregister(act));
        }
        old.push(sh);
        old.push(set);
    }
}

I've left it running for several hours. Its memory consumption wasn't growing so I believe this can be closed.