RemovedComponents not guaranteed to contain removed components

[Lane Kolbly]

So according to https://bevy-cheatbook.github.io/programming/change-detection.html#removal-detection, the RemovedComponents parameter doesn't hold its information forever. Additionally, the Index type only calls refresh when the index is used to lookup something, which means if the user despawns an entity and then doesn't use the lookup, the index might miss the removal.

Below is some example code which reproduces the issue (depends on just bevy and bevy_mod_index). In short, it spawns two UnitId entities with ID 0 and 1. An index is created to lookup entities by ID. Then a state machine runs:

• First, entities with ID 0 are looked up, and then despawned.
• Second, a new entity with ID 0 is spawned, without querying the index.
• Finally, the index is queries for entities with ID 0.

The expected print is something like this:

First lookup:
 - Entity 1v0 has value 0, despawning it

Re-inserting element with 0

Second lookup:
 - Entity 1v1 has value 0
But actually:
 - Entity 2v0 has value 1
 - Entity 1v1 has value 0
Done, exiting

but the actual print I get is this:

First lookup:
 - Entity 1v0 has value 0, despawning it

Re-inserting element with 0

Second lookup:
 - Entity 1v1 has value 0
 - Entity 1v0 has value 0
But actually:
 - Entity 2v0 has value 1
 - Entity 1v1 has value 0
Done, exiting

Note that on the second lookup, entity 1v0 (which was despawned in the first lookup) is returned.

(the existence of the "expected output" does indeed imply that I got a fix working for me locally, but I'm very not sure that it's the right fix, so I'll make a PR for it separately)

use bevy::{app::AppExit, prelude::*};
use bevy_mod_index::prelude::*;

#[derive(Component)]
struct UnitId(u32);

struct UnitLookup;

impl IndexInfo for UnitLookup {
    type Component = UnitId;
    type Value = u32;
    type Storage = HashmapStorage<Self>;

    fn value(c: &Self::Component) -> Self::Value {
        c.0
    }
}

fn setup(mut commands: Commands) {
    commands.spawn(StateMachine::Start);
    commands.spawn(UnitId(0));
    commands.spawn(UnitId(1));
}

#[derive(Component)]
enum StateMachine {
    Start,
    Reinsert,
    SecondLookup,
    Done,
}

fn run_sm(
    mut commands: Commands,
    mut sm: Query<&mut StateMachine>,
    mut unit_idx: Index<UnitLookup>,
    mut exit: EventWriter<AppExit>,
    units: Query<(Entity, &UnitId)>,
) {
    let mut sm = sm.single_mut();
    *sm = match *sm {
        StateMachine::Start => {
            println!("First lookup:");
            for e in unit_idx.lookup(&0) {
                println!(" - Entity {:?} has value 0, despawning it", e);
                commands.entity(e).despawn();
            }
            println!();
            StateMachine::Reinsert
        }
        StateMachine::Reinsert => {
            println!("Re-inserting element with 0");
            commands.spawn(UnitId(0));
            println!();
            StateMachine::SecondLookup
        }
        StateMachine::SecondLookup => {
            println!("Second lookup:");
            for e in unit_idx.lookup(&0) {
                println!(" - Entity {:?} has value 0", e);
            }
            println!("But actually:");
            for (e, u) in units.iter() {
                println!(" - Entity {:?} has value {}", e, u.0);
            }
            StateMachine::Done
        }
        StateMachine::Done => {
            println!("Done, exiting");
            exit.send(AppExit);
            StateMachine::Done
        }
    }
}

fn main() {
    App::new()
        .add_plugins(MinimalPlugins)
        .add_systems(Startup, setup)
        .add_systems(Update, run_sm)
        .run();
}

[Chris Juchem]

Hi @lkolbly, thanks for your PR. I was aware of this issue but unfortunately have not had time to work on this crate lately (though I expect that to change in the new year).

I'm a little bit wary of doing a full refresh every frame because it would hurt the performance for large unchanging indexes when they are not frequently used.

The two potential solutions for this this issue I had in mind are:

1. Split refresh into refresh_removals and refresh_changes, and call only refresh_removals every frame.
2. Add a configuration option to the IndexInfo trait that would specify whether that index should be updated automatically every frame.

The second option could effective be done today by manually adding a system like

fn refresh_index(mut idx: Index<UnitLookup>) {
    idx.refresh()
}

I think this is something it would be reasonable to provide from this library.

Would you find this to be an acceptable solution until I get the chance to return to this crate and implement somthing more proper?

[Lane Kolbly]

Hey! I don't have a particularly pressing need to have this fixed upstream, I have a local copy that works for what I need for the foreseeable future, so I'm fine to wait for a proper fix.

The solution you mention of splitting refresh into two separate refreshes makes a lot of sense to me. It does mean worse performance for cases where the user is removing a lot of entities, but if the user is removing lots of entities then they probably want the index to reflect that (and so the second option, of adding a configuration option, would pretty much force the user to do a full refresh every frame in that scenario).

I'd be down to try taking a stab at this in a PR.

[Chris Juchem]

@lkolbly I implemented additional configs for refresh behavior here.
You should just need to add RefreshPolicy = SimpleRefreshPolicy; to your IndexInfo definition.

Take a look and let me know if it works for you. I'd like to include this in a release with a few other features soon.

[Lane Kolbly]

@chrisjuchem Awesome, thanks! It works great for me.

lookup_single is also quite handy. I have both situations where I know that the entity is in the index, and also situations where I want to check - panicking is good for the first situation, but the second it'd be nice if I could get an Option out of it. I don't know what a good name would be, maybe lookup_single and get_single (or just get)? Not sure.

Right now I only check if the index has at least one value for the key, if there's multiple values it's a logic error in my application (that currently would go uncaught). Then we start to get into "UniqueIndex" type of things.

[Chris Juchem]

I renamed lookup_single to single and made lookup_single return a Result. Released in v0.4.0.