Question - Not getting expected result

Question

Question - Not getting expected result

kliegr opened this issue 4 years ago · comments

Hi,

This is a question rather than a bug report.
I've created a small test case, cf below (data is also attached).

I was wondering why AMIE+/AMIE 3 does not return any rule (when mining without constants) on this dataset?
Why is the rule depicted in the figure on the right not found? This rule seems to have at least three positive examples. If the example is constructed correctly, is there any setting that would make it found? Would, e.g., Aleph find this rule?
sample5.tsv.zip

The mining setup:

java -jar amie_plus.jar -mins 1  -minis 1  -minc 0.1  -maxad 10  sample5.tsv

java -jar amie-dev.jar -mins 1   -minis 1  -minc 0.1  -maxad 10 -rl 10 -noHeuristics   sample5.tsv

Neither return any rules.

Thanks!

Jonathan Lajus · Answer 1 · Sat Jun 27 2020 02:56:29 GMT+0800 (China Standard Time)

Hi,

You should use the additionnal option -pm support in order to take into account the support thresholds as it will still use the head coverage here (and ignore the -mins option).

I realize it is counter-intuitive and I will fix this.

Jonathan

Luis Galárraga · Answer 2 · Tue Jun 30 2020 05:23:55 GMT+0800 (China Standard Time)

Hi,

I added the -pm support option and it did not work. The problem is the recursivity limit which is harcoded to 3, i.e., AMIE does not allow more than 3 times the same predicate in a rule. If you want to mine that rule you have to increase the recursivityLimit attribute in the class MiningAssistant (amie/mining/src/main/java/amie/mining/assistant/MiningAssistant.java line 50). This will make AMIE slower though.

@lajus Do you think we should provide ways to also adjust that parameter in the command line arguments?

Cheers,
Luis

kliegr · Answer 3 · Wed Jul 01 2020 16:02:49 GMT+0800 (China Standard Time)

Thanks for tracing of this problem. I have not yet tried recompiling the code, but it seems to be that the value 3 is a default value and can be set to a different value using the rl argument by the user (mining/src/main/java/amie/mining/AMIE.java line 1328): mineAssistant.setRecursivityLimit(recursivityLimit);

which is something that I tried, cf.

 java -jar amie-dev.jar -mins 1   -minis 1  -minc 0.1  -maxad 10 -rl 10 -noHeuristics   sample5.tsv

Luis Galárraga · Answer 4 · Wed Jul 01 2020 16:11:33 GMT+0800 (China Standard Time)

Hi,

I had completely forgotten about this command line argument, wow. Did it work then?

Luis

kliegr · Answer 5 · Wed Jul 01 2020 16:26:50 GMT+0800 (China Standard Time)

Unfortunately no, I've tried both in AMIE+ and AMIE 3 using the -rl argument, I also tried other settings like -pm (this actually led me to try oneVar). There might be something wrong with my example but cannot figure out what. Based on other experiments with different examples (I tried multiple before I arrived at this minimum working example) it looks like AMIE does not output rules that have one predicate multiple times in the body (i.e., it really looks like a problem with recursion but not as simple as the rl argument). I do not remember if I saw rules with 2 or 3 same predicates in the body, or what is the minimum count when the problem starts.

Jonathan Lajus · Answer 6 · Wed Jul 01 2020 22:44:14 GMT+0800 (China Standard Time)

Hi,

I think (have not checked) that the skyline pruning discards the rule.

More precisely, the following rule or an equivalent should be in your output:
?c interacts_with ?p & ?c interacts_with ?b & ?d interacts_with ?b => ?d interacts_with ?p

The skyline pruning will remove every specialization of an outputted rules if the specialized rule is not strictly better than the outputted rule. Thus, as the rule you want to mine is a specialization of the above rule, it will be outputted only if its confidence is strictly superior to the confidence of the rule above.

There is no option for disabling the skyline pruning but I could add one.

Cheers,
Jonathan

kliegr · Answer 7 · Wed Jul 01 2020 23:34:43 GMT+0800 (China Standard Time)

Sorry for being vague about the output I am getting, I actually do not get any results (no discovered rules at all), but it could be something like that.

Jonathan Lajus · Answer 8 · Thu Jul 02 2020 00:02:43 GMT+0800 (China Standard Time)

Hi,

This is because you are using the release that removes diamond patterns. It is not the case anymore in the development version as we noticed it was breaking stuff.
I updated amie-dev.jar and:
amie-dev.jar -maxad 10 -mins 1 -minis 1 -rl 8 -nc 8 sample5.tsv gives me 3 rules.
I can have 15 rules using:
amie-dev.jar -maxad 10 -mins 1 -minis 1 -rl 8 -nc 8 -dpr -noSkyline sample5.tsv

Note also that by default AMIE does not enforce injective mappings, it means that when it counts the support/confidence of a rule ?c and ?d may be mapped to the same entity (or ?p and ?b) which messes with the counts we would expect (it was a reason diamond patterns were removed by Luis).

If you want to enforce injective mappings (different variables must be mapped to different entities), you need to compile the "gpro" branch of this repository and use:
-bias amie.mining.assistant.experimental.InjectiveMappingsAssistant.
It works but I need to perform further testing to ensure the gro branch does not break the existing code.

Cheers,
Jonathan

kliegr · Answer 9 · Sun Jul 05 2020 18:29:04 GMT+0800 (China Standard Time)

Thanks for the fast action:) I also get now these rules, I visualized one of them (renaming some variables) cf.

It looks quite sensible if compared with the visualization of the input data above. The rule though does not exactly look like the expected rule, but this seems to be caused by the injective mappings, as you suggest. I therefore recompiled the gpro branch and run

java -jar amie3.jar -bias amie.mining.assistant.experimental.InjectiveMappingsAssistant -maxad 10 -mins 1 -minis 1 -rl 8 -nc 8  -dpr  sample5.tsv

but there I am not getting any rules. Maybe the fix that was done to the master has not been committed to this branch? Or maybe I have some parameters wrong? It should be the gpro branch compiled because the code referring to InjectiveMappingsAssistant runs (i.e., the class is found).

Could you provide some more pointers for the diamond patterns? I could not find this term in AMIE literature, but it seems quite important for understanding AMIE+ behaviour.

Thanks,

Tomas

Jonathan Lajus · Answer 10 · Mon Jul 06 2020 16:35:59 GMT+0800 (China Standard Time)

Hi,

I did not merge the fixes in master into the gpro branch. Should work now.

As for the diamond patterns, I am not sure it is documented in the literature (as we mostly consider rules of size 3 anyway). @lgalarra ?

It is a output pruning method used in the DefaultMiningAssistant removing patterns in forms of diamond (see amie.rules.Rules#containsDisallowedDiamond).

Cheers,
Jonathan

Luis Galárraga · Answer 11 · Tue Jul 07 2020 05:54:58 GMT+0800 (China Standard Time)

Hi everyone,

An example of a diamond rule is:

?i <hasCapital> ?b ?a <hasGeonamesId> ?f ?i <hasGeonamesId> ?f => ?a <hasCapital> ?b
where we observed that ?i and ?a were bound to the same values. At some point in the past we used the method containsDisallowedDiamond to filter the rules that contained that kind of patterns. We then decided to remove that filtering for the sake of efficiency and simplicity. We thought that if a user wants to avoid such patterns, he/she could write a mining assistant to do so or even implement the difference operator for two variables (as the condition ?a != ?i would do the job)

Cheers,
Luis