Loading of Filter Tests

[Robin]

I think there might be something broken with the filter tests, at least when I extended the test.json of the TextContainsKeywordsFilter to contain another test case:

{
    "type": "keywords",
    "test_cases": [
        {
            "class": "TextContainsKeywordsFilter",
            "args": {
                "keywords": ["in", "at"]
            },
            "inputs": {
                "sentence": "Andrew played cricket in India"
            },
            "outputs": true
        },
	{
            "class": "TextContainsKeywordsFilter",
            "args": {
                "keywords": ["sad"]
            },
            "inputs": {
                "sentence": "Andrew played cricket in India"
            },
            "outputs": false
        }
    ]
}

And then ran: pytest -s --f=keywords

It fails the test, although from my understanding it should still work properly. In particular, after printing self.keywords in the filter method, it seems like there is no new instance created for the new test case and the old keywords are still used which causes the second test case to fail.

Am I misusing something here? I ran into this when writing the tests for my addition of a filter.

[Kaustubh Dhole]

Thanks for pointing this out. Seems the filter was loading with the previously seen arguments (i.e. "in" and "at"). This should be fixed in this PR.

[Robin]

Thanks for taking a look at this and fixing the problem! Maybe the summary should be updated as well to say the number of test cases i.e. "2 passed in x seconds" instead of always staying at "1 passed in x seconds". Other than that, LGTM!

You might also need to re-trigger the tests for the filter PRs that got created while the PR of the fix isn't merged yet or manually test the failing filter PRs yourself.