Loss function ideas

[Miles Cranmer]

Thought of a couple of improvements to the loss function.

1. The complexity frequencies should be a moving count, rather than summed over the entire search. This is because a new expression may be encountered later in the search that is much simpler, and so the frequencies need to be reset to account for this.
2. The adaptive parsimony should actually be part of the loss function. Right now it is part of the simulated annealing which doesn't make much sense - it will just result in expressions being kept far more often than is necessary.
3. When an expression is kept (a mutation fails), the expression should not replace the oldest expression - it should simply leave the population as is!

[Miles Cranmer]

3. is fixed by MilesCranmer/SymbolicRegression.jl@e1aa4c1, which adds the skip_failed_mutation option.

Edit: now included in PySR 0.7.8 as default.

[Miles Cranmer]

These should all be implemented now.