Detect new plugins

[Jacob Milhorn]

We should cache the list of plugins used and if this changes make sure the plugin chains are re-run. This is harder to do with serialized functions which we tried when implementing change detection for options.