cursorless-dev / cursorless

Don't let the cursor slow you down

Home Page:https://www.cursorless.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support migrating old recorded tests using bulk facet recorder

pokey opened this issue · comments

Once #2383 is merged, we should simplify the process of migrating old recorded tests to scope fixture tests using the new bulk fixture recorder.

We proceed as follows:

  1. Find all recorded tests with the given language id

  2. Canonicalize and inspect the command payload. We may want to filter to tests that use a specific set of actions and particular targets. We may also only want to migrate a specific subset of scopes, eg based on if they're marked as supported for the given language

  3. For each chosen test, get initial document state

  4. Skip if we've already processed a fixture with the same document state (or if there already exists a fixture with same document state?)

  5. Generate an entry in the bulk recorder file with a candidate header for each possible facet that has the given scope. For example:

    [[python]]
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    if aaa:
        pass
    ---
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    try:
        pass
    except:
        pass
    ---
    
  6. The user will then delete the unnecessary lines and clone the block as necessary eg to test both scope and iteration scope

  7. When the user saves, we should probably delete the recorded tests that they actually ended up migrating

To discuss: which tests to migrate?

We need to figure out how to decide which recorded tests to migrate. Some ideas:

  • All fixtures for a given language, regardless of whether they're marked as supported
  • Only ones marked as supported
  • User picks specific scope type

We also need to figure out if we want to drop recorded tests if their command payload doesn't match a particular shape. Eg skip if multiple scope types are mentioned, or of the command isn't one of a few expected commands

update from meet-up:

  • Only facets marked as supported
  • Only recorded tests in the folder languages/{languageId}
  • Only recorded tests with action of "chuck", "take", "pour", "clone", "drink", "clone up", "clear", primitive target, containing scope modifier

Migrate the above, and then see what's left and maybe make filter more lenient or just do them manually

Does this mean I should hold off an updating old tests for now to see how this goes? Porting the lua, bash, and nix tests are still on my list todo and was thinking recently about circling back to to it, but if it can mostly be automated that works too :D

Up to you. I've been adding new tests in new format, and some piecemeal migrations if I'm messing with the given scope, but waiting to do wholesale migrations till we have this