Dagster - One Deployment Multiple Code Locations

[Ethan Leifer]

Right now each chain deployment needs to each run its own dagster server which is specified through the workspace.yaml in the dags package

This looks like this currently:

load_from:
  - python_module: dags
execution:
  config:
    multiprocess: 
      start_method:
        forkserver:

We can change it to the following

load_from:
  - python_file:
      relative_path: path/to/dataengineering_spark_team.py
      location_name: dataengineering_spark_team_py_38_virtual_env
      executable_path: venvs/path/to/dataengineering_spark_team/bin/python
  - python_file:
      relative_path: path/to/team_code_location.py
      location_name: ml_team_py_36_virtual_env
      executable_path: venvs/path/to/ml_tensorflow/bin/python

This will let us only have to deploy dagster once and we can just update the code locations. We can still keep the separate workspace.yaml for local development too.