DataJunction / dj

A metrics platform.

Home Page:http://datajunction.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add moving / renaming / copying for nodes

shangyian opened this issue · comments

commented

Moving, renaming, and copying nodes share a lot in terms of implementation. Moving and renaming are actually exactly the same implementation-wise, since a DJ node's name contains its namespace.

We can expose two API endpoints to support all three actions:

  • Rename/Move Node: POST /nodes/{name}/rename?new_name={new_name}
  • Copy Node: POST /nodes/{name}/copy?new_name={new_name}

Steps

  1. Copy the node to a new entry in the Node table with the name altered to the new name.
  2. Copy the latest node revision to a new entry in the NodeRevision table with the name altered to the new name.
  3. Add a history event indicating that node rename happened.
  4. Propagate the node rename to all affected nodes.
  5. Deactivate the old node (this will leave node versions intact, but the node won't be directly visible to users).

Affected Node Types

Node Type Actions
Cube Renaming is not an issue, since there are never any downstreams. However, it may affect materialization.
Metric Renaming can affect any cubes that reference this metric. We'll need to find all downstream cubes and repoint the metric reference.
Source Renaming can affect all immediate downstream nodes [*]
Transform Renaming can affect all immediate downstream nodes [*]
Dimension Renaming can affect all immediate downstream nodes AND dimension links [**]

[*] For all affected immediate downstream nodes (note that it's not all downstreams, only the immediate ones), we can do the following:

  • Find the relevant immediate downstream nodes (dimensions, metrics, transforms only).
  • Repoint the reference in the query.
  • Update the parents saved in the db.

[**] For dimension links, no changes are required because all dimension links are done via ID.

commented

@agorajek I found this old issue and updated it with more details. We can discuss more tomorrow.

commented

Allow for copying multiple nodes together.

  • If some of the nodes are downstream of other nodes, we'll modify the queries of the copied nodes to source from the copied nodes.
  • All other attributes remain the same
POST /nodes/copy 
{
    "nodes": [
        {"name": "...", "new_name": "..."}
    ]
}

In the UI, we can allow users to select nodes they want to copy (including entire namespaces) and copy them to other namespaces/names.

commented

Functionality needed:

  • Ability to copy multiple nodes to new node names. The queries on any nodes that depend on each other should be modified during the copy to continue the dependency.
  • Ability to rename multiple nodes. The queries on any nodes that depend on each other should be modified during the rename to continue the dependency.
  • cascade will determine whether or not downstream nodes' queries are modified.
  1. Find all nodes that need to be copied or renamed. If cascade is true, also find any nodes downstream of these nodes.
  2. Build an object to track renaming (i.e., old node name -> new node name) + all the nodes where naming is needed.
  3. Copy all the nodes. For any nodes where renaming is needed, parse the query and do the rename.
  4. Finish.