Make lookup (enumeration) table available as project variable

[signedav]

QGIS Enhancement: Make lookup (enumeration) table available as project variable

Date 2022/23/12

Author Dave Signer @signedav

Contact david at opengis ch

maintainer @signedav

Version QGIS 3.24

Summary

When having enumeration values stored in lookup-tables,the use of expressions is complex in syntax and intense in computing.

As a reference to the lookup-table usually their id is used as foreign key. So we need to compare the current attribute value with the lookup-tabel's id. So we need to get this id according to the enumeration value like this:

attribute(get_feature([...])

The goal of this proposal is to improve readability and performance of this type of expressions by making those enumeration values (and ids) available as project variables.

Proposed Solution

Being able to add project variables with a type called "enumtable" where we specify the table, a key column and a value column.

The values are read from the table and written into a "key/value" map, stored as value of this project variable.

This map is always generated and not saved in the project file. Means we will have an internal helper method refreshLookupCache(layer) that is called during project load or when the configuration changes during runtime.

Like this, in the expression those values would be available with @projectvariable[value] to receive the key.

Example

Table "house_table":

tid |        name       | building_type |
----+-------------------+----------------
  1 | House in the Green|           102 | 
  2 | Big Glass House   |           104 | 
  3 | Brutalist Block   |           103 | 

Table "building_type_table":

  id | description | info   |
-----+-------------+---------
 101 | Office      | xy     | 
 102 | Appartement | xy     | 
 103 | Shop        | xy     | 
 104 | Public      | xy     | 

Expression we can (currently) use to check the building_type description in the table "house_table"

attribute( get_feature( 'building_type_table', 'id', building_type), 'description') = 'Office'

Being able to add project variables of type "enumtable"

• Add Project Variable called "buildingtypes" lets you choose a type (VariableDefinition) "String"/"Enumtable"/...
• When "Enumtable" is chosen you can set a table (building_type) a key column (id) and a value column (description) -> this is stored to the VariableDefinition of the project variable.
• As well a map is stored as the value of the projet variable: ['Office':101,'Appartement':102,'Shop':103,'Public':104]

The variable "buildingtypes" is stored in the project with the type "Enumtable" and the configured table "building_type", key column "id" and value column "description". The map wont be stored in the project - it will be filled at runtime (when loading the project or changing the configuration).

Expression we can (with this implementation) use to check the building_type in the table "house_table"

building_type = @buildingtype['Office']

API proposal

struct VariableDefinition {
  QString name;     // buildingtype
  QString type;     // EnumTable
  QVariantMap config;  // {layer: "building_type", key: "id", value: "description"}

  // helpers to persist settings,
  writeXml();
  static readXml();
};

QList<VariableDefinition> QgsProject::variableDefinitions();

// updates based on the name
QgsProject::setVariableDefinition( const VariableDefinition& )

// delete
QgsProject::deleteVariableDefinition( const QString &name ) 

// Adjust to also return generated values --> check how other generated values are handled here:
QgsExpressionContextUtils::projectScope()

Open questions

• Should we connect to edit signals on enumumeration table layers (lookup table) to update the cached map?
• What is it the correct point in time to load the layer data (while loading the project might be too early, better post loading probably)?

Performance Implications

Could have impacts on project loading but is faster than get_feature functions in the expressions.

Backwards Compatibility

No impacts.

Votes

(required)

[Nyall Dawson]

struct VariableDefinition

There's a compelling argument that expression context variables could be typed in all contexts, not just project variables. I'd suggest this struct be moved outside of QgsProject to the QgsExpressionContext classes. (Also promote to class, with getters/setters for future flexibility)

QString type; // EnumTable

Could we use QVariant meta types here instead? That'd help the generic typed-context variable case

[Nyall Dawson]

building_type = @buildingtype['Office']

To me this looks somewhat of a Frankenstein solution -- I'm not convinced that treating the enum as a variable is a good solution. Have you considered making this a function instead? Something like project_enum('name','key')?

[signedav]

Thanks for your inputs @nyalldawson

To me this looks somewhat of a Frankenstein solution -- I'm not convinced that treating the enum as a variable is a good solution. Have you considered making this a function instead? Something like project_enum('name','key')?

Does this input just concern the syntax in the expressions or the general idea to store the enumerations as maps in the project variables?

If it's just about the syntax, I would not mind that much to have a function like you proposed, but still I think it would be expected to be able to access the variables with the proposed syntax. Or what should be the output of @buildingtype otherwise?

[signedav]

To resume the discussion here, I wanted to ask you @nyalldawson again, if it looks like a Frankenstein solution for you because of the syntax or the general idea to store the enumerations as maps in the project variables?