sb-ai-lab / Sim4Rec

Simulator for training and evaluation of Recommender Systems

Home Page:https://sb-ai-lab.github.io/Sim4Rec/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Type check fails for complex types

monkey0head opened this issue · comments

If complex type is present in schema (e.g. ArrayType), it could not be compared using symmetric_difference as done here

def _check_names_and_types(df1_schema, df2_schema):
as the type is not hashable.
Need to rework this code part to compare only names and types (do not compare nullability) and be able to work with complex types.
Proposed: sort two lists with (name, DataType) as done now and compare columns one-by-one. Raise error on the first difference.

Add test showing the functionality works for simple and complex types.