bigdata-ustc / EduNLP

A library for advanced Natural Language Processing towards multi-modal educational items.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature] Optimize Tokenzation incluing multi-mode problems, Parser and Formula optimization

KenelmQLH opened this issue · comments

Description

(A clear and concise description of what the feature is.)

  • Handle multi-mode problems
    • AST Graph
    • Image
  • Handle noise problems when identify $...$ in Parser (need better rules)
  • Handle Formula ast problems when identify $AB=BC$ and $123$ (consider preprocessing)

References