iofu728 / Model_retrieval

πŸš‘Some ML Model Retrieval

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Spider logo

Model retrieval

GitHub GitHub tag GitHub code size in bytes

Some ML Model retrieval


VSM = Vector Space Model

This a hand write VSM retrieval

β”œβ”€β”€ utils
β”‚   └──         // public function
└── vsm
    β”œβ”€β”€           // data preprocessing shell
    └──           // vsm py

VSM process:

  1. word alignment
  2. TF - IDF (smooth, similarity)
  3. one by one calaulate
  • VSM.vsmCalaulate()
    • Consider about bias by smooth
    • Choose one tuple(artile1, artile2) have specific (tf-idf1, tf-idf2)
    • In this way, we have low performance, even we have two class Threadings
  • VSM.vsmTest()
    • Ignore bias by smooth
    • Calculate tf-idf in the pre processing which decided by artile instead of tuple(artile1, artile2)
    • In this way, we have fantastic performance
    • We calculate dataset of 3100βœ–οΈ3100 in 215s


SMN = Sequential Matching Network

some change from MarkWuNLP/MultiTurnResponseSelection

β”œβ”€β”€ NN
β”‚   β”œβ”€β”€              // CNN function
β”‚   β”œβ”€β”€       // classifier function
β”‚   β”œβ”€β”€     // NN optimization function
β”‚   β”œβ”€β”€              // RNN function
β”‚   └──     // sgd function
β”œβ”€β”€ SMN
β”‚   β”œβ”€β”€       // pre deal function
β”‚   β”œβ”€β”€         // model function
β”‚   β”œβ”€β”€       // cnn pool & conv
β”‚   └──    // got negative and true sample
└── utils
    β”œβ”€β”€         // constant parameter
    └──            // public function

SMN process:

  1. word embemdding
  2. GRU
  3. CNN
  4. GRU
  5. score
  • SMN.PreProcess.ParseMultiTurn(input_file)
    • prepare deal sample to matrix
  • SMN.PreProcess..ParseMultiTurnTest(input_file)
    • prepare deal test sample to matrix
  • SMN.sampleConduct.preWord2vec(input_file, out_file)
    • embedding sample
  • SMN.sampleConduct.SampleConduct()
    • got negative & true sample
  • SMN.SMN_Last.run_model()
    • run SMN model









πŸš‘Some ML Model Retrieval

License:MIT License


Language:Python 94.7%Language:Jupyter Notebook 4.4%Language:Shell 0.9%