karpathy / llama2.c

Inference Llama 2 in one file of pure C

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Please implement a project

win10ogod opened this issue · comments

It does not use any neural network components except multi-head attention, nor does it use traditional neural network structures. Instead, it directly uses the multi-head attention mechanism to process
The following is the framework of the program:

  1. Think of folders as a replacement for neural networks
  2. Not using traditional neural networks
  3. Each sub-folder has 6 attention heads
  4. Use files in the folder as data
  5. Do not use files for training, but directly use all the files in the folder as the neural network
  6. No neural network components are used except multi-head attention.
  7. Attention prediction based on user input text
    I thought of it yesterday but didn’t have the ability to implement it.
    Please @karpathy verify my idea.
    This system does not learn by training the model, but directly uses the text files in the folder as data to make predictions.