Inspired by our hatred for maintaining documentation, we developed MindFlow, a code search engine powered by ChatGPT. MindFlow answers natural language queries about your repos and provides tools for streamlining your developer experience.
We efficiently index your code repo by leveraging hierarchical embeddings and summaries for files/directories. Furthermore, we keep them updated with each file hash so you can save those precious API tokens!
- Follow us on Twitter
- Join our discord
- BONUS: Consider becoming a patron ❤️
Pre-requisite: You'll need to create an OpenAI API account; you'll be able to do so here.
- Run
pip install mindflow,or you can clone this repo and runpip install -e path/to/mindflow. - Run
mf login {OPENAI_API_KEY}; you can find your OpenAI API key here. - Now, you're ready to start using MindFlow!
There are multiple levels to using mindflow's chat feature.
- Simplest
mf chat "explain what a programming language is"- Interact with chatGPT directly just like on the chatGPT website. We also have chat persistence, so it will remember the previous chat messages.
- With File Context
mf chat "please summarize what this code does" path/to/code.py- You can provide single or multi-file context to chatGPT by passing in any number of files as a separate argument in the
mf chatcall. For sufficiently small files (see: chatGPT token limits), this will work and also maintain chat history.
- You can provide single or multi-file context to chatGPT by passing in any number of files as a separate argument in the
- With Directory Context
mf chat "what are these submodules responsible for? path/to/submodule1/ path/to/submodule2/- Providing directories will actually run an indexer over your code subdirectories/files recursively. So it may take a while to fully index everything -- don't worry; we'll warn you if the cost becomes a concern! Right now the warning triggers if the index job costs >$0.50USD.
- Custom pre-indexed context
mf index path/to/subdir/ file1.txt path/to/file2.txtmf chat -s "How do all of my classes relate to one another?" ./- If you pre-index your repository, you can narrow the scope for the context provided to the chat. Passing
-swill skip the auto-indexing, and instead will defer to the currently existing index. This index is generated in the first stepmf indexwhere only those files/subdirs will be included. - This can save you time and money if your repository is significantly large.
- If you pre-index your repository, you can narrow the scope for the context provided to the chat. Passing
By default, simple chat messages (when referencing no files or very small files) will be stored locally so that you can retain chat persistence.
To see stats about your chat history, you can run mf history stats.
If you want to clear your chat history, you can run mf history clear and it will forget all previous messages that you've sent.
If you try adding directories to your chat messages, chat persistence will be disabled, and no previous context will be used. This will change as MindFlow matures, and the openAI API supports more token levels/conversation histories natively.
Note: Git diff summaries do not support chat persistence yet.
Make some changes to your git repo without staging/committing them. Then, run mf diff! You should get a response that looks like this:
`mindflow/commands/diff.py` changes:
- Added import statement for `List` and `Tuple` from the `typing` module.
- Added a function `parse_git_diff` that takes in the output of a `git diff` command and returns a list of tuples containing the file name and the diff content.
- Added a function `batch_git_diffs` that takes in the list of tuples returned by `parse_git_diff` and batches them into smaller chunks of diffs that are less than 3000 characters long.
- Modified the `diff` function to use the new `parse_git_diff` and `batch_git_diffs` functions to batch the diffs and send them to the GPT model for processing.
`mindflow/commands/inspect.py` changes:
- Removed the `print` statement used to output the result of a database query. The git diff shows changes in two files: `mindflow/commands/diff.py` and `mindflow/commands/inspect.py`.
`mindflow/commands/diff.py` changes:
- Added import statement for `List` and `Tuple` from the `typing` module.
- Added a function `parse_git_diff` that takes in the output of a `git diff` command and returns a list of tuples containing the file name and the diff content.
- Added a function `batch_git_diffs` that takes in the list of tuples returned by `parse_git_diff` and batches them into smaller chunks of diffs that are less than 3000 characters long.
- Modified the `diff` function to use the new `parse_git_diff` and `batch_git_diffs` functions to batch the diffs and send them to the GPT model for processing.
`mindflow/commands/inspect.py` changes:
- Removed the `print` statement used to output the result of a database query.
Make some changes to your git repo and stage them. Then, run mf commit! You should get a response that looks like this:
[formatting 7770179] Add needs_push() function and check in run_pr() function.
1 file changed, 14 insertions(+)
Make some changes to your branch and stage, and then commit them. Then, run mf pr for GitHub or mf mr for GitLab! A pull request/merge request should be created with a title and body generated by GPT, and a link to the PR should be printed to the console.
- To use this feature, you must first install and authenticate the GitHub CLI.
This tool allows you to build an index of text documents and search through them using GPT-based embeddings. The tool takes document paths as input, extracts the text, splits the documents into chunks, summarizes them, and builds a summarization tree. The tool then uses this tree to generate embeddings of the indexed documents and your query and selects the top text chunks based on the cosine similarity between these embeddings. The generated index can be saved to a JSON file for later reuse, making subsequent searches faster and cheaper.
In the future, MindFlow plans on becoming an even more integral part of the modern developer's toolkit. We plan on adding the ability to ditch traditional documentation and instead integrate directly with your private documents and communication channels, allowing for a more seamless and intuitive experience. With MindFlow, you can have a true "stream of consciousness" with your code, documentation, and communication channels, making it easier than ever to stay on top of your projects and collaborate with your team. We are excited to continue pushing the boundaries of what's possible with language models and revolutionizing how developers work.