This is the companion material for the paper "Helping novice architects to make quality design decisions using an LLM-based assistant", submitted to ECSA 2024.
The materials include:
- The source code implementing the different copilots for the ArchMind tool.
- A running demo of ArchMind via Streamlit.
- The data used for the experiments.
The copilots are implemented with Langchain and can be configured with your LLM of choice. For the experiments in the paper, we used OpenAI (an API KEY needs to be provided).
The Streamlit demo is configured to use Mistral (served through HuggingFace), so the results might differ from those in the paper and you might experience usage or connection limitations.
If the ArchMind tool is run locally, it needs to access to the patterns_chromadb and system_chromadb databases, which are based on ChromaDB. The first database stores information about well-known bibliographic sources of architectural knowledge, while the second database keeps information about the system under design.
In the data folder you'll find the ADRs used in the experiments, either generated by humans as well as generated by our tool (RAG and zero-shot modes). Some of the comparisons were made with the help of Deepeval.
For convenience, the prompts used by the copilots are located in the prompts folder.