Welcome to ClearCouncil, an innovative repository dedicated to democratizing access to local government information. This repository facilitates the collection, organization, and integration of publicly available PDF documents, including meeting minutes, agendas, actions, and other relevant materials from specific local state county councils. ClearCouncil also empowers users to query and obtain objective answers to local county-related questions through an embedded model.
- Retrieval Augmented Generation (RAG) Pipeline: Utilizing langchain for the creation of embedded models.
- Efficient Data Collection: Python, JavaScript, and wget scripts for automated document retrieval from county council websites.
- Data Parsing and Organization: Tools to parse and organize collected PDF files.
- Integration with Large Language Models (LLMs): The ability to seamlessly integrate data into embedded LLMs.
- Community Involvement: Encouraging contributions from the community, whether it's code, ideas, or data.
- Comprehensive Documentation: Detailed guides and documentation for effective tool usage.
ClearCouncil aims to achieve the following objectives:
- Empowering Informed Citizenship: Enabling users to ask simple questions about key county projects, using publicly available data on budget, county council votes, rezoning, ordinances, and more.
- Enhancing Transparency: Making local government records accessible and understandable to the general public.
- Supporting Research and Analysis: Facilitating in-depth analyses of local governance decisions and actions.
- Promoting Innovation in Governance: Advancing applications like AI-driven council analysis to improve local governance processes.
For setup instructions and details on running the provided scripts, please refer to our comprehensive 'Getting Started' guide in the documentation section.
- Implement PDF text extraction.
- Implement youtube transcript generation from public meetings
- Create an embedded model for PDF data.
- Extract metadata for specific doc types
- Implement a user-friendly web interface (streamlit maybe)
- Test, iterate and Improve search functionality.
- Add support for additional document formats.
- Integrate with ClearCouncil custom GPT
ClearCouncil is open to contributions! If you have suggestions or improvements, feel free to fork the repository and submit a pull request. Join us in making local government data more accessible and actionable for everyone!
- Python:
- JavaScript:
- langchain:
- OpenAI: