projectayre / ayre

Visual Question Answering with added novel Semantic Analysis approach. Inspired in part by Armored Core 6: Fires of Rubicon.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ayre

"... I'll support you!"

Ayre is a Visual Question Answering project created by members of this organization as part of our semester V projects and coursework during our Bachelor's degree. This is primarily a research based project, since Visual Question Answering with Sentiment Analysis is a significant research gap that we would like to address. This project will feature several components coming together as a complete full-stack AI project.

Planned/Implemented features [checklist]

AI

All AI development to done using Torch, for better support with transformers. (20/11/2023: Proof of concept is complete)

  • Sentiment Analysis on images using integrating fuzzy logic.
  • NEW: Semantic Image Segmentation for Improved Understanding of the image.
  • Visual Question Answering task using data from semantic segmentation
  • MLOps pipelines that can be easily imported and deployed inside the backend.
  • (Future Work) At least 1 Journal and 1 Conference paper based on novel approaches made here.
  • (Optional) HuggingFace integration (Probably future work now).

Frontend

A simple, beautiful UI/UX with the following features:

  • Responsive Design for accessibility.
  • Light/Dark themes.
  • OAuth integration, with some of the benefits.
  • Deployment on Vercel/Github Pages.
  • (Optional) A Loading screen.

Backend

  • Minimal and fast backend created in FastAPI.
  • Modular design for improved CI/CD.
  • Basic self-hosting for anywhere access.
  • (Future Work) Database connectivity for live storage of queries and information.
  • (Optional) Improved cloud-based hosting with accessible APIs.

About

Visual Question Answering with added novel Semantic Analysis approach. Inspired in part by Armored Core 6: Fires of Rubicon.


Languages

Language:Jupyter Notebook 98.9%Language:TypeScript 0.5%Language:Python 0.5%Language:JavaScript 0.0%Language:CSS 0.0%