Fjob backend
FJob is a fullstack application (main technologies: Django Rest Framework, Nextjs, Selenium). FJob, by collecting job offers from many job portals (currently only JustJoinIT, but there are already scripts ready to collect data from: olx, Pracujpl, nofluffjobs, Pracujpl) and enabling companies to post their own job offers will allow you to easily search for job offers.
Report Bug
Β·
Request Feature
System Architecture
Fjob consists of 3 repositories:
- Backend https://github.com/DEENUU1/fjob_backend
- Frontend https://github.com/DEENUU1/fjob_frontend
- Scrapers (current)
Features
- JWT and OAuth Google Integration:
- Utilizes JSON Web Tokens (JWT) in conjunction with OAuth Google for secure authentication and authorization processes.
- Web Scraping and API-driven Database Storage:
- Implements web scraping techniques to extract job offers from various websites.
- Processes and transmits the gathered data via API for storage in the database.
- Error Reporting and Favorites Functionality:
- Incorporates a robust error reporting mechanism.
- Enables users to save job offers to favorites for convenient retrieval.
- Advanced Filtering, Sorting, Pagination, and Search Capabilities:
- Provides sophisticated filtering, sorting, and pagination options for efficient data exploration.
- Facilitates advanced job search functionalities.
- Stripe Integration for Business Listings Purchase:
- Integrates with Stripe payment gateway, allowing businesses to purchase listings seamlessly.
- User and Company Account Management:
- Enables the creation of both regular user accounts and company accounts.
- Django Signals for Automated Services:
- Utilizes Django Signals for the automation of various services, including updating available job offers, sending email notifications post-payment, and handling errors in job offers.
- Instantiates Company objects following the creation of a company account.
- Custom Commands and Permissions:
- Implements custom commands and permissions for enhanced control and flexibility.
- Default Data Upload and Localization:
- Includes functionality for uploading default countries, cities, and regions (currently focused on Poland).
- NextJS and Typescript for Frontend:
- Develops the frontend application using NextJS and Typescript for a robust and type-safe user interface.
- CI/CD with GitHub Actions:
- Implements Continuous Integration/Continuous Deployment (CI/CD) using GitHub actions.
- Automates unit tests and deploys the latest code version to an AWS EC2 instance.
- NGINX Configuration for Network Traffic Handling:
- Utilizes NGINX on the EC2 instance to efficiently manage network traffic.
- SSL Certificate Renewal with Certbot:
- Configures Certbot for automatic SSL certificate renewal, ensuring secure communication.
- Background Task Handling with Redis and Celery:
- Incorporates Redis and Celery for background task processing.
- AWS Integration for Scalability and Reliability:
- Integrates Django Rest Framework with AWS S3 Bucket for storing static files and model data.
- Utilizes Google Cloud SQL for a reliable and secure database.
- Leverages AWS services such as CodeBuild for web scraping functions on Lambda, ElasticIP Addresses for fixed IP assignment, Route 53 for domain management, and Amplify for hosting and deployment.
- Event-Driven Architecture with AWS EventBridge:
- Implements AWS EventBridge to automatically trigger each web scraping function.
- Sentry Integration for Error Monitoring:
- Integrates with the Sentry platform to monitor and analyze errors in the system.
Technologies:
Backend
- Python
- Django
- Django Rest Framework
- Celery
- Docker
- Docker Compose
- Pytest
- Redis
- CI/CD
- AWS EC2
- AWS S3 Bucket
- AWS Elastic IP
- AWS Route 53
- Certbot
- NGINX
- Google Cloud SQL (postgre)
- SQLite
- Stripe
Frontend
- Nextjs
- Typescript
- AWS Amplify
Scrapers
- Python
- Selenium
- Requests
- Beautiful Soup 4
- AWS Lambda
- AWS EventBridge
- AWS CodeBuild
Local environment installation
Installation
# Clone repository
git clone <url>
# Install libraries
pip install -r requirements
# Create .env file
cp .env_example .env
Authors
License
See LICENSE.txt
for more information.