abinetha / Dyte_Project

Best-README-Template

An awesome README template to jumpstart your projects!
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

Log Data Retrieval is a comprehensive web-based application designed to simplify the retrieval and analysis of log data. The project includes features for searching, filtering, and visualizing log entries, making it an invaluable tool for managing large sets of log data.

Key Features:

Search Functionality: Easily search for log entries based on various criteria such as level, message, timestamp, resource ID, and more.

Date Range Filtering: Retrieve log entries within a specific date range to focus on relevant data.

Flexible Search Fields: Choose from a variety of search fields, including level, message, resource ID, timestamp, trace ID, span ID, commit, and parent resource ID.

JSON Parsing and Database Storage: Efficiently parse JSON files, store the log data in a MySQL database, and enable seamless retrieval.

Visualize Data: View log data in a structured tabular format on the website, providing a user-friendly interface for analysis.

(back to top)

Video Demo

Dyte_Project_video.4.mp4

Built With

• Django
• HTML & CSS
• Javascript
• Python
• MySQL

(back to top)

Getting Started

The Log Ingestor is a component of this project responsible for collecting and storing log entries. It allows you to search and filter logs based on various criteria.

Prerequisites

Make sure you have the following installed:

• Python (Version 3.10 or above)
• Django
• MySQL (Version 8.0 or above)

Installation

Clone the repository:

git clone https://github.com/abinetha/Dyte_Project.git

Navigate to the project directory:

cd Dyte_Project

Create a virtual environment:

python -m venv venv

Activate the virtual environment:

On Windows:

.\venv\Scripts\activate

On macOS/Linux:

source venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Run migrations:

python manage.py migrate

Start the development server:

python manage.py runserver

The project should now be running locally. Access the admin panel at http://127.0.0.1:3000/admin/ and log in with the superuser credentials.

Usage

1. Viewing Logs

Description: The Log Viewer allows you to easily view logs through a web-based interface.

Steps:

1. Open your web browser and navigate to the Log Viewer website (http://localhost:3000/).
2. The main page displays log entries in a tabular format.
3. Logs are automatically loaded from the database and updated periodically.

2. Searching Logs

Description: Use the search functionality to filter logs based on specific criteria.

Steps:

1. On the main page, locate the search form.
2. Choose a search field from the dropdown menu (e.g., Level, Message, Timestamp).
3. Enter the search text in the designated input field.
4. Optionally, specify a start and end date for a date range search.
5. Click the "Search" button.
6. The table will update to display logs that match the specified criteria.

3. Adding Logs

Description: Use the /save_json_data/ endpoint to add logs to the system.

Steps:

1. Make sure to include the json_file_path in the \json_viewer_project\json_viewer\views.py the json file in which you are adding data in the given format.
2. Make a POST request to http://localhost:3000/save_json_data/ with your JSON log data.
3. The system will respond with a success message if the data is saved successfully.
4. If a duplicate entry is detected, a message indicating the duplication will be returned.
5. In case of an error, an appropriate error message will be provided.

Example using cURL:

curl -X POST -H "Content-Type: application/json" -d '{"level": "info", "message": "Log entry content", "timestamp": "2023-11-18T12:00:00Z"}' http://localhost:3000/save_json_data/

4. Examples

Filter Logs:

To view all logs, simply navigate to the main page of the Log Viewer. Searching for Errors:

Select "Level" from the "Search Field" dropdown. Enter "error" in the "Search Text" input. Click "Search" to see logs with the "error" level.

Date Range Search:

Enter a start date and end date in the respective input fields. Click "Search" to see logs within the specified date range.

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". If you encounter issues, have suggestions, or would like to contribute to the project, feel free to open issues, submit pull requests, or provide feedback. Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

(back to top)

Acknowledgments

Use this space to list resources you find helpful and would like to give credit to. I've included a few of my favorites to kick things off!

• Django
• Python
• MySQL

(back to top)