![Logo](https://raw.githubusercontent.com/paulmuenzner/powerplantmanager/main/assets/golang-manage-renewable-power-plants.png)
Administration - Logging - Statistical Analysis
EXPLORE DOCS
High Flexibility
·
Report Bug
·
Request Feature
Table of Contents
Introducing the Photovoltaic Plant Management System, a comprehensive software solution designed for seamless management of photovoltaic power plants. Users can register, verify accounts, and leverage a secure authentication system for streamlined access. Plant registration is made easy, allowing users to input technical details and upload related images and files for each power plant.
The system provides a dedicated Logging API for each registered power plant. This API facilitates the logging of various information sent by the power plants at configurable time intervals – a flexible solution for data tracking, whether it be at 15-minute intervals, 1-minute intervals, or a custom setting.
Security is a top priority, and the system implements robust measures to protect APIs. Users can secure their Logging APIs using keys, secrets, and IP filters. IP whitelisting adds an extra layer of security, ensuring that only authorized IP addresses have access.
In addition to efficient data logging, the Photovoltaic Plant Management System incorporates a statistical evaluation and data assessment feature. This functionality allows users, whether individual enthusiasts or businesses managing multiple photovoltaic installations, to analyze and interpret the logged data comprehensively. The system's data assessment tools provide valuable insights, making the management of solar assets straightforward and effective.
- Create and use accounts using registration, verification, signin, signout functionality
- Register your photovoltaic power plants and related technical information into the system and upload related plant images and files
- Create individual logging API for each registered power plant to log several information sent from our power plant in configured time intervals (eg. each 15 minuts, each 1 minute, ...)
- Protect your APIs with key, secret and IP whitelisting
- Validation middleware for individual assessments implemented for each route
- Validation handler for chained input validation individually customizable according to your own needs
- Flexible error handler covering 32 HTTP 4** status codes
- Flexible response handler for positive 2** responses covering 10 HTTP status codes
- Central collection, overview and management of configuration values in config folder
- Central Regex Expression library with several predefined and tested expressions
- Setup of MongoDB connection including prepared templates to query, save and delete
- Logging with logrus to daily log files in log folder: Logging error, time, log level.
- Deletion of log files in log folder older than 5 days (default). Can be changed in base_config file with 'DeleteLogsAfterDays'
- AWS functions to upload and delete files and images for your power plant to S3 change their names
- Strict file upload validations: number, sizes, type
- Fully-featured cookie handler
- This Golang server boasts a robust architecture designed for flexibility, reduced coupling and testibiliy through a dedicated Dependency Injection (DI) setup. The core functionalities of database communications, and AWS operations and sending email notifications are seamlessly integrated, providing a cohesive and modular solution.
- Context extensions enabling to reuse queried data amongst more than one controller within one route an thus reducing number of database requests
- JWT setup needed for authentication and authorization
- Documentation an description of function an controllers
- Practical examples for testing with Go Testify
- Transactions implemented. Transactions in MongoDB ensure the atomicity, consistency, isolation, and durability (ACID) properties for multiple database operations, allowing developers to group multiple statements into a single unit of work that either succeeds entirely or fails without leaving the database in an inconsistent state.
- Encryption, Decryption, token and hashing functions
- Implemented dependency injection to manage dependencies and improve testability.
- Email setup for sending transactional emails with your provider
- Key, Secret & URL ID Protection: Secure Logging APIs using unique keys, secrets and an individual URL id.
- IP Filtering: Implement IP whitelisting to allow logging access only to authorized IP addresses.
- Validation Middleware: Validation middleware for individual assessments implemented on each route.
- Validation Handler: Implement a validation handler for chained input validation, expandable and customizable to specific needs.
- Error Handling: Flexible error handler covering 32 HTTP 4xx status codes for robust error management.
- Response Handling: Flexible response handler for positive 2xx responses, covering 10 HTTP status codes.
- Cookie Security: Fully-featured cookie handler for enhanced security.
- JWT Authentication: Set up JWT for authentication and authorization.
- Verification token: Encrypted verification token for verifying email addresses and registrations.
- Encryption and Hashing: Implement encryption, decryption, and hashing functions for data security.
- Robust Error Handling Mechanism: Any encountered errors are diligently logged to the designated log folder and simultaneously partly dispatched via email notifications.
- Method Validation: Custom middleware to check request methods with the allowed methods parameter.
- Provides partial defense against Slowloris attacks.
- Rate limiter implemented.
This project is basically built with and for:
Prior to launching the program, clone the repo, install go dependencies and ensure that all configurations are set.
- Make sure MongoDB is installed and available.
- Make sure a properly configured AWS S3 Bucket is ready.
- Clone the repo
git clone https://github.com/paulmuenzner/powerplantmanager.git
- Install go dependencies by running
go get
Before running the program, you need to set up the required environment variables by creating a .env file in the root directory of the project. This file holds sensitive information and configurations needed for the proper functioning of the application. To streamline the process, the provided template for environment variables saves time in creating the .env file. Simply convert .env.template file to .env.
AWS S3 & MongoDB Configuration:
If your application involves interactions with AWS S3, you must provide the following key-value pairs in the .env file:
- AWS_S3_BUCKET_NAME: The name of your AWS S3 bucket.
- AWS_REGION: The AWS region where your S3 bucket is located.
- AWS_ACCESS_KEY_ID: Your AWS access key ID.
- AWS_SECRET_ACCESS_KEY: Your AWS secret access key.
- MONGODB_SCHEME: MongoDB Scheme (likely mongodb)
- MONGODB_HOST: MongoDB Host (localhost if self-hosted running locally. Read more on mongodb.com.)
- MONGODB_PORT: MongoDB Port, eg. 27018 or standard port 27017. Read more on mongodb.com.
- MONGODB_DATABASE_NAME: Name of your MongoDB database you like to backup.
Email Notification Configuration:
If you intend to use email notifications (configured with SendEmailNotifications in the config file), include the following additional variables in your .env file:
- EMAIL_PROVIDER_PASSWORD: Password for the email provider.
- EMAIL_PROVIDER_USERNAME: Username for the email provider.
- EMAIL_PROVIDER_SMTP_PORT: SMTP port for the email provider.
- EMAIL_PROVIDER_HOST: Hostname of the email provider.
- EMAIL_ADDRESS_SENDER_BACKUP: Sender email address for backup notifications.
- EMAIL_ADDRESS_RECEIVER_BACKUP: Receiver email address for backup notifications.
- MONGODB_USERNAME: Username as part of your MongoDB connection string if needed. Read more on mongodb.com.
- MONGODB_PASSWORD: Password as part of your MongoDB connection string if needed. Read more on mongodb.com.
Make sure to keep your '.env' file secure and do not share it publicly.
The program relies on these configurations to run successfully. Without the correct values in the .env file, certain features may not work as expected.
Here's an example .env template in code format. Replace "your-..." placeholders with your actual values. Ensure that this file is kept secure, and sensitive information is not shared publicly. Users should fill in the appropriate values for their specific configurations.
# Port
PORT=your-port
# Token and Secrets
KEY_VERIFY_TOKEN=your-key-verify-token
JWT_SECRET_KEY=your-jwt-secret-key
# AWS S3 Configuration
AWS_ACCESS_KEY_ID=your-access-key-id
AWS_SECRET_ACCESS_KEY=your-secret-access-key
AWS_REGION=your-aws-region
AWS_S3_BUCKET_NAME=your-s3-bucket-name
# Database Configuration
MONGODB_SCHEME=mongodb
MONGODB_USERNAME=your-mongodb-username # Optional
MONGODB_PASSWORD=your-mongodb-password # Optional
MONGODB_HOST=localhost
MONGODB_PORT=your-mongodb-port # Likely 27017
# Email Notification Configuration (Optional)
EMAIL_PROVIDER_PASSWORD=your-email-provider-password
EMAIL_PROVIDER_USERNAME=your-email-provider-username
EMAIL_PROVIDER_SMTP_PORT=your-smtp-port
EMAIL_PROVIDER_HOST=your-email-provider-host
EMAIL_ADDRESS_SENDER_BACKUP=your-sender-email-address
EMAIL_ADDRESS_RECEIVER_BACKUP=your-receiver-email-address
The following configurations can be modified in the config file located at => /config/... All values are pre-configured with default settings, ensuring the server is ready to run and the program is ready for use.
Key | Description | Type | Example |
---|---|---|---|
URL | The URL for accessing the server hosting this program. | string | https://www.example.com |
WriteTimeout | This parameter determines the maximum duration allowed for the server to write a response to a client. This ensures timely completion of write operations. The timeout is set to config.WriteTimeout seconds, providing flexibility in adjusting the duration based on specific requirements. | int | 5 |
ReadTimeout | This parameter parameter sets the maximum duration permitted for the server to read an entire request from a client. It helps manage the time allocated for processing incoming requests. The timeout is configured to config.ReadTimeout seconds, allowing customization based on the desired duration. | int | 20 |
IdleTimeout | This parameter dictates the maximum duration the server can keep an idle (keep-alive) connection open. This is crucial for optimizing resource usage and maintaining efficient connections. The timeout is adjusted to config.IdleTimeout seconds, providing control over the duration of idle connections. | int | 60 |
DeleteLogsAfterDays | Errors are logged to the 'log/' folder, with log file names assigned based on the day. All logs generated within a day are consolidated into a designated backup file. This parameter determines the number of days after which log files will be automatically deleted. | int | 5 |
PlantNameLength | Maximum permitted length of a plant name each user can register. | int | 50 |
IntervalSecDefault | Default interval, in seconds, for enabling data logging to the plant logger. | int | 15 * 60 |
DatabaseNameUserAuth | Name of the database used to store user authentication information. | string | PlantDB |
DatabaseNameFiles | Name of the database used to store uploaded file information. | string | PlantDB |
DatabaseNamePlants | Name of the database used to store photovoltaic plant information. | string | PlantDB |
DatabaseNamePlantLoggerConfig | Name of the database used to store individual logger settings for each plant separately. | string | PlantDB |
DatabaseNamePlantLogger | Name of the database used to store collections for each plant logger. Each plant has separate collection in this database. | string | PlantDBLogger |
UserAuthCollectionName | Name of the collection used to store user auth information. | string | user_auth |
CollectionNameFiles | Name of the collection used to store file information. | string | files |
CollectionNamePhotovoltaicPlant | Name of the collection used to store plant information. | string | pv_plants |
CollectionNamePlantLoggerConfig | The first part of the name of the plant collection is used to store logs, while the second part is a randomly generated ID (Eg. plant_logger_config_365889547). | string | plant_logger_config |
MinLengthPassword | Minimum permitted length of provided user passwords for account registration. | int | 12 |
MaxLengthPassword | Maximum permitted length of provided user passwords for account registration. | int | 50 |
MaxLengthEmailAddress | Maximum permitted length of provided user email address for account registration. | int | 60 |
AuthCookieLifetimeSeconds | Lifetime of authentication cookies. | int | 1 * 60 * 60 |
AuthCookieJWTLifetimeSeconds | Lifetime of JWT token for authentication cookies. | int | 1 * 60 * 60 |
AuthCookieName | Name of authentication cookies. | string | authCookie |
TimeValidVerifyTokenMinutes | Validity of verification token used for finalization registration process. | int | 15 |
EmailSendNotifications | Decide whether you want to send email notifications or not. Emails are sent in both cases error and successfully completed backup. | bool | false |
EmailProviderUserNameEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_PROVIDER_USERNAME" |
EmailProviderPasswordEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_PROVIDER_PASSWORD" |
EmailProviderSMTPPortEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_PROVIDER_SMTP_PORT" |
EmailProviderHostEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_PROVIDER_HOST" |
EmailAddressSenderEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_ADDRESS_SENDER_BACKUP" |
EmailAddressReceiverEnv | Name of .env key. The value behind this .env key is placed in your .env file. Needed, if you want to send transactional email notifications. Ask your provider for this value. | string | "EMAIL_ADDRESS_RECEIVER_BACKUP" |
S3BucketEnv | Name of .env key to configure bucket name. The value behind this .env key is placed in your .env file. Needed, to configure AWS S3. Check your S3 AWS dashboard for this value. The bucket with the exact same name must be ready in your AWS account. | string | "AWS_S3_BUCKET_NAME" |
S3RegionEnv | Name of .env key to configure S3 region. The value behind this .env key is placed in your .env file. The region with the exact same name is mentioned in your AWS account. | string | "AWS_REGION" |
S3AccessKeyEnv | Name of .env key to add S3 access key. The value behind this .env key is placed in your .env file. The access key is available in your AWS account. | string | "AWS_ACCESS_KEY_ID" |
S3SecretKeyEnv | Name of .env key to add S3 secret key. The value behind this .env key is placed in your .env file. The secret key is available in your AWS account. | string | "AWS_SECRET_ACCESS_KEY" |
MongoDatabaseSchemeEnv | Name of .env key to define a MongoDB scheme. The value behind this .env key is placed in your .env file. | string | "MONGODB_SCHEME" |
MongoDatabaseUsernameEnv | Name of .env key to define a MongoDB user name if needed. The value behind this .env key is placed in your .env file. | string | "MONGODB_USERNAME" |
MongoDatabasePasswordEnv | Name of .env key to define a MongoDB password if needed. The value behind this .env key is placed in your .env file. | string | "MONGODB_PASSWORD" |
MongoDatabaseHostdEnv | Name of .env key to define a MongoDB host. The value behind this .env key is placed in your .env file. | string | "MONGODB_HOST" |
MongoDatabasePortEnv | Name of .env key to define a MongoDB port number. The value behind this .env key is placed in your .env file. | string | "MONGODB_PORT" |
Run program by: go run main.go
or use live-reloader such as air with air
This server exposes various endpoints to facilitate authentication, file management, and plant-related operations.
The authentication API '/auth' offers secure endpoints for user authentication. Response format is always JSON besides file upload which is form-data.
-
/auth/registration
- Method: POST
- Description: Register a new user with the system.
- Authentication Required: No
- Request Body Example:
{ "email": "myname@emailexample.com" }
-
/auth/verify
- Method: POST
- Description: Verify registered account with link using verify token sent to the registered email address.
- Authentication Required: No
- Request Body Example:
{ "password": "StrongPassword", "passwordVerify": "StrongPassword", "verifyToken": "y6X2NpA8bPAAdLZRxUT7aqNL-e9eqDcNuIG0VosGg96G0vr4jAkpwGLkDeI0fZwlsfkA65PreOiuZpza4M66cWrz7QYKicjfKWdUUdOAe-v0_EYiA-3D19pK3CS-Vj1U" }
-
/auth/signin
- Method: POST
- Description: Sign in with valid credentials.
- Authentication Required: No
- Request Body Example:
{ "email": "myname@emailexample.com", "password": "StrongPassword" }
-
/auth/signout
- Method: POST
- Description: Log out the currently authenticated user.
- Authentication Required: Yes
- Request Body Example:
{}
The files API '/files' provides functionality for managing files and documents.
-
/files/upload-file
- Method: POST
- Description: Upload a file to AWS S3 bucket and create related document in file collection storing meta data and owner (user). Key for file form-data is 'files'.
- Authentication Required: Yes
-
/files/delete-file
- Method: DELETE
- Description: Delete a file from the server. Provide file id in request body.
- Authentication Required: Yes
- Request Body Example:
{ "publicFileId": "374755771952217" }
The plants API '/plants' offers endpoints for managing plant-related data.
-
/plants/add
- Method: POST
- Description: A signed-in user can add new power plants as desired by providing a new power plant name. This creates a separate logging collection, configuration document and plant document. Collection and database names can be configured in 'congig/' folder.
- Authentication Required: Yes
- Request Body Example:
{ "name": "MyPlantName" }
-
/plants/log/{apiID:[0-9]+}
- Method: POST
- Description: API logging power plant details for a specific plant by providing its ID. Only a numerical ID is accepted.
- Authentication Required: No. However, valid key, secret and apiID are requiered. Furthermore requesting IP must be whitelisted.
- Request Body Example:
{ "key": "7446579140876818687525890004949221730587", "secret": "c2a1d375159502956e552e0e5d57de6735ec", "voltageOutput": 40, "currentOutput": 2.87, "powerOutput": 114.8, "solarRadiation": 246, "tAmbient": 5, "tModule": 5, "relHumidity": 77, "windSpeed": 5 }
-
/plants/setconfig
- Method: PUT
- Description: Modify configuration settings for plant.
- Authentication Required: Yes
- Request Body Example:
{ "publicPlantId": "970407102018637", "ipWhiteList": ["2001:0db8:85a3:0000:0000:8a2e:0370:7334"], "intervalSec": 8000 }
-
/plants/keysecret
- Method: PUT
- Description: Create new key and secret needed for utilizing logging API from point 2).
- Authentication Required: Yes
- Request Body Example:
{ "publicPlantId": "970407102018637" }
-
/plants/delete
- Method: DELETE
- Description: Delete own plant together with related configuration document and its logging collection created in point 1).
- Authentication Required: Yes
- Request Body Example:
{ "publicPlantId": "970407102018637" }
-
/plants/statistics
- Method: GEt
- Description: Retreaving statistical analysis for a provided period.
- Authentication Required: Yes
- Request Body Example:
{ "dateStart": "2022-12-11T12:23:57.734+00:00", "dateEnd": "2023-12-21T12:23:57.734+00:00", "publicPlantId": "970407102018637" }
Feel free to explore and integrate these API routes into your applications! If you have any questions or need further assistance, please refer to the detailed documentation for each route.
Possible calculations:
- Interquartile Range
- Lower bound
- Mean
- Median
- Outliers
- Quantile 25, Quantile 75, Quantile 90, Quantile 95
- Skewness
- Standard deviation
- Upper bound
- Variance
- ✅ Storing images on and implementing AWS S3
- ✅ Extend email notification feature
- ⬜️ Addressing more nuanced linting issues.
- ⬜️ Implement brute-force protection for authorization process
- ⬜️ Extend testing
- ⬜️ Add option to backup and upload MongoDB database to S3
See the open issues to report bugs or request fatures.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
Contributions are more than welcome! See CONTRIBUTING.md for more info.
Distributed under the GNU General Public License v2.0. See LICENSE for more information.
Paul Münzner: https://paulmuenzner.com
Project Link: https://github.com/paulmuenzner/powerplantmanager
Use this space to list resources you find helpful and would like to give credit to. I've included a few of my favorites to kick things off!