Giters

fiftin / vUpdate

Cloud-native GitOps system without K8s

Home Page:https://vupdate.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

cicddevopsdevops-toolsgitops

Table of Contents generated with DocToc

Introduction

Update-services performs versioning, updating, starting and monitoring of services. History storage allows you to roll back changes. Centralized storage of log journals and crash reports is convenient for monitoring system performance and troubleshooting errors.

Required platforms

Currently, update-services works on Linux Ubuntu servers. Source codes must be in the Git VCS repositories. Microsoft Azure cloud service is supported.

Agreements

The following is accepted to display the structure of Json files.

Pair name/value:

  • name : string
  • name:number : number
  • name:boolean : boolean

Object

  • name
    • contents

Array

  • name[] - strings array
  • name:number[] - numbers array
  • name[] - objects array
    • ... - contents

Optional value:

  • [name]

Services

A service is an entire application, or a part of it. For example, a service could be an executable file with a set of shared libraries, or only shared libraries.

For each service, the assembly version and installation rules must be described. If the service is an executable program, the startup rules are also described for it, logging and crash reporting.

Service lifecycle

  • Creating a development version of the service
  • Creation of a client version of the service with customization of settings for a specific client
  • Installing/Upgrading a Service Instance
  • Для выполнимых сервисов:
    • Run service instance
    • Writing service logs to database
    • Write service crash reports to database

Service profiles

Profiles are named groups of services. Profiles are used to designate distributed groups of services.

Service role

The service can perform one of the assigned roles. The role can be specified in the launch options or in the configuration file.

Service versions

When creating a service, an archive of installation files and meta information about this archive are created, marked with a unique version number.

Developer version

Format of developer version is distribution-build:

  • distribution - the name of the distribution server on which the version was created.
  • build - build number from numbers separated by dots, for example 4.21.2.

For the distribution server jetto and build 3.5.1, there will be a version jetto-3.5.1.

Client version

The client version is created from the developer version on the distribution server where it will be used. The client version contains customization of the developer's version for a specific client. It can specify specific settings, set keys, certificates, etc.

The initial client version has the same name as the developer version it was created with. If the client settings have changed, the client version is recreated. The new version will end with ._N, where N - client version generation number. For example, created from the developer's version jetto-3.5.1, the client version will be called jetto-3.5.1, next builds will be called jetto-3.5.1_1, jetto-3.5.1_2, etc.

Desired versions

The distribution server stores a number of the latest generated versions of the service.

In order to indicate which version to use at the current moment, is intended the list of desired versions in the format service-version.

There is a list of desired developer versions and desired client versions.

Desired versions are changed in a whole list, since the versions of the services depend on each other.

When generating a new version, by default, the new version number becomes desirable for the given service. The desired version number can be changed manually.

Services settings

Settings for updating and starting services

The file update.json is created in the main service development repository. It describes the rules for building, installing and running.

File format:

  • update[] - service descriptions
    • service : service name
    • build - service version assembly rules
      • [buildCommands[]] - version build commands
        • command : build command
        • [args[]] - build options
        • [env[]] - command run environment
          • 'name' : value
        • [directory] : directory to execute, if not specified, current
        • [exitCode:number] : exit code
        • [outputMatch] : matching output to stderr and stdout, regular expression
      • copyFiles[] - files and directories to copy to assembly
        • sourceFile : source file in the service repository
        • destinationFile : file in version image
        • except[] - exclude this files
        • [settings[]] - macro values in source files. When copied to an assembly, the macro is replaced with a value.
          • 'macro name' : macro value
      • install
        • [installCommands] : version install commands, as in buildCommands
        • [postInstallCommands] : commands after installing the version, as in buildCommands
        • [runService] - service start rules
          • command : run command
          • [args[]] - run arguments
          • [writeLogs] - write service logs to local disk
            • directory : directory to write logs
            • filePrefix : start of log file name, full name .
            • maxFileSizeMB : maximum log file size
            • maxFilesCount : maximum number of log files
          • [uploadLogs:boolean] - uploading service logs to the update server
          • [faultFilesMatch] - regular expression to search for files with crash information when the service fails unexpectedly. The files are included in the crash report.
          • [restartOnFault:boolean] - restart service after fault
          • [restartConditions] - conditions for service restart
            • [maxMemoryMB:number] : the service used more than the specified number of megabytes of memory
            • [maxCpu] - the service used more CPU resources than specified
              • percents:number : percentage of CPU usage
              • durationSec:number : within the specified time interval
            • [makeCore:boolean] : for Unix-like systems, service termination with SIGQUIT signal, core dump is created by this signal
            • checkTimeoutMs:number : interval between check conditions

Macros may be present in the string values of the settings:

  • %%version%% - Developer version
  • %%role%% - Service role. Can be specified in parameters of commands: installCommands, postInstallCommands and runService.

When generating the developer version, the install section from the update.json file is written to the install.json file version image root directory.

Service settings by the developer

Common settings for the service, which are determined by the developer, are placed in the source files. In common settings defined default values, or macros %%name%%. Common settings are recorded to developer version image.

Customizing settings for the client

When generating a client version, the settings from the developer may be defined, supplemented and overridden. To do this, a directory with additional settings is created, with the same structure as the version image directory.

The complement of common settings by optional settings done according to the following rules:

  • If it is required to complete the settings of a file with the extension .conf (typesafe format), .json or .properties, a file is created with the same name and path to this file. This file adds or redefines settings section. The content of the files will be automatically merged.
  • If you need to define macro values %%name%% in the settings file, a file with the same name and extension .defines is created, which contains values for macro definitions in the format name=value.
  • For one configuration file, several additional files can be created with adding an index to the name .. Files are applied sequentially, according to the index. This allows you to change the client configuration from different sources.
  • If you just want to add a settings file, an additional file with a unique name is created.

The install.json file can be completed for the client according to the same rules.

Private files

In addition to source codes, sometimes it is required to include to the image of service version of files with private information, such as access keys. This is often needed for client versions. Such files are not stored in source repositories, but are uploaded to the distribution server directly.

Own Services

Update-services itself consists of services. Thus, update-services creates versions of itself and updates itself.

Own services:

  • scripts - Startup and initializing scripts (Shell, YAML)
  • distribution - Distribution server
  • builder - Version builder (Scala command line application)
  • updater - Services installer and executor (Scala command line application)

Scripts

Perform initial installation and service upgrades.

  • .update.sh - base script, updates the scripts themselves, starts and updates the service
    • distribution - distribution server scripts
      • .create_distribution_service.sh - installs the distribution server as an operating system service, only Linux systemd is currently supported
      • .make_distribution_config.sh - creates an initial configuration file
      • distribution.sh - start distribution server
    • builder - builder scripts
      • builder.sh - run builder
    • updater - updater scripts
      • .updater_setup.sh - installs updater
        • creates updater configuration file
        • installs updater as operation system service, currently only Linux systemd is supported
      • instance.yaml - start script of cloud-init
      • updater.sh - runs updater

Distribution Server

The main component of update-services is the distribution server, managed through the Distribution Dashboard. New versions of services are created on the distribution server. To create a new version, distribution starts builder.

Create distribution server

The distribution server can be built and installed from source, or from another distribution server.

Building and installing the distribution server from source codes

Install data base MongoDB.

Clone source repository update-services to distribution server:

git clone git@github.com:jorkey/update-services.git

Execute the command in the repository directory:

sbt "project builder; run buildProviderDistribution [cloudProvider=?] distribution=? directory=? host=? port=? [sslKeyStoreFile=?] [sslKeyStorePassword=?] title=? mongoDbConnection=? mongoDbName=? [logsMongoDbConnection=?] [logsMongoDbName=?] [persistent=?]"

Replace ? with values:

  • cloudProvider
    • type of cloud service, if the distribution server is in the cloud:
      • Azure - the only currently supported cloud
  • distribution
    • unique distribution server name, no spaces
  • directory
    • installation directory
  • host
    • DNS name for the service
  • port
    • TCP port
  • [sslKeyStoreFile]
    • file with private key and certificate for HTTPS service
  • [sslKeyStorePassword]
    • password to file with private key and certificate for HTTPS service
  • title
    • title of distribution server
  • mongoDbConnection
    • MongoDB connection URL, for example mongodb://localhost:27017
  • mongoDbName
    • MongoDB database name
  • [logsMongoDbConnection]
    • MongoDB connection URL for store log records, for example mongodb://localhost:27017.
    • Log records are stores in the common database if this parameter is not specified.
  • [logsMongoDbName]
    • MongoDB database name for store log records. Must be defined when logsMongoDbConnection is defined.
    • Log records are stores in the common database if this parameter is not specified.
  • persistent
    • currently only for installation on Linux
    • true, if the distribution server is installed as a Linux systemd service

After the command completes, the distribution server process should start.

Installing a distribution server from another distribution server

Create an account for the new customer on the source distribution server (see Service accounts).

Install the MongoDB database.

Copy the file .update.sh to the temporary directory from the directory of the source distribution server. Create the script clone_distribution.sh in the same place:

#!/bin/bash -e
set -e

serviceToRun=builder

distributionUrl=?
accessToken=?

. .update.sh buildConsumerDistribution providerUrl=? [cloudProvider=?] distribution=? directory=? host=? port=? [sslKeyStoreFile=?] [sslKeyStorePassword=?] title=? mongoDbConnection=? mongoDbName=? [logsMongoDbConnection=?] [logsMongoDbName=?] provider=? consumerAccessToken=? testConsumerMatch=? [persistent=?]

Replace ? with values:

  • providerUrl
    • source distribution server URL
  • cloudProvider
    • type of cloud service, if the distribution server is in the cloud
      • Azure - the only currently supported cloud
  • distribution
    • distribution server unique name
  • directory
    • installation directory
  • host
    • DNS name for the service
  • port
    • TCP port
  • [sslKeyStoreFile]
    • file with private key and certificate for HTTPS service
  • [sslKeyStorePassword]
    • password to file with private key and certificate for HTTPS service
  • title
    • title of distribution server
  • mongoDbConnection
    • MongoDB connection URL, for example mongodb://localhost:27017
  • mongoDbName
    • MongoDB database name
  • [logsMongoDbConnection]
    • MongoDB connection URL for store log records, for example mongodb://localhost:27017.
    • Log records are stores in the common database if this parameter is not specified.
  • [logsMongoDbName]
    • MongoDB database name for store log records. Must be defined when logsMongoDbConnection is defined.
    • Log records are stores in the common database if this parameter is not specified.
  • provider
    • source distribution server name
  • consumerAccessToken
    • Access Token for consumer account
      • Go to Settings/Accounts/Distribution Consumers and click on the key icon for the relevant entry.
  • testConsumerMatch
    • test installation distribution server name
  • persistent
    • currently only for installation on Linux
    • true, if the distribution server is installed as a Linux systemd service

Run script clone_distribution.sh. After the script completes, the distribution server process will start.

Distribution server setup

Configuration file

The distribution.json configuration file is located in the working directory of the distribution server and has the following format:

  • distribution : distribution server unique name
  • title : distribution server header
  • instance : virtual machine instance identifier
  • jwtSecret : encoding key JWT
  • mongoDb - setting up a connection to Mongo DB
    • connection : URL подключения
    • name : base name
  • [logsMongoDb] - setting up a connection to Mongo DB for store log records. If not defined, log records are stored in the mongoDb.
    • connection : URL подключения
    • name : base name
  • network
    • port:number : HTTP or HTTPS server port number
    • [ssl] - settings for HTTPS service
      • keyStoreFile : JKS file with keys and certificates
      • keyStorePassword : key store access password
  • versions - version history
    • maxHistorySize:number : maximum number of service versions in history
  • serviceStates - state of services
    • expirationTimeout:FiniteDuration : duration of storage of the last state of the service. If the state is not updated during this time, the service instance is considered dead.
  • logs - logs of services
    • serviceLogExpirationTimeout:FiniteDuration : duration of service log record storage in the database
    • taskLogExpirationTimeout:FiniteDuration : duration of task log record storage in the database
  • faultReports - fault reports
    • expirationTimeout:FiniteDuration : duration of fault report storage in the database
    • maxReportsCount:number : maximum number of fault reports in the database

FiniteDuration format:

  • unit - unit of time: "SECONDS", "HOURS", "MINUTES", "DAYS"
  • length:number - number of units

Functions of distribution server

GraphQL queries

The distribution server accepts GraphQL requests from users, other update-services services, and other distribution servers. GraphQL queries serve:

  • accounts
  • distribution servers connection settings
  • source code repositories
  • profile of services
  • developer versions
  • desired developer versions
  • client versions
  • desired client versions
  • store/find state of services
  • store/find logs of services
  • store/find fault reports

To build versions, the distribution server runs builder locally or on another distribution server.

Logging

Service logs, including logs of distribution itself, are written to a MongoDB collection. Each entry in the collection contains one log entry and consists of the following fields:

  • service - service name
  • instance - instance identifier of the server on which the service is running
  • directory - current process directory
  • process - process identifier
  • [task] - the task identifier, if the entry was received while the task was running
  • time:Date - time and data of log record
  • level - debug level
  • unit - functional unit
  • message - lgo message
  • terminationStatus:Boolean - termination status
    • true - success
    • false - error

The GraphQL schema has a query to search for log entries by different criteria. It is also possible to do a GraphQL subscription on the logs, and then new records in the log will be flow to the GraphQL client in real time.

Tasks

Actions asynchronous to GraphQL queries are framed as tasks. Task information is written to the MongoDB collection in the following format:

Actions asynchronous to GraphQL queries are executed in tasks. Task information is written to a MongoDB collection in the following format:

  • id - task identifier
  • type - task type
  • parameters - set of parameters, in the format:
    • name - parameter name
    • value - parameter value
  • creationTime:Date - time and data when task is created

The execution log and task completion status can be retrieved from the logs.

Services state handling

Updater periodically sends to distribution server status of running services. The state of the services is written to the MongoDB collection, and to the local disk.

State record format:

  • distribution - identified of distribution server
  • instance - instance identifier
  • service - service name
  • directory - service working directory
  • state:ServiceState - service state
    • time - time of state
    • [installTime] - installation time
    • [startTime] - service start time
    • [version] - current version of service
    • [updateToVersion] - target version, if the service is currently being updated
    • [updateError] - update error
    • [failuresCount] - count of service failures after install and run updater
    • [lastExitCode] - last exit code

Handling fault reports

Fault reports are store to MongoDB collection, and to local file storage.

Document format:

  • distribution - identifier of distribution server
  • id - identifier of fault report
  • info - last service state
    • time - time of state
    • instance - identifier of instance
    • service - service name
    • [serviceRole] - service role
    • serviceDirectory - working directory of service
    • serviceState:ServiceState - service state
    • logTail:LogLine[] - last 500 log records
  • files[] - files with fault information

Interaction with other distribution servers

Distribution servers can be networked on a provider/consumer basis.

The provider profile defines the services for the consumer. If the provider has new versions, the consumer can download and install them. Thus, a company developing services can create a consumer account on its distribution server and determine the list of services that it wants to give to it.

The consumer periodically sends information to the provider about the provider's installed services:

  • Installed versions numbers
  • Instances state
  • Fault reports

The provider can also execute builder for the consumer. This is necessary if the consumer server does not have sufficient resources to build the version, or the provider has the necessary set of system libraries.

It is possible to automatically update the services of the consumer when appeared new developer versions at the provider. The consumer periodically checks for updates, and when updates are found, runs an update task.

Update of distribution server

The distribution server is a update-services service, which means that it is built and updated by general rules. The server periodically checks its current version against the client version in the database, and if they differ, restarts. Scripts after the completion of the distribution server install and run new version.

The distribution server also terminates if the client version of the scripts has been updated. In this case, the scripts update themselves and start the distribution server again.

Document history in Mongo DB

All update-services documents are stored in collections of a special format.

Each document, except for the content part, has additional fields:

  • _sequence - sequence number of adding/modifying the document in the collection
    • indexed by ascending and descending
  • _modifyTime - time of adding/modifying the document
  • _archiveTime - document deletion time
    • indexed by ascending with option expire after 7 days

Adding custom fields allows to:

  • Sort documents in collections by the order they were added/modified
  • Store the last modification date of a document
  • Keep a history of deleted documents for 7 days, and restore deleted document if necessary

Builder

Produces:

  • Build/install distribution server
  • Build developer versions
  • Build client versions

Builder install and update

Builder is installed by the distribution server in the /builder/ directory. Here:

  • distributionDir - distribution server working directory
  • distribution - distribution server name

Installation and updating is performed by update-services scripts from the distribution server for which the assembly is performed.

Distribution server install

Has been described in the section (см Create distribution server).

Build developer version

The distribution server executes the command:

builder.sh buildDeveloperVersion service=? version=? sourceRepositories=? [macroValues=?] [buildClientVersion=true/false] comment=?

Here:

  • service - service for which the version is being produced
  • version - number of new version
  • sourceRepositories - source repositories configuration, Json format:
    • []
      • name - repository name, is the name of the directory where the sources are copied
      • git
        • url - Git repository access URL
        • branch - Git repository branch
        • [cloneSubmodules] - clone submodules
        • [subDirectory] - subdirectory inside repository If specified, sources are taken only from this directory.
  • privateFiles[] - list of private files to include in version, array in Json format
  • [macroValues] - macro values in settings in Json format:
    • []
      • name - name
      • value - value
  • comment - comment to the new version

Version build steps:

  • clone or pull repositories are produced in the <builderDir>/developer/services/<service>/source directory, specified in sourceRepositories. Each repository is copied into a subdirectory called name from the configuration.
  • The version is built in the first directory described in sourceRepositories. This directory should contain the update.json file described above. A link to another directory from another repository should look like ../<name>.
  • When building, commands from build/buildCommands are executed. If macros are present in the description of commands, macro substitutions will be made:
    • macro %%version%% is replaced with the build version
    • other values are taken from the values parameter
  • If exitCode is given, its value is compared with the command's return code.
  • If outputMatch is set, the output of the command must match this regular expression.
  • After executing the build commands, the files specified in copyFiles are copied from directory <builderDir>/developer/services/<service>/source to directory <builderDir>/developer/services/<service>/build
  • The install section of the update.json file is written to the install.json file of the build directory.
  • The privateFiles files are downloaded from the distribution server and written to the build directory.
  • Packing the build directory to ZIP file and downloading the version image and version information to the distribution server.
  • Then the source file repositories are tagged with the generated version number. Tag format: <service>-<distribution>-<build>.

Build client version

Distribution server runs command:

builder.sh buildClientVersion service=? developerVersion=? clientVersion=? [settingsRepositories=?] [macroValues=?]

Where:

  • service - service for which the version is being produced
  • developerVersion - source developer version from which the client version is made
  • clientVersion - client version
  • [settingsRepositories] - developer version with which the client version is made:
    • []
      • name - repository name, is the name of the directory where the settings are copied
      • git
        • url - Git repository access URL
        • branch - branch of Git repository
        • [cloneSubmodules] - clone with с submodules
      • [subDirectory] - directory name inside the repository If specified, settings are taken from this directory only
  • privateFiles[] - list of private files to include to the version, array in Json format
  • [macroValues] - macro values in settings in Json format as in buildDeveloperVersion

Build version steps:

  • The version image is downloaded from the distribution server and unpacked to directory <builderDir>/client/services/<service>/build.
  • Produces clone of repositories, specified in settingsRepositories to the directory <builderDir>/client/services/<service>/settings. Each repository is copied into a subdirectory called name from the configuration.
  • Additional settings are merged with developer version settings (see Customing settings for client)
  • The privateFiles files are downloaded from the distribution server and written to the build directory.
  • The build directory is packaged into a ZIP file, which is uploaded to the distribution server along with version information.

Updater

Installs, runs and updates service instances.

Install

Produces by script .updater_setup.sh. Script creates configuration file updater.json and installs updater as service systemd.

.updater_setup.sh <cloudProvider> <name> <services> <distributionUrl> <accessToken> [environment]

Where:

  • cloudProvider
    • Name of cloud service. The only value currently supported is Azure.
  • name
    • Service name part systemd. The full name will be update-${name}.service.
  • services
    • Services to run, separated by commas, can be added to the service name the role it runs as -<role>
  • distributionUrl
    • server distribution URL
  • accessToken
    • Access token for access to distribution server

Configuration file updater.json

  • instanceId
    • VM instance identifier
  • distributionUrl
    • server distribution URL
  • accessToken
    • Access token for access to distribution server

Run updater

updater.sh runServices services=<service1>[-<role>][,...]

Where:

  • services - services for run, as for .updater_setup.sh

Updater execution

For each of the services specified in the arguments, updater does the following.

Service install

The desired client version for this service is downloaded from the distribution server and unpacked to the build directory <service[-role]>/new.

The install.json file is read from the build directory. In the build directory, installCommands commands are executed sequentially with pre-executed macro extensions:

  • role - service role
  • version - developer version

Build directory <service[-role]>/new renamed to work directory <service[-role]>/current.

In the working directory, the postInstallCommands commands are executed sequentially with pre-executed macro expansions:

  • role - service role
  • version - developer version

The installation commands are divided into two groups to reduce the time the service is inactive during an upgrade. The installCommands group contains commands that can be executed from any directory. They are executed until the old version is stopped. The postInstallCommands group contains commands that can only be executed from the working directory.

Start service

In the <service[-role]>/current directory, the runService command is executed with pre-made macro expansions:

  • role - service role
  • version - developer version

Logging

The service should write the operation log to standard output. The log line format should be: date level unit message

Where:

  • date - date in format yyyy-MM-dd HH:mm:ss.SSS
  • level - logging level: TRACE, DEBUG, INFO, WARN или ERROR
  • unit - functional unit
  • message - logging message

The service's standard output and error output are grabbed by updater.

If the writeLogs section is defined in the configuration, the received work log lines are written to the <service[-role]>/current/ directory according to the specified rules.

If the uploadLogs parameter is set to true, log lines are also sent to the distribution server in real time.

Sending the status of services

Once every 10 seconds, updater sends the state of services ServiceState to the distribution server (see Services state handling).

Services failures handling

If the service terminates unexpectedly, a crash report is generated.

The regular expression contained in the faultFilesMatch setting describes files with fault information to be placed in fault report. For example, it could be a core dump file, or Java hprof file.

The fault report is written to the directory <service[-role]>/faults/. The fault report is archived and sent to the distribution server.

If the setting value restartOnFault is not defined, or is defined as true, the service is started again.

Service abort by updater

The restartConditions setting describes the conditions for forcibly terminating the service.

To limit the service by the amount of memory used, the maxMemoryMB parameter is specified. If the service process exceeds the specified limit, the service is forcibly terminated.

To limit the service by CPU usage, the maxCpu parameter is specified. If the service process uses more than the specified percents during durationSec, the service is forcibly terminated.

The makeCore parameter indicates that the service should be terminated with a SIGQUIT signal, to form a core dump.

The interval for checking the occurrence of service interruption conditions is set by the checkTimeoutMs parameter.

Service version update

If the desired version of the service changes on the distribution server, updater:

  • downloads new version
  • executes commands from installCommands
  • stops old version
    • sends stop signal to service process and its descendants
  • service logs are copied to the directory <service[-role]>/log.history/-.log>
  • executes commands from postInstallCommands
  • runs new version

Self-update

If the desired version of updater or scripts changes on the distribution server, updater downloads new versions, they are not updated immediately, but waits until one of the updater's services is updated.

As soon as the desired version of one of the services changes on the distribution server, updater downloads updates and restarts.

Distribution Dashboard

Settings

Open the link to the distribution server in the browser. Enter Account Name: admin and Password: admin. You are logged in as an administrator.

Change the initial password. To do this, go to Settings/Accounts/Users, select the user admin, and click "Change Password".

Accounts

Administered in Settings/Accounts.

Users

Register Dashboard developers and administrators.

Services

Register service accounts. After installing the distribution server are already present builder and updater entries.

By clicking on the key icon, you can get an Access Key for this service. The Access Key is used in service startup and update scripts.

By clicking on the key icon, you can get an Access Key for this service. The Access Key is used in service startup and update scripts.

Distribution Consumers

Accounts of other distribution servers. The account includes, among other things:

  • Distribution server URL. It is used by builder to build the service remotely.
  • Services Profile - a list of developed services that are supplied to this distribution server.

Build

Version build settings.

Developer

Settings for the development version build. Includes:

  • Setting distribution server to run builder
  • Description of parameters for run builder:
    • Сервер дистрибуции для запуска builder
    • Environment
    • Source code repositories, macro values (see Build developer version)
    • Private files
      • Path - the path to the file in the build directory
      • File To Upload - local file to download

Client

Settings for building the client version. Includes:

  • Setting the distribution server to run builder
  • Description of parameters for launching builder:
    • Distribution server to run builder
    • Environment
    • Settings repositories, macro values (see Build client version)
    • Private files
      • Path - the path to the file in the build directory
      • File To Upload - local file to download

Profiles

Service profiles are subsets of the list of development services. Next, the profile is assigned to distribution consumer in the settings.

To refer to services in development for which client versions are also being created on a given distribution server, the self profile is defined.

Providers

This defines the providers of this distribution server. Among other things, for the provider are determined:

  • Access Token - Access key for requests to the server
  • Test Consumer - The name of the consumer, after testing which, services can be installed on this server
  • Upload State Interval - State load interval per provider
  • Auto Update - When you change the desired versions at the provider, they are automatically downloaded and installed on the distribution server. The interval for checking is 10 sec.

Build developer version

After the development services are defined, open Build/Developer. A table of services and their versions will appear:

  • Service - service under development
  • Last Build - last developer version build
  • Client Last Build - last clientr version build
  • Author - author of last developer version build
  • Time - time of last developer version build
  • Comment - comment of last developer version build
  • Status
    • In Process - version is currently being built
    • Completed - build completed

If the service version is not currently being built, you can run the task of creating a new version by selecting an entry. The new version number will be one more than the old one, or it can be assigned manually. If this service is present in the 'own' client services profile, by default you will be prompted to create a client version as well.

Clicking on "Get Lat Commit Comment" will read the change history of the service repositories. The Comment field will display the comment for the last commit in the service repository.

After starting the new version creation task, the task execution log will be displayed in real time. You can interrupt the version creation task.

If a version is currently being created, by selecting an entry, you can view the creation log in real time. You can interrupt the version creation task.

Build client versions

Open Build/Client if development services are defined, otherwise Build.

If client versions are not currently being built, a build start dialog will appear.

With download from the provider

If a provider is selected, a table of that provider's services will appear, in relation to the consumer's profile:

  • Service - service for installation
  • Provider Desired Version - the desired version of the service from the provider
  • Developer Desired Version - desired developer version on the distribution server
  • Last Client Build - last client version build
  • Tested Developer Version - last tested version

There are already selected services for building for which there are no client versions with the desired version from the provider. Other services cannot be selected for assembly.

Without download from the provider

If no provider is selected, the native services described in the 'own' profile are displayed:

  • Service - service for installation to client
  • Developer Desired Version - desired developer version on the distribution server
  • Last Client Build - last client version build

There are already selected services for assembly for which there are no client versions with the desired developer version on the distribution server.

You can also mark for assembly services for which client versions have already been built. This makes sense if the client configuration has changed and you need to rebuild the client version. In this case, a client version will be generated with a new client version build number.

Monitoring the build process

After the build is run, a real-time task execution log will be displayed. If necessary, you can interrupt the build task.

The task execution log is also displayed if, when opening the build client versions dialog assembly is already in progress.

Marking versions as tested

Before installing services on production servers, it is advisable to conduct comprehensive testing. In order for services to be installed on working servers only after they are tested, update-services has a mechanism to mark the list of service versions as tested.

For example, there is a distribution server named production, in the configuration of which the test distribution server test is specified. Installation of new versions of services on the production server will occur only if they have been tested on test and marked as tested by test.

Desired versions

In this page, you can edit the lists of desired developer and desired client versions. For example, you can roll back an update, or several recent updates. This is possible because the distribution server stores the last few installed versions and history changes to desired versions. Editing is done in one of two ways.

Rollback to a previous revision of a list

To move to an older revision, press the 'down' button, to move to a newer revision, press the 'up' button. The list revision time and author are listed at the top. The entries in the table display the desired versions of the selected revision compared to the latest revision:

  • unmodified records are displayed in standard font
  • bold - services for which the desired version has changed
  • red - in this revision there is no service that is in the latest revision
  • green - in this edition there is a service that is not in the latest edition

To make the list of desired versions current, click Save.

Assigning versions manually

You can also arbitrarily assign versions to services. To do this, click on the service version, after which a list of available versions in the history will appear. When the version is changed, the record with the service is highlighted in bold.

To make the edited list of desired versions the current one, click Save.

Monitoring logs

You can view logs in the context of a task or service execution.

Monitoring task logs

The task table is displayed in reverse chronological order:

  • Creation Time - task creation time
  • ID - task identifier
  • Type - task type
  • Parameters - task parameters with format: name:value
  • Active - task is currently running

When a task is selected, entries in the task log are displayed:

  • Time - logging time
  • Level - log level
  • Line - record

You can select logs by:

  • Level - log level
    • when the level is selected, records of this level and above are displayed
  • Find Text - sample text

When Follow is selected, the end of the log is displayed, new log entries are displayed in real time.

Monitoring service logs

The logs of the selected service are searched with an optional selection by:

  • Instance - instance identifier
  • Directory - work directory of service
  • Process - service process number
  • Level - log level
    • when the level is selected, records of this level and above are displayed
  • From/To - time range
  • Find Text - sample text

When Follow is selected, the end of the log is displayed, new log entries are displayed in real time.

Fault reports

The distribution server receives reports about failures of its services, and the services of its consumers. By default, all reports are displayed in descending chronological order. You can narrow search by the following criteria:

  • Distribution - distribution server
  • Service - service name
  • From/To - time range

When you select a report, information about the failure and the latest entries of the service log are displayed.

It is possible to download the ZIP archive of fault report to a file.

Dashboard

The Dashboard page is intended for operational control of update-services. With its, you can see how update-services lives, quickly find possible problems, rollback installed versions.

Panels are used to display information from different angles.

Panel Builds

Designed to monitor active and history of builds.

A table with fields is displayed:

  • Service - service name
  • Version - target build version
  • Author - build author
  • Time - build time
  • Comment - comment to build version
  • Target - build target
    • DeveloperVersion
    • ClientVersion
  • Status
    • InProcess
    • Success
    • Failure

Panel Versions

Designed to monitor the distribution and operation of service versions.

A table with fields is displayed:

  • Service - service name
  • Developer Desired Version
  • Client Desired Version
  • Working Version - client version currently running
  • Directory - service working directory
  • Instances - instances identifiers list
  • Info - service state

Thus, this table entry displays the full distribution path of the service version from developer version to the version running on the instance and its state.

If the distribution path is completed completely, all three versions will be the same. Otherwise, the first version that does not match in the path is displayed in red. When Only Alerts is selected, only services with an incomplete distribution path are displayed.

When a consumer is selected, the versions installed on that consumer are displayed.

About

Cloud-native GitOps system without K8s

https://vupdate.org

cicddevopsdevops-toolsgitops

License:Apache License 2.0

Languages

Language:Scala 54.4%Language:TypeScript 42.1%Language:JavaScript 1.3%Language:Java 1.2%Language:Shell 1.0%Language:HTML 0.1%Language:SCSS 0.0%