Digital Citizenship
This is the main repository of the Digital Citizenship project, managed by AgID and the Digital Transformation Team.
This repository stores technical documentation and code, for a more friendly introduction on the topic, check out the following pages (Italian):
- The Digital Citizenship project page in the Digital Transformation Team site for high level introduction.
- The Digital Citizenship documentation site for more indepth explanation of the project components and goals.
Child projects
The Digital Citizenship APIs
The Digital Citizenship mobile app
- backlog: app and app backend
- code: app and app backend
Contributing
Architecture decision records
In a world of evolutionary architecture, it's important to record certain design decisions for the benefit of future team members as well as for external oversight. Architecture Decision Records is a technique for capturing important architectural decisions along with their context and consequences. We store these details in source control, along with code, as then they can provide a record that remains in sync with the code itself.
We use ADRs to track architectural decisions of this initiative.
This repository is configured for Nat Pryce's adr-tools.
Here's the decisions we taken so far:
Azure infrastructure
The infrastructure drectory contains scripts and Terraform configuration to deploy the infrastructure on the Azure cloud.
Deploy Environments
For each of the three different deployment environments
(test, staging and production) a directory in
infrastructure/env contains the relative configuration.
The configuration consists in a JSON file with the name of the Azure services that need to be provisioned (ie. web applications, databases) in a specific resource group (one for each environment).
The automated setup tools (discussed below) take the value
from the ENVIRONMENT environment variable, that must be set
at the beginning of the whole procedure.
ie:
ENVIRONMENT=staging
Prerequisites
- An active Azure subscription
- Git
- NodeJS >= 0.6.x
- Terraform >= 0.10.x
- Yarn >= 1.0.x
- NPM packages: run
yarn install
All binaries must be in the system path.
Set up an Azure Active Directory B2C tenant
Step 1 - Add an Azure Active Directory B2C resource
To authenticate Digital Citizenship API users (through the developer portal) we use an Active Directory B2C (ADB2C) tenant.
As it's actually not possible to create (and manage) an ADB2C tenant programmatically, it must be manually created. See the get started guide:
https://docs.microsoft.com/en-us/azure/active-directory-b2c/active-directory-b2c-get-started
The ADB2C tenant must exists before running any task illustrated below.
Once created, before going on with the installation procedure,
put the tenant name in ADB2C_TENANT_ID environment variable:
ADB2C_TENANT_ID=<yourtenant>.onmicrosoft.com
Step 2 - Add custom users attributes in ADB2C
During user sign-in to the Digital Citizenship API we collect some custom attributes relative to the user account.
Go to the ADB2C blade in the Azure portal, then select "User attributes" and add the following custom attributes (type is always "String"):
- Organization
- Department
- Service
Step 3 - Add and configure an ADB2C Sign-in / Sign-up policy
For users to be able to sign-in and sign-up through ADB2C you need to create a Sign-in / Sign-up Policy.
Go the Azure ADB2C blade in the Azure portal -> "Sign-up or Sign-in policies" -> "Add".
Policy Name:
- SignUpIn
Identity Providers:
- Email signup
Sign-up attributes:
- Department
- Display Name
- Given Name
- Organization
- Service
- Surname
Application claims:
- Department
- Display Name
- Given Name
- Organization
- Service
- Surname
- Email Addresses
- Identity Provider
- User is new
- User's Object ID
Multifactor authentication:
- On
Page UI customization:
- Set up for every page the following custom page URI:
https://teamdigitale.github.io/digital-citizenship-onboarding/unified.html - Save the policy
- Customize the "Multifactor authentication page"
- Open "Local account sign-up page"
- Mark all fields as required (optional = no)
- Reorder fields and rename labels at wish
Step 4 - Add the Developer Portal Applications in the Azure ADB2C tenant
Finally, you need to register (create) the Developer Portal ADB2C Applications:
instructions on how to create an ADB2C application
- Register an ADB2C Application
dev-portal-app
Set the return URL of this application to:
https://${apim}.portal.azure-api.net/signin-aad
(replace ${APIM} with the value of config.azurerm_apim in your tfvars.json file)
Generate an application key, then set the two environment variables:
DEV_PORTAL_CLIENT_ID=<Application Id>
DEV_PORTAL_CLIENT_SECRET=<Application Key>
- Register an ADB2C Application
dev-portal-ext
Set the return URL of this application to:
https://${PORTAL}.azurewebsites.net/auth/openid/return
(replace ${PORTAL} with the value of config.azurerm_app_service_portal in your tfvars.json file)
Generate an application key, then set the two environment variables:
DEV_PORTAL_EXT_CLIENT_ID=<Application Id>
DEV_PORTAL_EXT_CLIENT_SECRET=<Application Key>
Deploy instructions
- Create an Active Directory Principal
and get
CLIENT_ID,CLIENT_SECRET,TENANT_ID(the Active Directory Id) and AzureSUBSCRIPTION_ID.
Set up the following environment variables to let the automated setup tools authenticate against the Azure subscription:
ARM_SUBSCRIPTION_ID=<subscription Id>
ARM_CLIENT_ID=<service principal client (app) Id>
ARM_CLIENT_SECRET=<service principal client secret (key)>
ARM_TENANT_ID=<Active Directory domain Id>
- Check your enviroment configuration then run:
yarn infrastructure:deploy
Running the above command will deploy the following services to an Azure resource group:
- App service plan
- Functions app (configured)
- CosmosDB database (and collections)
- Storage account
- Storage queues (for emails and messages)
- Blob storage
- API management (with configuration)
- Application insights
- Log analytics
- EventHub
- Web App service
Component diagram
Tasks
Some services aren't yet supported by Terraform (ie. CosmosDB database and collections, Functions, API management).
These ones are created by NodeJS scripts (infrastructure/tasks) that provision the services through the
Azure Resource Manager APIs and are supposed to be run from
command line using the relative npm (or yarn) script:
Finishing the installation
Some tasks cannot be carried out programmatically and require a manual intervention by the Azure subscription administrator through the Azure portal interface.
Activate "Managed Service Identity" for the onboarding Web App Service
To ease the onboarding of new developers (API users) we use a dedicated Application that starts some automated tasks once the user sign-in into the developer portal.
This web application exposes an HTTP endpoint that triggers some actions on the authenticated user's account (ie. create a subscription to use the Digital Citizenship API). These actions are triggered when the logged-in user clicks on a call-to-action button that redirects her browser to the exposed endpoint.
To give the needed permissions (manage API management users account) to the onboarding Web App we use Managed Service Identity. In this way we can manage developer portal users directly from the Web application without hardcoding any client credential into the App Service settings.
To activate Managed Service Identity and assign the needed role to the App Service:
-
Navigate to the Azure Portal App Service blade (for the Web App Service ${config.azurerm_app_service_portal}) -> Managed Service Identity -> Register with Azure Active Directory -> set the value to 'On'.
-
Navigate to the Azure Portal API management blade -> Access Control (IAM) -> Add the registered Web application as a "Contributor".
-
Restart the Web App Service.
Shared Terraform state
The Terraform state is shared through an Azure storage container.
Before running any command involving Terraform you must request access to the Azure container to the project administrators (or use your own for testing purposes when deploying to a staging resource group).
Example environment configuration
Make sure you have the following environment variables set up before launching any npm task to provision Azure resources.
To do that, place a .env file in the root of this directory
that contains the following configuration variables (all mandatory):
# May be `test`, `staging` or `production`
ENVIRONMENT=staging
# Tenant name of the Active Directory B2C
ADB2C_TENANT_ID=XXXXXXXXX.onmicrosoft.com
# Azure service principal credentials (main AD tenant)
ARM_SUBSCRIPTION_ID=XXXXX-XXXX-XXXX-XXXX-dXXXXXXXXX
ARM_CLIENT_ID=XXXXXXX-XXXX-XXXX-XXX-XXXXXXXXX
ARM_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXX=
ARM_TENANT_ID=XXXXXXX-XXXXX-XXXX-XXXX-XXXXXXXXXXX
# Client credentials for dev-portal-app ADB2C App
DEV_PORTAL_CLIENT_ID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX
DEV_PORTAL_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXX
# Client credentials for dev-portal-ext ADB2C App
DEV_PORTAL_EXT_CLIENT_ID=XXXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX
DEV_PORTAL_EXT_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXX
# Sync API products and policies
INCLUDE_API_PRODUCTS=1
INCLUDE_API_POLICIES=1
# Mail service API key
SENDGRID_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXExample output
$ yarn infrastructure:deploy
> digital-citizenship@0.1.0 resources:deploy ...digital-citizenship
> npm-run-all resources:tf-init resources:tf-apply resources:cosmosdb resources:functions resources:api
> digital-citizenship@0.1.0 resources:tf-init ...digital-citizenship
> terraform init -var-file=infrastructure/tfvars.json infrastructure
Initializing provider plugins...
...
Terraform has been successfully initialized!
...
> digital-citizenship@0.1.0 resources:tf-apply ...digital-citizenship
> terraform apply -var-file=infrastructure/tfvars.json infrastructure
...
Apply complete! Resources: 9 added, 0 changed, 0 destroyed.
> digital-citizenship@0.1.0 resources:cosmosdb ...digital-citizenship
> ts-node infrastructure/tasks/00-cosmosdb.ts
successfully deployed cosmsodb database and collections
> digital-citizenship@0.1.0 resources:functions ...digital-citizenship
> ts-node infrastructure/tasks/10-functions.ts
...
Deploy new releases
When you are ready to deploy a new release, you need to synch the source code in the Git repository to the App Service (or to Azure Functions).
Azure gives you the option to configure continuos deployment from a GitHub branch,
automatically triggering a new deploy through a webhook when changes are pushed.
To do that you must give your Azure subscription access to your
GitHub account in order to set up the webhook:
https://docs.microsoft.com/en-us/azure/azure-functions/functions-continuous-deployment
We chose to not setup this kind of continous deployment, but to provide a script that, when launched from the command line, will synch the code from the GitHub repository to Azure services.
To deploy new code to the developer portal web application run:
yarn deploy:devapp:sync
To deploy new code to Azure Functions run:
yarn deploy:functions:sync
Other
Building the documentation site
The source of the Digital Citizenship documentation site is under the docs
directory of this repository.
To build the sphinx documentation from this repository:
yarn docs:build
To deploy the documentation site (via GitHub pages):
yarn docs:publish