todo-azurewebsites

This repository contains the code used to show how to manipulate deployment to Azure Websites and WebJobs. Topics covered include:

Performing gulp based builds for stand-alone client side components.
Integrating client side components to a WebAPI backend. SignalR used to notify clients of changes.
Adding a WebJob to peform background tasks and notify WebAPI of changes.

Learning how to customize deployment using Kudu is the primary focus exercise.

What you will need

An active subscription to Azure
Visual Studio 2013 (possibly 2012, that is just untested)
Azure SDK 2.2
Node.js
Azure Cross-Platform tools
A text editor that understands opening a directory in the file system

Getting Started

Fork this repository in github to your github account.

Use that fork button in the upper right. It is really easy; Don't fear the fork

Clone the fork to a local github repository

git clone git@github.com:MY_GITHUB_USERNAME/todo-azurewebsites.git

##Running the stand-alone client side web application locally

Install the build tools
```
cd src/web
npm install
```
Make sure gulp is installed globally
```
npm install -g gulp
```
```
Run the build
```
gulp run
```
If a browser did not open automatically open a browser to localhost:3000
After you have experimented with the functionality in the browser exit the gulp process with Ctrl-C

##Deploying the web side

Make sure your current working directory is the repository root

Create the WebSite on Azure

azure site create YOUR_TODO_SITENAME --git --gitusername USERNAME_FOR_AZURE_DEPLOYMENT

Create a sample deployment script for a node project
```
azure site deploymentscript --node -o nodescript
```
This will create a deploy.cmd in the nodescript folder that we will use as a template
Open the deploy.cmd file in the nodescripts folder
Copy its contents to the clipboard
Open the deploy.cmd at the repository root
Paste the clipboard contents
Close the deploy.cmd from the nodescripts folder to avoid confusion

Editing Kudu script for gulp build

Locate the :Deployment section in the script
Note the sections for
1. KuduSync
2. SelectNodeVersion
3. Install npm packages

Move the 1. KuduSync block below the 3. Install npm packages block

:Deployment
echo Handling node.js deployment.

:: 2. Select node version
call :SelectNodeVersion

:: 3. Install npm packages
IF EXIST "%DEPLOYMENT_TARGET%\package.json" (
    pushd "%DEPLOYMENT_TARGET%"
    call :ExecuteCmd !NPM_CMD! install --production
    IF !ERRORLEVEL! NEQ 0 goto error
    popd
)

:: 1. KuduSync
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
    call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_SOURCE%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
    IF !ERRORLEVEL! NEQ 0 goto error
)

Change the :: comment blocks to echo to help with diagnostic output

:: 2. Select node version
:: 3. Install npm packages
:: 1. KuduSync

should become

echo 1. Select node version
echo 2. Install npm packages
echo 3. KuduSync

wrap the Install npm packages block in a directory change
remove directory prefix on package.json check
remove inner pushd popd

remove --production from npm install

pushd src\web
echo 3. Install npm packages
IF EXIST "package.json" (
  call :ExecuteCmd !NPM_CMD! install
  IF !ERRORLEVEL! NEQ 0 goto error
)
popd

This mirrors the npm install you did locally

Add a block to execute gulp locally this goes after the Install npm packages block
```
)

echo "Execute Gulp"
IF EXIST "Gulpfile.js" (
  call .\node_modules\.bin\gulp
  IF !ERRORLEVEL! NEQ 0 goto error
)

popd
```
The equivalent to this locally would be running gulp from the command line. The difference being that you do not have permissions to install globally (npm install -g) on Azure. So, you need to run the local copy.
Whew, that is quite a few changes. Looks like a good time to commit changes

Kudu sync gulp build output to Azure Websites

add \src\web\dist to the -f (from) parameter on the call to Kudu sync

echo 5. KuduSync
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_SOURCE%\src\web\dist" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
  IF !ERRORLEVEL! NEQ 0 goto error
)

Try your deployment changes locally

From the repository root run deploy.cmd
```
.\deploy.cmd
```
.\deploy.cmd is the powershell version deploy works at the command prompt
Verify the gulp build
1. Navigate up a directory from the repository root
2. You should see an artfacts folder
3. Verify the presence of the wwwroot folder and an index.html file inside it
  
  There will also be some other folders in here. This is just a spot check

Deploy to Azure

Return the working directory to the repository root
Commit your changes
```
git add .
git ci -m "add gulp build to azure deployment"
git push azure master
```
This initial commit will be time consuming. All of the npm packages as well as bower packages need to be installed for the first time. This is a process similar to NuGet package restore. Subsequent deployments will take significantly less time. It is also worth noting that if you are running on the free teir of Azure Websites, there are cpu limits. Depending on the complexity of your build process you may hit them.
Visit the website

Introduce WebAPI project

Open the TodoSample.sln file in Visual Studio
Rebuild Solution to pull in the NuGet packages

Configure for camelCase JSON serialization

Open the App_Start\WebApiConfig.cs file

remove comments from code setting up the JSON serializer settings

// Web API configuration and services
var settings = GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings;
settings.ContractResolver = new CamelCasePropertyNamesContractResolver();

Determine port number used by IISExpress for Api
1. Right click on TodoSample.Api project
2. Select Properties
3. Select the Web tab on the left
4. Note Project Url as localhost:31008
Configure client app to use Api instead of local service
1. Remove the "TodoService" from web\app\js\app.coffee (lines 5-35)
2. Remove comments from the Api "asTodoApi" and "TodoService" (lines 7-47 after delete in previous step)
Add reference to signalr/hubs script
1. Open web\app\pages\index.html
2. Remove comments from script tag including signalr/hubs
```
<script type="text/javascript" src="/js/vendor.js"></script>
<script type="text/javascript" src="/signalr/hubs"></script>
<script type="text/javascript" src="/js/app.js"></script>   
```
The order of these scrips is important. The signalR base libaries must be included before the hubs. The hubs must be included before the client code.
Create Database
1. Build
2. Set TodoSample.Api as the startup project
3. Open the Package Manager Console (Tools -> NuGet Package Manager -> Package Manager Console)
4. In Package Manager Console, set default project to TodoSample.Data
5. Run Migrations from the PM> prompt
```
Update-Database
```
Test Client and WebAPI combined locally
1. Start debugging with TodoSample.Api as the startup project
2. \api\todos to the url localhost:3108/api/todos 3. You should see an empty array
3. Ensure the current directory is \src\web in the powershell window
4. Start the web project with a proxy to IIS
```
gulp run --proxy --proxyPort 31008
```
Add a few items to make sure it works
Stop the gulp server with Ctrl-C at the console
Stop debugging in Visual Studio
This seems like a good place to commit changes

Integrate WebAPI project into deployment script

Ensure current working directory is the repository root

Create a scaffolding script for the WebAPI project

azure site deploymentscript --aspWAP .\src\TodoSample.Api\TodoSample.Api.csproj -s .\src\TodoSample.sln -o apiscript

Select no at the prompt to overwrite the .deployment file

Open the deploy.cmd in the apiscript directory

Copy the items after the definition of Kudu Sync to the clipboard (lines 50-62)

IF NOT DEFINED DEPLOYMENT_TEMP (
  SET DEPLOYMENT_TEMP=%temp%\___deployTemp%random%
  SET CLEAN_LOCAL_DEPLOYMENT_TEMP=true
)

IF DEFINED CLEAN_LOCAL_DEPLOYMENT_TEMP (
  IF EXIST "%DEPLOYMENT_TEMP%" rd /s /q "%DEPLOYMENT_TEMP%"
  mkdir "%DEPLOYMENT_TEMP%"
)

IF NOT DEFINED MSBUILD_PATH (
  SET MSBUILD_PATH=%WINDIR%\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
)

Paste them in the deploy.cmd in the repository root after the definition of KuduSync and before goto Deployment

  :: Locally just running "kuduSync" would also work
  SET KUDU_SYNC_CMD=%appdata%\npm\kuduSync.cmd
)

IF NOT DEFINED DEPLOYMENT_TEMP (
  SET DEPLOYMENT_TEMP=%temp%\___deployTemp%random%
  SET CLEAN_LOCAL_DEPLOYMENT_TEMP=true
)

IF DEFINED CLEAN_LOCAL_DEPLOYMENT_TEMP (
  IF EXIST "%DEPLOYMENT_TEMP%" rd /s /q "%DEPLOYMENT_TEMP%"
  mkdir "%DEPLOYMENT_TEMP%"
)

IF NOT DEFINED MSBUILD_PATH (
  SET MSBUILD_PATH=%WINDIR%\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
)

goto Deployment

Note the Deployment commands in the :: Deployment area
1. Restore NuGet packages will pull referenced packages from NuGet before the build
2. Build to the temporary path with compile the project
3. KuduSync will copy the compiled output to the target folder
Copy all 3 sections to the clipboard (lines 68-89)

Paste this in the deploy.cmd file in the repository root after :Deployment but before echo Handling

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Deployment
:: ----------

:Deployment

echo Handling .NET Web Application deployment.

:: 1. Restore NuGet packages
IF /I "src\TodoSample.sln" NEQ "" (
  call :ExecuteCmd "%NUGET_EXE%" restore "%DEPLOYMENT_SOURCE%\src\TodoSample.sln"
  IF !ERRORLEVEL! NEQ 0 goto error
)

:: 2. Build to the temporary path
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\src\TodoSample.Api\TodoSample.Api.csproj" /nologo /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder /p:_PackageTempDir="%DEPLOYMENT_TEMP%";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release /p:SolutionDir="%DEPLOYMENT_SOURCE%\src\\" %SCM_BUILD_ARGS%
) ELSE (
  call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\src\TodoSample.Api\TodoSample.Api.csproj" /nologo /verbosity:m /t:Build /p:AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release /p:SolutionDir="%DEPLOYMENT_SOURCE%\src\\" %SCM_BUILD_ARGS%
)

IF !ERRORLEVEL! NEQ 0 goto error

:: 3. KuduSync
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_TEMP%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
  IF !ERRORLEVEL! NEQ 0 goto error
)

echo Handling node.js deployment.

Update the commented numbers to echo and renumber for clean output

:: 1. Restore NuGet packages
:: 2. Build to the temporary path
:: 3. KuduSync
echo 2. Select node version

change these lines to

echo 1. Restore NuGet packages
echo 2. Build to the temporary path
echo 3. KuduSync
echo 4. Select node version

Yes there are more below select node version, you should update those too

Make sure NuGet is available for local build

This is an awful hack. The NUGET_EXE environment variable is not set up when running localy. We need to find the NuGet executeable and set the environment variable NUGET_EXE to point at it. Mine happens to be installed by Chocolatey, yours may be elseware. Just ensure that the version is 2.8 or greater
1. Above goto Deployment after the definition of MSBUILD_PATH, add a line that ensures NUGET_EXE is available
```
IF NOT DEFINED MSBUILD_PATH (
  SET MSBUILD_PATH=%WINDIR%\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
)

IF NOT DEFINED NUGET_EXE (
  SET NUGET_EXE=c:\Chocolatey\lib\NuGet.CommandLine.2.8.0\tools\nuget.exe
)

goto Deployment
```

Azure Deployment

Create a SQL Database with a name of todosample on Azure and obtain its connection string

Note that this shoud be in the same region as your WebSite

In Package Manager Console, run update-database, specifying the Azure connection string

update-database -ConnectionString "<string>" -ConnectionProviderName "System.Data.SqlClient"

Visit the CONFIGURE tab of the Azure Website in the management portal
Add a connection string entry for todosdb and set its value to the connection string from step 1 of Azure Deployment

What is awesome here is that this allows for your web.config to remain safe. It will only ever point at a local database. You do not have to expose your production secrets anywhere but on the portal (or configuration script if you prefer). This setting will override the value in the web.config at runtime.
commit your changes and push to Azure

Update Api to send notifications to storage queue

Create a storage account in the management portal
Copy the management key to the clipboard
Update connection strings from Emulator connection string to live connection string

This requirement should go away with a future release of the storage emulator. As of this writing the storage emulator is behind what is actually deployed in Azure. The team working on the WebJobs SDK is using some of these features not yet available in the emulator. This is a known issue and will likely be addressed soon(ish).
1. Update the web.config in the TodoSample.Api project
```
<add name="storage" connectionString="DefaultEndpointsProtocol=https;AccountName=[accountname];AccountKey=[accesskey]"/>
```
2. Update the app.config in the TodoSample.Processor project
```
<add name="AzureJobsRuntime" connectionString="DefaultEndpointsProtocol=https;AccountName=[accountname];AccountKey=[accesskey]"/>
<add name="AzureJobsData" connectionString="DefaultEndpointsProtocol=https;AccountName=[accountname];AccountKey=[accesskey]"/>
```
You may want to create two storage accounts one for local development and one for deployment. The connection strings will be overriden in the portal in a later step much like we did with the database connection string earlier.
Open the NotificationController in the TodoSample.Api project

Remove the comments from the AddChangeNotification helper method

private void AddChangeNotification()
{
    var queue = new EventQueue();
    queue.AddNotification();
}

Testing locally

Set TodoSample.Api as the startup project
Start without debugging Ctrl-F5
Set TodoSample.Processor as the startup project
Start Debugging F5

You may only attach the debugger in Visual Studio to a single process. In this case we are choosing to attach to the TodoSample.Processor because it is the project we have not run yet. You could also start up the Processor from the command line as it is simply a console application. The choice is yours.
Start up the local proxy
```
gulp run --proxy --proxyPort 31008
```
Validate new task items being created and in the console window the jobs being picked up from the queue.
Shut down the proxy and stop debugging in Visual Studio. You may also want to kill off the instance of IISExpress that was running the Api.

Including WebJob in the deployment script

Ensure current working directory is the repository root

Create a scaffolding script for the WebJobs project

azure site deploymentscript --dotNetConsole .\src\TodoSample.Processor\TodoSample.Processor.csproj -s .\src\TodoSample.sln -o webjobscript

Open the deploy.cmd in the webjobscript directory
Inspect the Deployment section
1. Note that the restore nuget is the same as what we have. Package restore is done at the solution level
2. Note that the output path is different than our current msbuild process. This one builds to a sub-folder. That indicates to us that this msbuild needs to follow our existing one.
3. Note the presence of the WEB_JOB_DEPLOY_CMD. At first glance, it would seem like something we need. What this command actually does is copy a HTML file into the website if only a WebJob is deployed. You can think if it as being similar to the hostingstart.html only this page will notify the visitor that the WebSite is running a WebJob
4. The KuduSync is the same as our current sync for msbuild
5. Shiny, not a lot to do this time.

Copy the msbuild step to the clipboard (lines 76-78)

:: 2. Build to the temporary path
call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\src\TodoSample.Processor\TodoSample.Processor.csproj" /nologo /verbosity:m /t:Build /p:Configuration=Release;OutputPath="%DEPLOYMENT_TEMP%\app_data\jobs\continuous\deployedJob" /p:SolutionDir="%DEPLOYMENT_SOURCE%\src\\" %SCM_BUILD_ARGS%
IF !ERRORLEVEL! NEQ 0 goto error

Open the deploy.cmd in the repository root
Paste this msbuild step after step 2 and before step 3

Change the comments to echo and renumber for diagnostics. (If you are lazy like me you can use 2a instead of renumbering)

  call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\src\TodoSample.Api\TodoSample.Api.csproj" /nologo /verbosity:m /t:Build /p:AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release /p:SolutionDir="%DEPLOYMENT_SOURCE%\src\\" %SCM_BUILD_ARGS%
)

IF !ERRORLEVEL! NEQ 0 goto error

echo 2a. Build to the temporary path
call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\src\TodoSample.Processor\TodoSample.Processor.csproj" /nologo /verbosity:m /t:Build /p:Configuration=Release;OutputPath="%DEPLOYMENT_TEMP%\app_data\jobs\continuous\deployedJob" /p:SolutionDir="%DEPLOYMENT_SOURCE%\src\\" %SCM_BUILD_ARGS%
IF !ERRORLEVEL! NEQ 0 goto error

echo 3. KuduSync

Perform a local deployment

dennisburton / todo-azurewebsites