Objective

  • Setup git repository in VSTS
  • Build Continuous Integration (CI)/Continuous Deployment (CD) pipeline from scratch using VSTS
  • Enable CI
  • Create free web app service on Azure
  • Trigger CD after successful CI
  • End-to-end flow execution

Abstract

Continuous Integration (CI)/Continuous Deployment (CD) are widely used terms in the DevOps world. If you are new or don’t fully understand the buzzword DevOps then read my article DevOps, What, Why & How. Continuous Integration (CI) and Continuous Delivery(CD) enables and empowers any software delivery team in an organization to continuously deliver value to their end users.

Prerequisites

The application which you want to setup CI/CD pipeline for, that project code must be available in a code repository in VSTS (Visual Studio Team Services) read my article Beginning with git using VSTS can be helpful to set up a git repo from scratch using VSTS. Also, an Azure portal account is a must.

Creating a Build Definition

After successful Project and code setup in the git, you will see that your VSTS account will show code repository and a project like as shown below. Here, repository named TestProjects has an ASP.NET MVC application named WebApplication1.

Click on “Set up build” and begin with selecting a template “ASP.NET” as shown in the image below, click Apply. You may want to choose a template which is suitable for your application.

Setting the build tasks

If not already open then open the Tasks tab. By default, build definition name will be <ProjectName>-<BuildTemplateName>-CI (you can change this to something else if you wish to) and Agent queue will be empty, which needs to be set to Hosted (in case of .NET) or Hosted 2017 (in case of .NET Core)  as shown under Process blade in image below.

By default, build is being set up at the master branch on the project you initiated “set up build” from, which was TestProjetcs. You can verify this and change settings if needed using the “Get sources” blade as shown in the image below.

Build Template comes pre-configured with most of the tasks and hence you can review other tasks, but no changed would be required for successful execution from this step onwards (verified for MVC application), however, in your case, you may have to tweak some settings.

Queuing a CI build

To produce deployable artifacts, you need to submit a build of your code. To do so, click on Save & queue or if build definition is saved already then click on Queue which will ask for confirmation to kick-off a build as shown in the image below.

Click on Queue to initiate the process and you will notice that your screen will show that a Build #yyyymmdd.n have been queued.

To track the status of a build you can click on the build number link (as shown in the image above) and if everything was setup correctly then you will see “Build succeeded” as shown in the image below that all the build steps executed successfully.  To explore more about each build step, you can click on a step link shown in the left-pane of the window.

Exploring the Build Artifacts

CI build produces deployable artifacts and those can be seen via clicking on “Artifacts” link shown above Build details.

Setting up the Continuous Integration (CI)

Continuous Integration (CI) is when the code is being built on each check-in and that can be deployed to an environment. To set up this process, navigate to the build definition (click on Edit) and open the “Triggers” tab. By default, continuous integration is Disabled as shown in the image below.

Click on “Enable continuous integration” to enable the CI for TestProjetcs and it will include master branch as the default setting.

Click “Save” to keep the changes to the build definition.  Do not choose “Save and queue”

Continuous Integration (CI) in Action

Ensure that VSTS is open and you have Builds page open. Now switch to the Visual Studio and update the code of WebApplication1 under TestProjects repository

Click on “Commit All” to check-in the changes and push the code to master. Soon after that under the Builds in VSTS, you will notice that another build is kicked off and shows “in progress” status which was triggered by the commit “Title update in About.cshtml”.

After successful build process, you will see “succeeded” status

Setting up the Azure Web App

Before creating a Release definition, an environment needs to be set up for deployment. An azure web app is a perfect choice to host the MVC application.

Login to the azure portal and create a web app “WebApplication1-dev” for dev environment and so other flavors (qa, uat, staging, and production etc.) can be created as needed.

 

By default, the Web app is created as an S1 (small) pricing tier, but you can create your own service plan and select Free pricing tier. Refer to my article Free Web App Hosting on Azure

After successful Web app creation, it will be available at https://webapplication1-dev.azurewebsites.net/

Setting up the Continuous Delivery (CD)

Continuous Delivery (CD) is a software engineering approach in which teams produce deployable software which can be reliably released to an environment (dev, qa, uat, staging, and production) at any time.

To set up CD click on Build and Releases, then select Releases. Click on “New definition”.

A “New Release Definition” will be created with the template for release as shown in the image below. Change the name of the release to “TestProjects-ASP.NET-CD”, then click on Artifacts and select TestProjects-ASP.NET-CI.  Because, CI build artifacts will be deployed using Release dominion as CD process.

Next, it’s time to setup the Environment, rename it to dev and set values under “Tasks” tab as appropriate.

Creating a Release

After a Release definition has been set up, it’s time to deploy the CI build using the created release, this can be easily achieved by clicking on “Create release” link on the top-right corner.

“Create release” will summarize the setup of showing dev environment and created CI builds to choose from. By default, CI build will automatically kick off the release to the dev environment.

Select the latest build from the list of successful build versions and then click “Create” and notice that “Release-1” has been created.

Click on the link “Release-1” to check the status which will be shown as “IN PROGRESS”

After few seconds Deployment status will be updated to “SUCCEEDED”

Now, if you access the  https://webapplication1-dev.azurewebsites.net.WebApplication1-dev.azurewebsites.net then you will see that web application has been successfully deployed to Azure web app.

Setting up end-to-end automated CI/CD pipeline

In the Release definition click on Artifacts trigger icon (lightning symbol) and enable the continuous deployment trigger and select the branch.

Now, go to Visual Studio and make some changes to the About.cshtml and check-in the code to master branch of WebApplication1 project. Which will then kickoff a CI Build.

Successful CI build will then kick-off the Release for CD to the dev environment.

Upon successful execution of Release, the website will be deployed as shown below (see an updated message on the about.cshtml page).

This point onwards, whenever any code changes will be made using Visual Studio, then a CI build will spin and upon successful build execution, a Release (CD) will be created to deploy the web app.

Summary

CI/CD is the most widely used t the rm in industry today and helps in delivering continuous value delivery to the end users. VSTS and Azure offers a lot of great and powerful features to build CI/CD pipeline. This article has shown the process of building CI/CD pipeline from scratch and how to enable various triggers for CI/CD automation during this process.

I spoke at Silicon Valley Code Camp 2017 in San Jose, CA on Oct 7th 2017 and my topic was Need of DevOps in the Enterprise.

• Need of DevOps in the Enterprise https://lnkd.in/eQf3dFm

It was nice experience to speak in Silicon Valley and meet college students to very well experienced professionals. I was really honored to have Ron Lichty attend my session and at the end mentioning that it was a great session.

Microsoft MVP team was also there to sponsor and socialize about all the MVP Speakers at Silicon Valley Code Camp (SVCC) 2017.

Slide of my session can be found at my Github

Overall it was nice experience, I also met some Microsoft Evangelists and fellow C# Corner and Microsoft MVPs.

Introduction 

DevOps is a new buzzword and it promotes continuous value delivery to the end users. To know more about DevOps read my article DevOps: What, why and how? 

DevOps is a “culture”, where development, test, and operations work together in a collaborative manner to automate delivery of quality software.  DevOps culture develops “production-first mindset”. I.e. applying DevOps ensures that your code is always ready to be deployed to production. 

Agile Manifesto 

Agile manifesto and principles have an influence on DevOps. Let’s review agile manifesto: 

Manifesto for agile software development  

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:  

  • Individuals and interactions over processes and tools 
  • Working software over comprehensive documentation 
  • Customer collaboration over contract negotiation 
  • Responding to change over following a plan 

Agile Principles 

Overall Agile Manifesto has twelve principles and four of those really gels well with DevOps. 

  • Our highest priority is to satisfy the customer through an early and continuous delivery of valuable software. 
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 
  • Business people and developers must work together daily throughout the project. 
  • Working software is the primary measure of progress. 

How Agile principles apply on DevOps  

  • Our highest priority is to satisfy the customer through an early and continuous delivery of valuable software. 

  When a team adapts the DevOps culture then team works   together to deliver a high-quality software to the end users consistently over and over.  

  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.DevOps promotes short delivery cycles over long delivery cycles. Short delivery cycles have small amount of change which is far easier to deploy and test and detect impact of changes.
     
  • Business people and developers must work together daily throughout the project.DevOps enable the business people to have look at the product early and provide candid feedback. DevOps also provides an easy and effective way of adding approval step in the CI/CD pipeline where business / product owner can approve changes for deployment to other environments.
     
  • Working software is the primary measure of progress.At the end business values working software having reliable feature delivered on time to end users. DevOps practices enable the software development teams to deliver working software on time and provided proper channel for feedback between various stages of the DevOps workflow.
     

DevOps, What , Why & How

June 7th, 2017 | Posted by Vidya Vrat in ALM / DevOps - (1 Comments)

What DevOps is not?

DevOps is not a “Packaged Solution”. I.e.  You cannot buy and install it.

What is DevOps?

“DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.” – Donovan Brown, MSFT

DevOps is a “culture”, where development, test, and operations work together in a collaborative manner to automate delivery of quality software.  DevOps culture develops “production-first mindset”. I.e. applying DevOps ensures that your code is always ready to be deployed to production.

Why DevOps?

First thing DevOps offers is that, it breaks the “Wall of Confusion” between development and operations team. These two teams have different agenda and expectations when it comes to software delivery and deployment.

DevOps breaks the wall of confusion between teams and fosters better communication and collaboration throughout the application development lifecycle and results into better visibility and small frequent deployments together.

DevOps Benefits and ROI – Source, Forrester Research Inc.

  • Improved IT operations efficiency. Saving 50% of allocated IT operations support.
  • Improved developer productivity with faster, automated release processes. Developers saw a 15% improvement in productivity.
  • Improved tester productivity with faster, automated release processes. 20% productivity gain with their deployment of the Microsoft DevOps solution.
  • Faster recovery from failures and reduction in release risk. Reduce time-to-resolution per incident by 2 hours.
  • Reduced cycle time by as much as 99% and more frequent deployments. Increased customer satisfaction and also gained new business.
  • Faster speed-to-market for new features, products, and services for customers. 20x more, leading to increased sales.
  • Improved release quality. Successful releases with a reduction in errors increase in release reliability, and less time on remediation of release errors.

How to apply DevOps?

Prime objective of DevOps is to “quickly ship the highest quality software to the end customers”. To make this happen DevOps needs to be implemented across all the phases of software development and delivery.

DevOps workflow from planning to release

The workflow model below shows how teams go through four phases and contribute towards DevOps adoption.

Plan Backlog represents well-defined and prioritized user stories with proper acceptance criteria etc.

Develop & Test Development will include good quality code written, debugged, code reviewed, checked-in including unit test coverage as appropriate. Testing will include verifying the user story’s acceptance criteria, possibly followed by performance and integration tests etc.

Release – Whenever a new version is ready and a sprint end, an automated process is used for deployment.

Monitor and Learn After release team can gather information related to how customers use the application /services and continue to monitor health of application.

Collaboration – Building code and running unit tests on each check-in small or big ensures that code is ready to be deployed. Then, deploying it to at-least production like environment and successfully performing tests there ensures that Dev and Production are in collaboration.

DevOps Automation

One major key aspect of DevOps is automation of software build, test, and deploy. DevOps automation enables continuous value delivery.  Let’s see the DevOps Automation process workflow.

Development – This is where a developer writes code in their local development environment.

Version Control – This is version control team uses to check-in code to the repository. E.g. Git, TFS, SVN etc.

Build and Unit Testing – This is where DevOps Continuous Integration (CI) takes place. At this step, code will build with latest checked-in code and latest packages and dependencies as applicable. If there are unit tests then all unit tests will be executed to ensure that there are no collateral damages caused within the code base.

If Build fails then a notification will be sent to the developer who submitted the build. Many teams have various policies and channels set up for this. For example, a bug is created with details when a build fails and team’s slack channel will receive notification when a build fails.

Automated Acceptance Testing – If build succeeds then a set of acceptance tests can be executed automatically to verify that code is working fine.

Any failure in an acceptance test will trigger a feedback initiated to the team and process will again start from beginning.

User Acceptance Test – Passed automated acceptance tests ensures and triggers the code to be promoted to UAT.

Release – Upon passing UAT, code can be released to any environment or production or production like environment. Manual push of code to production is known as Continuous Delivery (CD).

DevOps Toolchain

DevOps is technology agnostic and any development environment on any platform can fully adopt DevOps culture and can continuously deliver quality software to their customers. Here some SDLC phases and tools which will fit in and can help to setup end-to-end DevOps pipeline for your technology and platform of choice.

Planning and Analysis

  • Capturing and tracking (TFS, VSTS, JIRA, ServiceNow.
  • Documentation or Wiki page (Microsoft Teams, SharePoint, Confluence.
  • Collaboration (Slack, HipChat, Microsoft Teams).

Design and Development

  • SCM (TFS, VSTS, Subversion, Git, Mercurial.
  • IDE (Eclipse, IntelliJ, Visual Studio).

Build and Release (CI/CD)

  • Repository management (Artifactory, Nuget, Nexus).
  • Build tools (MSBuild, Jenkins, Bamboo).
  • Configuration management (Chef, Puppet, Ansible).
  • Cloud (AWS, Azure,OpenStack).
  • Containers (Docker).

Integration and Testing

  • Source code verification (SonarQube).
  • Security testing (HP Fortify).
  • Functional testing (MSTest, NUnit, JUnit, Cucumber, Selenium).
  • Performance testing (SOASTA, Apache Test Bench, Microsoft Load and Performance Test).

Beginning with Git using VSTS

February 9th, 2017 | Posted by Vidya Vrat in ALM / DevOps | VSTS - (0 Comments)

Introduction

Just like TFVC (Team Foundation Version Control), Git is another Source Version Control and it’s becoming very popular. VSTS (Visual Studio Team Services) supports both TFVC and Git. Let’s assume that you have no project or Repository and wants to begin with Git on VSTS.

Create a New Project

Login to your VSTS account and click on New Project

Fill the information

Click “Create” and you will notice that Project will be Created as shown below.

Clone it in IDE

Click “Clone in Visual Studio”. New Visual Studio IDE will start a dialog like shown below will popup.

Provide the Repository path and click on “Clone”.

Switch to Team explorer and Click on “Create a new project or solution”

When prompted choose the project type. After Project is added successfully you will notice that project is added in Solution Explorer and shows (+) in front of various items and also Team Explorer’s Changes tab shows all the items listed as newly being added.

Commit Changes and Publish

After project being added into Solution Explorer it’s considered as a change to the blank repository/project you created. Hence, this change needs to be added to remote Master branch.

In Team Explorer à Changes, add a comment for check-in and click on “Commit All”

Click “Commit All” and from the next Synchronization Tab, click “Publish”

Now your Git Repository in VSTS should be updated with the code you have pushed-in to it. Refresh your VSTS page and you should be able to see your project code added.

How to integrate slack with VSTS

October 10th, 2016 | Posted by Vidya Vrat in ALM / DevOps | VSTS - (0 Comments)

Abstract

All software development teams use some sort of team communication tool. Like many  team messaging tools  Slack is one of them. What I like most about slack is its ability to gel with VSTS (Visual Studio Team Services) and send notifications when an event is triggered for example, a Pull Request is created, or build succeeded or failed etc.

Note: VSTS was previously known as VSO. I.e. VSO is renamed to VSTS.

Introducing Slack

Slack is a widely used messaging application (web / desktop) it is widely used by many mission critical projects including Mars Curiosity Rover robots. In terms of software development, slack is very handy and efficient to notify when a VSTS event takes place. This feature reduces explicit team communication with peers that a pull requested is created and someone needs to look at that etc.

Let’s preview the final outcome

Objective is to  empower software development team(s) by enabling them to collaborate not only with peers but also with tools and services (VSTS in this case) and get immediate notification for any status updates instead of emailing or pinging someone in person for instance “Hi, Vidya Vrat. I have submitted a pull request.”

Now this article will show you Step-by-Step procedure to achieve “Slack integration with VSTS” and receive message as sown in the figure 0 below.

Figure 0 – Slack channel showing message received from VSTS

Create a Team on Slack

First step is to create a team (if not already) on https://slack.com click on “Create a new team” and enter a valid email-id which you have access to as shown in figure 1 below.

Figure 1- Slack.com main page to create a new team

When you click on “Create New Team” then you will be taken to a confirmation page as shown in figure 2 below.

Figure 2 – Page to enter confirmation code

Check your email which you have entered while creating a team and you shall see an email with Slack confirmation code as shown in figure 3 below.

Figure 3- Slack Confirmation code email received

After successful validation of confirmation code as asked in Figure-2, next step will be to enter your name and username etc. as shown in Figure 4 below.

Figure 4- Personal details

Next step will be to enter team information as shown in figure 5 below.

Figure 5 – Team information

Next Enter your company information as asked in figure 6 below.

Figure 6 – Company name

Next step will be to enter your company domain as asked in figure 7 below. It is OK to enter a domain name which is not registered or you don’t have plans to register.

Figure 7 – Company web domain info

Figure 8 – Team invitation

If you don’t want to invite people now, then click Skip and you will be taken to slack team for DotNetPassion as shown in the figure 9 below.

Figure 9 – DotNetPassion slack team page

Channel Creation on your Slack Team

Once your team is created, next step is to create a channel, which people will join and exchange messages.

Click on the + as shown in the figure 10 below.

Figure 10 – Creating channel under team

Choose channel name and purpose etc. as shown in figure 11 below.

Figure 11 – Enter channel details

Click on “Create Channel” will create the channel named “pull_requests” under your team “DotNetPassion” as shown in the figure 12 below.

Figure 12 – Successful channel creation

Integration with VSTS

Click “Add an app or custom integration link on your slack channel page as shown in the figure 12 above. Then select Developer Tools, and then select VSTS as shown in figure 13 below.

Figure 13 – Adding an app integration

Selecting Visual Studio Team Services will take you to next steps to Install as shown in figure 14 below.

Figure 14 – VSTS Integration page

Click on Install to begin the integration procedure.

Figure 15 – Selecting channel name.

After clicking on “Add Visual Studio Integration” you will see step-by-step Instructions to configure your VSTS for clack integration as shown in figure 16 below.

Figure 16 – VSTS setup instructions

Getting the Web Hook URL

Figure 17 – Wikipedia Webhook definition

Scroll down on this page to see Integration Settings and get the “Webhook URL” as shown in Figure 18 below. This Webhook url is most important piece of information to complete VSTS and Slack integration. Either keep this page open or copy and paste in notepad etc.

Figure 18 – Webhook url

Now open a new Tab in your browser and login to your VSTS account and navigate to your code Repository as shown in figure 19 below.

Figure 19 – Code repository in VSTS

Add VSTS Service Hook

From your VSTS code repository’s setting page, click on Service Hooks as shown in figure 20 below.

Figure 20 – Adding a VSTS Service Hook

Click on + to add a Service hook and choose Slack as shown in figure 21 below.

Figure 21 – Using Slack for Service Hook Subscription

Next you need to select the Trigger and repository settings etc. as shown in figure 22 below.

Figure 22 – Selecting Trigger settings

Next, confirm Action and webhook URL as shown in figure 23 below.

Figure 23 – Confirming Action with Webhook url

Next, click on “Test” to verify and test the VSTS integration as shown in figure 24 and 25 below.

Figure 24– Test integration with VSTS

Figure 25 – Test message from VSTS delivered in slack channel

Now, switch back to the VSTS Integration page and click Finish as shown in figure 26 below.

Figure 26 – Slack Integration Finish page

After Finish you will be taken to VSTS page and there you can see that service hook for slack is added as shown in figure 27 below.

Figure 27 – Service hook for slack added in VSTS

Let’s Submit a Pull Request

In Visual Studio’s Team Explorer connect to a VSTS Git Repository as shown in figure 28 below.

Figure 28 – Connecting with VSTS

Open the solution and make some changes as shown in figure 29 below.

Figure 29 – File changes in Visual Studio

After you  Submit a new Pull Request, then slack will receive a notification as shown in the figure 30 below.

Figure 30 – Slack received VSTS Pull Request message

To navigate to Pull Request you can click on the pull request link in the message (pull request 3) and it will take you to the VSTS as shown in figure 31 below. If all looks good, then you can take next action steps(s).

Figure 31 – Navigation to VSTS page from Slack message.

 

I am speaking at Seattle Code Camp and my session topic is “How to use Agile correctly“. It’s totally FREE community event. Please register and spend the weekend in learning all you can.

My session will be in Room#204 at 1-2 pm.