How to protect your web site using WAF-enabled Azure Application Gateway

Azure Application Gateway a Layer-7 HTTP load balancer that provides application-level routing and load balancing services. It distributes traffic requests based upon data found in application layer protocols such as HTTP/HTTPS and also on application specific data such as HTTP headers, cookies, or data within the application message itself, such as the value of a specific parameter.

You basically need to define rules to accept the traffic requests and route them to the appropriate back-end instances.

Application Gateway currently supports the following features:

    • Web Application Firewall (WAF)
    • Scaleble, highly-available HTTP load balancing solution
    • Cookie-based session affinity
    • SSL offload for better utilization
    • URL-based content routing
    • Multi-site routing
    • Web socket support
    • Health monitoring
    • Advance diagnostics

While Azure is responsible for securing the infrastructure and platform that your application runs on, it is your responsibility to secure your application itself. Now Web Application Firewall (WAF) in Azure Application Gateway can provide protection to your web applications against common threats such as SQL injection, cross-site scripting attacks, and session hijacks.

If your organization hosts highly sensitive information, the number-one priority is having a fully-isolated and dedicated environment for only your organization’s applications. Using an App Service Environment, your organization can have security and isolation for your web apps and use a virtual network for control over traffic.

An App Service Environment is a premium service plan option of Azure App Service that provides a fully isolated and dedicated environment. App Service Environments are isolated to run only a single customer’s applications and are always deployed into an Azure Virtual Network. At a high level, an App Service Environment consists of compute resources running in the Azure Hosted Service, Storage, Database, a Virtual Network, and a subnet with the hosted service running in it.

From a single open port, one option to block most traffic would be to use WAF in Application gateway in front of ASE to protect your Web apps.You can also Create a network security group, and assign it to a subnet in your Azure Virtual Network to restrict traffic to the App Service Environment from the WAF only by using the VIP address.

Architecture Overview

Here you have all the security with a straight forward architecture. Easy to provision, maintain and administer.

The path for request would be: App Gateway (WAF mode) –> ASE

image.png

To create this architecture here are the steps involved:

  • Create a virtual network (ex: frontend-vnet) for both App Service Environment (ASE) and Application Gateway(AG).
  • Create subnet for Application Gateway. Subnet for App Service Environment will be created as a part of ASE provision process.
  • Creates an App Service Environment in your virtual network with a private internal load balancer address using Azure Quickstart Template.  This step would take up to 2 hours to complete.
  • Deploy a test web app – The vnet (frontend-vnet) is not publicly accessible so in order to deploy app, you need to create a Virtual Machine that is living within the same Virtual Network and use that to deploy and access the Web App with its internal IP. Once you have deployed your test web app, you should successfully be able to  access it from any VM which is living within same vnet (frontend-vnet).
  • Create WAF-enable Application Gateway
  • Configure Application Gateway
  • Test your web app form public endpoint.

In this blog post I will go through the creation and configuration of Application Gateway in detail.

 

Create WAF-enabled Application Gateway

In Azure Portal, Go to New—>Networking and select Application Gateway. Provide the information for the basic setting as shown below. Make sure you select WAF tier.

image.png

In the settings, make sure to select the same Virtual Network (frontend-vnet) you used to configure ASE earlier and the subnet you created specifically for the Application Gateway. You also need configure the public IP address.

Configure the WAF specific settings.

  • Firewall status – This setting turns WAF on or off.
  • Firewall mode – This setting determines the actions WAF takes on malicious traffic. If Detection is chosen, traffic is only logged. If Prevention is chosen, traffic is logged and stopped with a 403 Unauthorized.

image.png

Review the results and click on OK to create the gateway.

Configure the Application Gateway

Add servers to backend pool – Once the application gateway is created, go to the Backend Pools and select the current backend pool.

image.png

Add the IP address of ILB ASE and Save. Now the incoming traffic that enters the application gateway would be routed to the backend address added here.

Configure SSL offload – Application gateway can be configured to terminate the Secure Sockets Layer (SSL) session at the gateway to avoid costly task of decrypting HTTPS traffic off your web servers. Application gateway decrypts the request and sends it to backend server and re-encrypts the response before sending it back to the client.

To configure SSL offload with an application gateway, a certificate (pfx format) is required. This certificate is loaded on the application gateway and used to encrypt and decrypt the traffic sent via SSL.

Add an HTTPS listener – It will look for traffic based on its configuration and helps route the traffic to the backend pools. Click Listeners and click the Add button to add a listener. Fill out the required information for the listener and upload the .pfx certificate.

image.png

Create a rule and associate it to the listener – Once listener is created, you need to create a rule to handle the traffic from the listener. Click the Rules of the application gateway, and then click Add. Type in the friendly name for the rule and choose the listener created in the previous step. Choose the appropriate backend pool and http setting and click OK.

image.png

Create the custom probe – Custom probes allow you to have a more granular control over the health monitoring. When using custom probes, you can configure the probe interval, the URL and path to test, and how many failed responses to accept before marking the back-end pool instance as unhealthy.

Probes are configured in a two-step process through the portal. The first step is to create the probe. Next you add the probe to the backend http settings of the application gateway. Create a Custom Probe with the Host set as your custom Web App domain, for example sample-app.com as shown below.

image.png

Add probe to the gateway – Go to the HTTP settings, and make sure that the setting has Custom Probes turned on and select the probe you just created. Otherwise, the Application Gateway will try to go to the IP of the App Service Environment without passing a Host header, which won’t work and will throw the probe into an Unhealthy state resulting in the 502 Gateway Proxy error.

image.png

Testing

There are couple of ways to do the testing. First you can use ModHeader Chrome extension to open the public IP address/hostname of the Application Gateway in the browser. You need to pass in the Custom Domain you configured on the Web App as a Host Header and the website should come up. Refer Sabbour blog post for further detail.

The other way is to add hostname (sample-app.com) to Custom Domains in the setting of app deployed in ASE as shown below.

image.png

You need to add an entry for your host in Hosts file on your local machine. The path would be c:\Windows\System32\Drivers\etc\hosts.

Now if you go to https://sample-app.com it should open up the sample web app as shown below.

image.png

Logging and troubleshooting

Application Gateway provides following capabilities to monitor resources.

Backend health – Application gateway provides the capability to monitor the health of individual members of the backend pools through the portal, PowerShell, and CLI.

image.png

Logging – There are different types of logs in Azure to manage and troubleshoot application gateways such as performance, firewall and access logs.

image.png

Here is a sample firewall log.

image.png

There are three different options to choose for storing your logs

  • Storage Account
  • Event Hubs
  • Log Analytics

image.png

Metrics – Application gateway currently has one metric. This metric measures the throughput of the application gateway in Bytes per second.

image.png

You can also set alert rule for application gateway based on metrics on a resource.

For example, an alert can email an administrator if the throughput of the application gateway is above, below or at a threshold for a specified period of time.

image.png

 

Summary

To summarize, we explored the option to protect your web applications against common threats such as SQL injection, cross-site scripting attacks, and session hijacks using Azure Application Gateway. We ‘ve hosted a Web App securely in an App Service Environment. This Web App isn’t publicly accessible as it is sitting in a subnet inside a Virtual Network and it isn’t exposed to the internet. The only way to access the site is through a Web Application Firewall enabled Application Gateway.

Microsoft Tech Summits 2017 and Global Integration Boot Camp

Microsoft Tech Summits kicked off today at Chicago with lots of keynotes, technical training sessions and hands-on labs to build and develop cloud skills of interested individuals.

There were also deep dive sessions covering a range of topics across Microsoft Azure and the hybrid platform including security, networking, data, storage, identity, mobile, cloud infrastructure, management, DevOps, app platform, productivity, collaboration and more.

image.png

The Microsoft Tech Summit provides Free, two-day technical training for IT professionals and developers with experts who build the cloud services across Microsoft Azure, Office 365, and Windows 10.

Here is the Agenda look like.

image.png

You can also find a city near you and Register for the event

Here’s a list of the currently published Tech Summit events around the globe:

  • Amsterdam,  March 23 – 24
  • Bangalore,  March 16 – 17
  • Birmingham, March 27 – 28
  • Chicago, January 19 – 20
  • Copenhagen, March 30 – 31
  • Frankfurt, February 9 – 10
  • Johannesburg, February 6 – 7
  • Milan, March 20 – 21
  • Seoul, April 27 – 28
  • Singapore, March 13 – 14
  • Washington D.C. March 6 – 7

.

Global Integration Bootcamp

There is another free event coming up for integration community – Global integration boot camp.

image.png

This event is driven by user groups and communities around the world, backed by Microsoft, for anyone who wants to learn more about Microsoft’s integration story. In this full-day boot camp there will deep-dive into Microsoft’s integration stack with hands-on sessions and labs, delivered to you by the experts and community leaders.

In this Boot Camp, the main focus will be on:

BizTalk 2016 –BizTalk Server 2016, what’s new, and using the new Logic Apps adapter
Logic Apps –Creating Logic Apps using commonly-used connectors
Servicebus –Build reliable and scalable cloud messaging and hybrid integration solutions
Enterprise Integration Pack –Using the Enterprise Integration Pack (EIP) with Logic Apps
API Management –How does API management help you organize your APIs and how does it increase security?
On-Premise Gateway –Connecting to on-prem resources using the On-Premise Gateway
Hybrid Integration –Hybrid integrations using BizTalk Server and Logic Apps
Microsoft Flow – Learn to compose flows with Microsoft Flow

image.png

If you are interested to be part of it or to host it on your location, you can reach out to organizers by providing your details.

Organizers

Azure Archive Storage

This week during our discovery session with one of our new customer and Azure Black Belt Team members we came to know about a new storage type called Archive Storage which is still in development phase.

It is also mentioned in the Microsoft Cloud Platform roadmap documentation which provides a snapshot of what Microsoft is working on in their Cloud Platform business. You can use the document to find out which cloud services are

  • recently made generally available
  • released into public preview
  • are still developing and testing
  • or are no longer developing

image.png

As per documentation,

Azure Archive Storage is a very low cost cloud storage for data that is archived and very rarely accessed with retrieval time in hours.  It can be useful for archive data such as medical reports, compliance documents, exchange mails, etc. that are accessed rarely but need to be stored for many years.

Currently Azure Storage offers two storage tiers for Blob – Hot and cool storage.

Azure hot storage tier – is optimized for storing data that is accessed frequently.

Azure cool storage tier – is optimized for storing data that is infrequently accessed and long-lived.

image.png

Azure Archive Storage is different than Hot and Cold because it’s in a way offline data storage. And that’s the reason if the Archive Storage data need to be accessed then it will be made online with retrieval time in hours. It will also be cheaper than Cold storage almost half price.

With Archive Storage there will be an option to apply a policy to move the data from hot or cold tier to Archive. For example, you can have an automated process based on a policy to move any data which is one-year-old to archive storage.

I hope archive storage type would provide an option to enterprise customers to store archival data in most cost-effective way in Azure.

Logic App to detect sentiment and extract key phrases from your text

Microsoft’s Cognitive services provides set of powerful intelligence APIs. These APIs can be integrated into your app on the platform of your choice to tap into ever-growing collection of powerful artificial intelligence algorithms for vision, speech, language, knowledge and search.

Integrating Cognitive Services into an application provides the app with the ability to SEE, RECOGNIZE, HEAR and even understand the SENTIMENT of your text.

In this blog post, I am trying to experiment with the Text Analytics API in Logic App. The API is a suite of text analytics services built with Azure Machine Learning to evaluate sentiment and topics of text to understand what user want.

For Sentiment analysis the API returns a numeric score between 0 and 1. Scores close to 1 indicate positive sentiment and scores close to 0 indicate negative sentiment. For Key phrase extraction the API returns a list of strings denoting the key talking points in the input text.

Cognitive Service account for the Text Analytics APIs

To build to Logic App to use Text Analytics APIs, first you need to sign up of the text analytic services.

  • Login to Azure Portal with your valid MSDN Subscription and Search for Cognitive Services APIs.

image.png

  • Create a Cognitive Service account by providing the details as shown below.

image.png

  • Make sure Text Analytics is selected as the ‘API type’ and select free plan – free tier for 5,000 transactions/month
  • Complete the other fields and create your account.
  • After you sign up for Text Analytics, find your API Key. Copy the primary key, as you will need it Logic App.

image.png

 

Logic App to detect sentiment and extract key phases

Logic Apps is a cloud-based service that you can use to create workflows that run in cloud. It provides a way to connect your applications, data and SaaS using rich set of connectors. If you are new to Logic App, please refer the Azure documentation for further details.

Now let’s create a Logic App to detect sentiment and extract key phrases from user’s text using the Text Analytic API.

Go to New >Enterprise Integration and select Logic App as shown below.

image.png

Create a Logic App by providing the details as shown below.

image.png

After our deployment success, we can start editing our Logic App.

To access it, in your left, browse All Resources > [Name of your Logic App].

Clicking in your Logic App will open the Logic Apps Designer. In welcome screen, there are a lot of templates ready to use. Choose a blank template from Logic Apps Designer

image.png

On Logic App designer, a search box is available where you can look for available Microsoft managed connectors and APIs available. Select the Request from the list which would act as a trigger to your Logic App and can receive incoming request.

image.png

Now we need to define a request body JSON Schema and the designer will generate tokens to parse and pass data from the trigger through the workflow.

We can use a tool like jsonschema.net to generate a JSON schema from a sample body payload

image.png

JSON schema for the above payload looks like below

image.png

Now use this JSON schema in the Request trigger body as shown below

image.png

Next step is to look for Cognitive Service API connector in the managed API list.

image.png

Select the Detect Sentiment  and provide a connection name and Cognitive Service Account Key which we have copied in the previous section and click on Create.

image.png

Now you need to provide the Text value to the Detect Sentiment API from “text” variable of the Request trigger as shown below.

image.png

Next step would be to add Cognitive Service connector for Key Phrases same way we did for Detect Sentiment.

image.png

Now we would use the Compose and Response action to send HTTP response for the sentiment and key phrase analysis.

image.png

This is how I have composed the response using a simple new JSON message using the variable “key phrase” and “score” from the Key Phrase and Detect Sentiment APIs.

image.png

You can also use code view to compose the response message as shown below.

image.png

And finally use the output of compose action to send the HTTP response.

image.png

So here is complete workflow look like

image.png

Quick and easy! Now once you save the workflow the topmost Request trigger will have the URL for this particular Logic App.

Now let’s invoke this Logic App from one of my favorite API testing tool, Postman.

I submitted the sample JSON message to the endpoint with following text – I had a wonderful experience! Azure cognitive services are amazing.

Sure enough, I got the key phrase and sentiment score as below:

image.png

image.png

Conclusion

Clearly Microsoft’s Cognitive Services are easy to use in your app on the platform of your choice. The Text Analytics API is just one of many different Artificial Intelligence APIs provide by Microsoft. I am sure this new platform would mature in the coming days and different types of app can leverage this technology.

 

References:

https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-text-analytics-quick-start

https://social.technet.microsoft.com/wiki/contents/articles/36074.logic-apps-with-azure-cognitive-service.aspx

Is it a Bird? Is it a Plane? No, it’s a Book #robustintegration

Few months back, I approached Abhishek  – 2 times MVP, for a quick interview and published it on my blog.  After that blog post Abhishek reached out to me and expressed his interest in writing a technical book about Modern Integration Solution using Azure. In fact he had already submitted his idea to PACKT publisher. He asked me if I would be interested in co-authoring the book. I found it as a great career opportunity and joined him immediately.

After more than a month of email communication with publisher PACKT about the book outline, they gave us green signal and also came up with the title of the book as – “Robust Cloud Integration with Azure”. They asked us to submit the detailed outline of the book in a week or so.

The book primarily talks about Microsoft Cloud Integration Technologies and how Azure App Services, PaaS platform can help you connect your applications, data sources and APIs in the cloud and on-premises.

Target audience for the book can be anyone who is interested and curious about developing apps for Cloud Integration Platform in Azure. Whether you work for a small start-up or for a large enterprise, this book can help you understand Microsoft Cloud Integration technologies to integrate application and business processes. By using this book, readers will be able to support their apps (web, mobile and enterprise) that connect to data anywhere – be it in the cloud or on-premises.

While preparing the detailed outline I and Abhishek reached out few people in integration community for the review of our detailed outline. We received lots of interested feedback and also surprisingly few people showed their interest in co-authoring the book. The idea of writing this book is to produce high-quality content which can be useful to its reader. I also thought if I pick any book and found it useful I really don’t care if it was written by one author or multiple. So it should really does not matter how many people teamed up to write the book as long as it solves its core purpose. So we went ahead and expanded our team with following great professional.

Mahindra Morar, @mmorarnzMahindra has a great expertise in Integration Space. In last few years he primarily focused on integrating systems as a principal integration consultant in Datacom, New Zealand. He also co-authored the book SOA Patterns with BizTalk Server 2013 (2nd edition) last year.

image.png

His areas of interest include exploring new technologies and deciding how to use them in the world of integration.You can view his blog at https://connectedcircuits.wordpress.com

 

James Corbould, @jamescorbould

image.pngJames also works in Datacom, New Zealand focusing on Azure and BizTalk Server and the surrounding Microsoft technology stack. He has designed and built integration solutions for a number of leading and well known organizations in New Zealand, across a broad range of industry sectors.

He is also a technical reviewer of eBook BizTalk Server Extensibility. Active in the community, James maintains a blog at https://jamescorbould.wordpress.com

 

Ashish Bhambhani, @ashbham

image.pngAshish is Integration Sr. Premier Field Engineer with Microsoft living in the Los Angeles, United State region. He has been working in the integration space for more than a decade. In his current role he helps Microsoft’s enterprise customers with architecting, designing, building and maintaining their integration solutions. He has worked with some of the world’s biggest customers for Microsoft in the integration space.

I and Ashish were in the same team when I was a PFE in Microsoft, India.

We have already started first couple of chapters and would be finishing and submitting them for review to PACKT. We are looking for the reviewer for our book. We are talking to few great experts in integration community including someone from Microsoft Logic Apps product team.

image.png

We are keeping our fingers crossed to finish this book with high-quality content to be released by end of this year.

Note: If this seems to be exciting to you and you are interested to be a reviewer for our book, feel free to reach us at #robustintegration.

Logic App Templates

Logic App Templates  are a set of curated pre-built Logic Apps to help you quickly get started building your own integration application.

You can find these templates in the Azure Marketplace under the Web + Mobile category and then you can search for “Logic Templates” which will show you a list of all of the logic app templates.

image.png

The intent of these logic app templates are to help you better understand patterns that can be used in logic apps. This would be really helpful for people coming for BizTalk background to understand which all BizTalk integration scenarios / patterns can be achieved using Logic Apps.

image.png

These templates show how to use some of the many connectors available in the marketplace as well as being a good way to discover various patterns that can be built using Logic Apps. You can either use these as is or modify and them to fit your scenario.

image.png

The logic app templates are intended to be used by anyone who is planning on building their own logic apps. They range from simple scenarios using your everyday consumer SaaS and productivity services such as Facebook, Twitter and Office365 to complex scenarios doing Hybrid Enterprise Integration including connecting to SAP, SQL, using messaging protocols, message transformations and EDI. They can be used to either to discover different integration patterns and learn how they are done or as a starter template that you can just modify and start using.

There have been lot of session in last couple of months delivered by product group to help people to under the concepts of Logic Apps. Last week product group also started a community webcast for Azure Logic Apps, hosted by the Logic Apps PM team.

image.png

Resource:

http://azure.microsoft.com/blog/2015/06/25/getting-started-with-logic-app-templates/

https://gautambiztalkblog.com/2015/06/18/exploring-and-evaluating-azure-logic-apps/

https://gautambiztalkblog.com/2015/06/28/logic-app-concepts/

Logic App Concepts

Logic Apps is the part of the new Azure App Service which is a fully managed PaaS (Platform as a Service) for developers that makes it easier to build web, mobile and integration apps.

image.png

Logic Apps provides a new way to automating business process and running them in reliable way in cloud. Anyone who can use Azure should be able to start building long running business process that orchestrate data and services across cloud and on premise data centre. It is a browser based workflow engine that makes integrating disparate data sources, from cloud to on-premises easy.

image.png

So Logic Apps are basically a foundation of connectivity which provides very slick orchestration engine that allows you to build integration solutions using different pieces available in Azure App Service.

image.png

 

 

The following are some of the key pieces that comprise the Logic Apps experience.

Workflow

Logic Apps provides a graphical way to model your business processes as a series of steps or a workflow. It’s a visual designer where you can bring in logics from many different places and put them into a common surface to create your business process work flow.

You can check this demo by Josh showing how you can create a simple work flow to get a notification message on you mobile if a new customer is getting added in your Salesforce application.

image.png

If you come from BizTalk background you can relate this to orchestration where you can use different shapes to create a business process workflow.

Orchestration is much more powerful and have lot more features. Product team has announced that they are working on to bring most of those features in Logic Apps.

image.png

 

Connectors

Your logic apps need access to data and services. A connector is a special type of API app. It is created specifically to aid you when you are connecting to and working with your data. See the list of available connectors as of now.

image.png

    • Social Connectors: Facebook, Yammer, Twitter, Chatter, Twilio
    • Enterprise Connectors: Salesforce, SAP, Marketo, QuickBooks, SugarCRM
    • App + Data Services: Azure Media Services, Azure Mobile Services, Azure Service Bus, Azure Storage Blog, Azure Storage Table, Azure WebJobs, Box, Dropbox, HDInsight, Microsoft SQL, Mobile App, MongoDb, Office365, OneDrive, Oracle Database, Sharepoint
    • Integration: AS2, BizTalk EDIFACT, BizTalk Flat file encoder, BizTalk JSON Encoder, BizTalk Rules, BizTalk Trading Partner, BizTalk Transformation Service, BizTalk X12, BizTalk XML Validator, BizTalk XPath Extractor, Informix Connector, MQ Connector, Wait
    • Protocol Connectors: File, FTP, HTTP, POP3, SFTP, SMTP

All these connectors are technically API apps that uses a metadata format called Swagger and REST as pluggable interfaces and JSON as the interservice data format.

Swagger, most popular metadata format, is a specification for documenting REST APIs. It is language-agnostic so there are different implementations for different platforms.

In Azure API Apps, Microsoft adapts Swashbuckle to implement Swagger2.0.

By the way, if you don’t find any API App for your solution in Azure Marketplace, you can create your own custom API App.

image.png

So API apps in Azure App Service make it easy to develop, publish, manage, and monetize APIs. If you have some capability you want to expose as an API you should deploy it as API App and benefit from scalable RESTful API with enterprise grade security, simple access control, automatic SDK and Access on-premises data using Hybrid Connections.

You can bring your API as-is. You can use ASP.NET, Java, PHP, Node.js or Python for your APIs. Your APIs can take advantage of the features of Azure App Service with no changes.

 

Triggers

Some connectors can also act as a trigger. A trigger starts a new instance of a workflow based on a specific event, like the arrival of an e-mail or an insert of new record in the table of your database or a change in your Azure Storage account.

There are following 4 types of trigger are supported as of now:

  • Recurring Schedule – “every X minutes”

image.png

  • Polling an API for a response
    • A 200 response means “Run”
    • A 202 response means “Wait”
  • WebHook – Every workflow has an endpoint you can POST to from any service
  • Registering an API App to “push” to a workflow – using a custom contract implemented for API Apps

You can also trigger a Logic app manually by clicking on “Run Now” button in portal.

image.png

 

Actions

·Each step after the trigger in a workflow is called an action. Each action typically maps to an operation on your connector or custom API apps. Actions can have dependency and they can be executed based on the condition like success or failure of the previous action.

image.png

3 ways to introduce dependencies between actions:

  • Implicitly – whenever you reference the output of an action you will depend on that action execution first.
  • Explicit “dependsOn” condition – you can mark certain actions to run only after previous ones have completed.

image.png

  • Explicit “expression” condition – a complex function that evaluates properties of other actionsimage.png

If there is no dependency between actions then they would run in parallel. In this demo there is no dependency between Twitter and Dropbox action and hence both get the same message every time a new post is posted on user’s Facebook timeline.

image.png

 

BizTalk

For more advanced integration scenarios, Azure App Services includes capabilities from BizTalk. BizTalk Server is Microsoft’s industry leading integration platform. The BizTalk API apps allow you to easily include validation, transformation, rules and more in to your Logic App workflows. Find out more in what are BizTalk API apps.

BizTalk API Apps also bring the EAI and B2B integration functionality to Logic Apps. Watch this session by Prashant where he cover how you can work with XML data in Logic Apps, receive, process and send B2B data and use the new Rules Engine to implement business policies that are a part of your logic.

Resources:

https://azure.microsoft.com/en-in/documentation/articles/app-service-logic-what-are-logic-apps/

https://gautambiztalkblog.com/2015/06/18/exploring-and-evaluating-azure-logic-apps/