Most awaited Azure Table Storage Connector is available now!

Recently Azure Table Storage Connector has been released in preview.

Now, the connector is only available in West Central US. Hopefully soon it will be rolled out to other data centers.

To play around this connector I created this very simple Logic App which pulls an entity from the table storage.

 

I have a table storage called RobustCloudIntegrationWithAzure as shown below.

This table basically stores all the authors and chapters name of the book Robust Cloud Integration With Azure.

The author or the chapter is the partition key and the sequence number is the row Key. To get any author or chapter details, you need to pass partition key and the row key to the logic app

First you need to make a connection to the Azure Storage table, by providing the Storage Account Name with Shared Storage Key. You also need to give a name to your connection.

Once you have made the connection successfully, you can use any action of CRUD operation. In this case I am using Get Entities which is basically a select operation.

Once you have selected the table, you have option to user Filter and Select OData query. In the Filter Query I have condition to check for partionKey which is coming from input request. In the Select Query, you can choose the columns of the table to display.

So, this logic app receives an request with the partionKey and rowKey as inputs.

Then it checks the value of partitionKey. If a partitionkey is equal to the author, Author action would be executed, else Chapter action. Depending on the partitionkey, either author or chapter details will be sent out as the response from the Logic App workflow

Here are the sample request and response using Postman.

Author

Chapter

 

Conclusion

Azure Table Storage Connector was one of the most voted request to Logic App team and now it’s available to use.

Advertisements

Getting started with Enterprise Integration Pack for Logic Apps

With Azure Logic Apps, you can now implement “serverless”, cloud-based enterprise integration workflows for EAI & B2B scenarios

  • EAI – Enterprise Application Integration
  • B2B – Business-to-Business communication

The Enterprise Integration Pack features include the B2B, EDI and XML capabilities for handling complex business to business workloads. With this features, Logic Apps can easily leverage the power of BizTalk Server, Microsoft’s industry leading integration solution to enable integration professionals to build the solutions they need.

The pack uses industry standard protocols, including AS2, X12, and EDIFACT, to exchange messages between business partners. Messages can be optionally secured using both encryption and digital signatures.

Enterprise Integration Pack is based on integration account, which is a secure and scalable container that stores the various artifacts you need for more complex business process workflow such as, schemas for XML validation, maps for transformation, and trading partner agreements.

 

Integration Account

Integration Account, a container that stores the various artifacts you need for more complex business process workloads such as trading partner agreements.

It is essential to create an integration account for a Logic App to use EAI and B2B capabilities. To create an integration account, log in to Azure portal and go to New –> Enterprise Integration, as shown below. Select Integration Account here.

Now enter the Name for the integration account and select the Subscription, Resource group, and Location, as shown below. Click on the Create button.

This is how the integration account container SampleIntegrationAccount look like.

To use the artifacts stored in the integration account, you need to create a Logic App and link the integration account to it.

 

Integration account can hold the following integration artifacts used for Enterprise Integration scenarios:

XML schemas: You can use XML schema to define the message / document format that you expect to receive and send from source and destination systems respectively.

XSLT-based maps: This can be used to transform XML data from one format to another format.

Trading partners: This is a representation of a group within organization or partner you do business with. These are the entities that participate in Business-To-Business (B2B) messaging and transactions.

Trading partner agreements: When two partners establish a relationship, this is referred to as an agreement. Trading partner agreements is an understanding between two business profiles to use a specific message encoding protocol or a specific transport protocol while exchanging EDI messages with each other. Enterprise Integration supports three protocol/transport standards:

  • AS2
  • X12
  • EDIFACT

Certificates: Enterprise Integration uses certificates for secure messaging of EDI data, which is achieved using public and private keys. Organization (Trading Partner) generates keys, distributes the public, and keeps the private secret. Data encrypted by the public key can only be decrypted by the private key.

Certificates are just electronic documents that contains a public key. These certificates are digitally signed by a trusted certificate authority (CA) and the signature binds owner’s identity to the public key.

 

Logic Apps Enterprise Integration Tool

The Enterprise Integration Tool is an extension for Visual Studio 2015, which can be downloaded from here.

Basically, it adds an integration project type to Visual Studio 2015 and lets you create XML schemas, Flat File Schemas, and maps to build an EAI/B2B integration solution.

It uses the Logic App Schema editor, Flat File Schema generator, and XSLT mapper to easily create integration account artifacts. These artifacts, XSD and XSLT map files are uploaded to integration account so that you can use them for Enterprise Messaging in Logic App.

 

Integration account connectors

The integration pack connectors enable you to easily validate, transform and process different messages that you exchange with different applications within your enterprise (EAI) or with your business partners (B2B). If you work with BizTalk Server, then these connectors are a good fit to expand your BizTalk workflows into Azure.

Following enterprise features can be achieved by using Integration account connectors

EAI features:

  • XML Validation
  • Transform XML
  • Flat File Encoding
  • Flat File Decoding

B2B features:

  • AS2 – Decode AS2 Message
  • AS2 – Encode to AS2 Message
  • X12 – Decode X12 message
  • X12 – Encode to X12 message by agreement name
  • X12 – Encode to X12 message by identities
  • EDIFACT – Decode EDIFACT message
  • EDIFACT – Encode to EDIFACT message by agreement name
  • EDIFACT – Encode to EDIFACT message by identities

Together all these features/capabilities enable customers to create end to end automated business processes that scale with the cloud connecting you to your business partners quicker than ever on Logic Apps.

 

Enterprise Integration templates

Logic Apps has rich set of pre-built template and few of them are for Enterprise Integration as shown below.

VETER – Validate, Enrich, Transform, Extract, Route.

There is a quick start template on GitHub to try these scenarios. Here is the GitHub link for VETER scenario

 

EDI over AS2

 

Message handling in Logic Apps

The Enterprise Messaging in Logic Apps have the following features:

Flexibility in content types: Logic Apps are flexible enough to support different content types, such as binary, JSON, XML, and primitives. Now you can receive different message types in Logic Apps and then convert them to JSON or XML format required for the downstream systems. We also have new BizTalk connectors, which can be used to push the message to the on-premise BizTalk server.

The Enterprise Integration pack provides XSD support in Logic Apps. So, you can upload your XML schemas to integration account and use them in Logic App workflow and further convert them to the binary or JSON format as per your requirement.

Mapping: you can also create XSLT-based map in Visual Studio and use them in Logic App workflows. You can also leverage your existing assets-schema and maps by uploading them to integration account and using them in Logic Apps.

Flat file processing: You can easily convert Flat files into XML and vice versa. Built-in connectors support Logic Apps to convert csv, delimited, and positional file into XML and then into JSON/base64.

EDI: With Enterprise Integration Pack, Logic Apps now supports EDI processing for business-to-business (B2B) integration scenarios with out-of-the-box X12 and EDIFACT support. By enabling both encode and decode for these EDI standards you are able to receive or send EDI documents from Logic Apps.

Summary:

Enterprise Application Pack for Logic Apps comes with the concept of integration account that stores various artifacts you need for more complex business process workloads such as trading partner agreements. You need to use Enterprise Integration Tool to create enterprise artifacts such as schema and maps which would be used to create “serverless”, cloud-based enterprise integration workflows for EAI & B2B scenarios.

You can check out the next post to build your first Enterprise Messaging solution in Logic Apps.

Synegrate is now a Gold Partner for Microsoft’s Azure Cloud Platform

I am proud to be part of Synegrate which is now certified as Gold Partner for Microsoft’s Azure Cloud Platform. Synegrate was already a Managed Microsoft Gold Certified Partner in Application Integration and a Microsoft Silver Certified Partner in Cloud Competency.

Today Synegrate achieved a Microsoft Gold Cloud Platform competency, demonstrating a best-in-class ability and competencies on the Azure platform.

The Cloud Platform competency is designed for partners to capitalize on the growing demand for infrastructure and software as a service (SaaS) solutions built on Microsoft Azure.

This is the highest attainable partnership level and is earned after achieving the defined competency requirements.

To earn a Microsoft gold competency, partners must successfully complete exams (resulting in Microsoft Certified Professionals) to prove their level of technology expertise, and then designate these certified professionals uniquely to one Microsoft competency, ensuring a certain level of staffing capacity. They also must submit customer references that demonstrate successful projects, meet a performance (revenue and or consumption/usage) commitment, and pass technology and/or sales assessments.

Achieving the Microsoft Gold Cloud Platform competency showcases Synegrate’s expertise in and commitment to today’s cloud technology market and demonstrates deep knowledge of Microsoft’s Cloud Platform.

Synegrate has proven a reliable partner for customers globally. Over the years the company has been successful in assisting customers in various Microsoft solution based endeavors that created value propositions ranging from reduced costs or complexity to increased availability and security. Here are some stories and examples of our loyal customers.

 

About Synegrate

Synegrate’s core focus is data. We have data coursing through our veins; it is in our DNA. We specialize in the storing, integration, dissemination, visualization and analytics of data. We create modern data driven applications, BPM (Business Process Management) processes to orchestrate data and dashboards for data analysis.

Synegrate is a 100% Microsoft focused company that is fully committed to the Microsoft Azure cloud services and solutions.

We utilize the platforms, products and tools provided by Microsoft, to provide our customers with innovation, analytics and insight. We’re a front runner in helping our customers realize their future state architectures on the Microsoft Azure cloud.

We have our Head Office in California, development centers in different regions, allowing us to service the US from coast to coast.

 

Connect with Synegrate @ http://www.synegrate.com/

When to use Logic Apps vs BizTalk Server

Now a days, this is a very common and valid question in the BizTalk community, both for existing BizTalk customer and for new one too.

Here is what Tord answered in the open Q&A with product group at 100th Episode of integration Monday.  Check at ~ 30.30 minutes of the video.

If your solution need to communicate with SaaS application, Azure workloads and cloud business partners (B2B) all in cloud then you should use Azure Logic Apps, but if you are doing lot of integration with on-premise processing by communicating with on-premise LOB applications, then BizTalk is the pretty good option. You can use both if you are doing hybrid integration.

So basically, it depends on scenario to scenario based on your need and architecture of your solution.

 

Cloud Integration

Many enterprises now use a multitude of cloud-based SaaS services, and being able to integrate these services and resources can become complex. This is where the native capability of Logic Apps can help by providing connectors for most enterprise and social services and to orchestrate the business process flows graphically.

If your resources are all based in the cloud, then Logic Apps is a definite candidate to use as an integration engine.

Natively, Logic Apps provides the following key features:

Rapid development: Using the visual designer with drag and drop connectors, you design your workflows without any coding using a top-down design flow. To get started, Microsoft has many templates available in the marketplace that can be used as is, or modified to suit your requirements. There are templates available for Enterprise SaaS services, common integration patterns, Message routing, DevOps, and social media services.

Auditing: Logic Apps have built-in auditing of all management operations. Date and time when workflow process was triggered and the duration of the process. Use the trigger history of a Logic App to determine the activity status:

  • Skipped: Nothing new was found to initiate the process
  • Succeeded: The workflow process was initiated in response to data  being available
  • Failed: An error occurred due to misconfiguration of the connector

A run history is also available for every trigger event. From this information, you can determine if the workflow process succeeded, failed, cancelled, or is still running.

Role-based access control (RBAC): Using RBAC in the Azure portal, specific components of the workflow can be locked down to specific users. Custom RBAC roles are also possible if none of the built-in roles fulfills your requirements.

Microsoft managed connectors: There are several connectors available from the Azure Marketplace for both enterprise and social services, and the list is continuously growing. The development community also contributes to this growing list of available connectors as well.

Serverless scaling: Automatic and built in on any tier.

Resiliency: Logic Apps are built on top of Azure’s infrastructure, which provides a high degree of resiliency and disaster recovery.

Security: This supports OAuth2, Azure Active Directory, Cert auth and Basic auth, and IP restriction.

There are also some concerns while working with Logic Apps, shared by Microsoft IT team at INTEGRATE 2017

You can also refer the book, Robust cloud integration with Azure to understand and get started with integration in cloud.

 

Hybrid Integration

When you have, resources scattered in the cloud and on premise, then you may want to consider BizTalk as a choice for this type of hybrid integration along with Logic Apps.

BizTalk 2016 include an adapter for Logic Apps. This Logic App adapter will be used to integrate Logic Apps and BizTalk sitting on premise. Using the BizTalk 2016 Logic App adapter on-premise, resources can directly talk to a multitude of SaaS platforms available on cloud.

The days of building monolithic applications are slowly diminishing as more enterprises see the value of consuming SaaS as an alternative to investing large amounts of capex to buy Commercial Off the Self (COTS) applications. This is where Logic Apps can play a large part by integrating multiple SaaS solutions together to form a complete solution.

BizTalk Server has been around since 2000, and there have been several new products releases since then. It is a very mature platform with excellent enterprise integration capabilities.

Below is a short comparison matrix between BizTalk and Logic Apps:

Conclusion

Microsoft Integration platform has all the option for all kind of customer’s integration need.

How to protect your web site using WAF-enabled Azure Application Gateway

Azure Application Gateway a Layer-7 HTTP load balancer that provides application-level routing and load balancing services. It distributes traffic requests based upon data found in application layer protocols such as HTTP/HTTPS and also on application specific data such as HTTP headers, cookies, or data within the application message itself, such as the value of a specific parameter.

You basically need to define rules to accept the traffic requests and route them to the appropriate back-end instances.

Application Gateway currently supports the following features:

    • Web Application Firewall (WAF)
    • Scaleble, highly-available HTTP load balancing solution
    • Cookie-based session affinity
    • SSL offload for better utilization
    • URL-based content routing
    • Multi-site routing
    • Web socket support
    • Health monitoring
    • Advance diagnostics

While Azure is responsible for securing the infrastructure and platform that your application runs on, it is your responsibility to secure your application itself. Now Web Application Firewall (WAF) in Azure Application Gateway can provide protection to your web applications against common threats such as SQL injection, cross-site scripting attacks, and session hijacks.

If your organization hosts highly sensitive information, the number-one priority is having a fully-isolated and dedicated environment for only your organization’s applications. Using an App Service Environment, your organization can have security and isolation for your web apps and use a virtual network for control over traffic.

An App Service Environment is a premium service plan option of Azure App Service that provides a fully isolated and dedicated environment. App Service Environments are isolated to run only a single customer’s applications and are always deployed into an Azure Virtual Network. At a high level, an App Service Environment consists of compute resources running in the Azure Hosted Service, Storage, Database, a Virtual Network, and a subnet with the hosted service running in it.

From a single open port, one option to block most traffic would be to use WAF in Application gateway in front of ASE to protect your Web apps.You can also Create a network security group, and assign it to a subnet in your Azure Virtual Network to restrict traffic to the App Service Environment from the WAF only by using the VIP address.

Architecture Overview

Here you have all the security with a straight forward architecture. Easy to provision, maintain and administer.

The path for request would be: App Gateway (WAF mode) –> ASE

image.png

To create this architecture here are the steps involved:

  • Create a virtual network (ex: frontend-vnet) for both App Service Environment (ASE) and Application Gateway(AG).
  • Create subnet for Application Gateway. Subnet for App Service Environment will be created as a part of ASE provision process.
  • Creates an App Service Environment in your virtual network with a private internal load balancer address using Azure Quickstart Template.  This step would take up to 2 hours to complete.
  • Deploy a test web app – The vnet (frontend-vnet) is not publicly accessible so in order to deploy app, you need to create a Virtual Machine that is living within the same Virtual Network and use that to deploy and access the Web App with its internal IP. Once you have deployed your test web app, you should successfully be able to  access it from any VM which is living within same vnet (frontend-vnet).
  • Create WAF-enable Application Gateway
  • Configure Application Gateway
  • Test your web app form public endpoint.

In this blog post I will go through the creation and configuration of Application Gateway in detail.

 

Create WAF-enabled Application Gateway

In Azure Portal, Go to New—>Networking and select Application Gateway. Provide the information for the basic setting as shown below. Make sure you select WAF tier.

image.png

In the settings, make sure to select the same Virtual Network (frontend-vnet) you used to configure ASE earlier and the subnet you created specifically for the Application Gateway. You also need configure the public IP address.

Configure the WAF specific settings.

  • Firewall status – This setting turns WAF on or off.
  • Firewall mode – This setting determines the actions WAF takes on malicious traffic. If Detection is chosen, traffic is only logged. If Prevention is chosen, traffic is logged and stopped with a 403 Unauthorized.

image.png

Review the results and click on OK to create the gateway.

Configure the Application Gateway

Add servers to backend pool – Once the application gateway is created, go to the Backend Pools and select the current backend pool.

image.png

Add the IP address of ILB ASE and Save. Now the incoming traffic that enters the application gateway would be routed to the backend address added here.

Configure SSL offload – Application gateway can be configured to terminate the Secure Sockets Layer (SSL) session at the gateway to avoid costly task of decrypting HTTPS traffic off your web servers. Application gateway decrypts the request and sends it to backend server and re-encrypts the response before sending it back to the client.

To configure SSL offload with an application gateway, a certificate (pfx format) is required. This certificate is loaded on the application gateway and used to encrypt and decrypt the traffic sent via SSL.

Add an HTTPS listener – It will look for traffic based on its configuration and helps route the traffic to the backend pools. Click Listeners and click the Add button to add a listener. Fill out the required information for the listener and upload the .pfx certificate.

image.png

Create a rule and associate it to the listener – Once listener is created, you need to create a rule to handle the traffic from the listener. Click the Rules of the application gateway, and then click Add. Type in the friendly name for the rule and choose the listener created in the previous step. Choose the appropriate backend pool and http setting and click OK.

image.png

Create the custom probe – Custom probes allow you to have a more granular control over the health monitoring. When using custom probes, you can configure the probe interval, the URL and path to test, and how many failed responses to accept before marking the back-end pool instance as unhealthy.

Probes are configured in a two-step process through the portal. The first step is to create the probe. Next you add the probe to the backend http settings of the application gateway. Create a Custom Probe with the Host set as your custom Web App domain, for example sample-app.com as shown below.

image.png

Add probe to the gateway – Go to the HTTP settings, and make sure that the setting has Custom Probes turned on and select the probe you just created. Otherwise, the Application Gateway will try to go to the IP of the App Service Environment without passing a Host header, which won’t work and will throw the probe into an Unhealthy state resulting in the 502 Gateway Proxy error.

image.png

Testing

There are couple of ways to do the testing. First you can use ModHeader Chrome extension to open the public IP address/hostname of the Application Gateway in the browser. You need to pass in the Custom Domain you configured on the Web App as a Host Header and the website should come up. Refer Sabbour blog post for further detail.

The other way is to add hostname (sample-app.com) to Custom Domains in the setting of app deployed in ASE as shown below.

image.png

You need to add an entry for your host in Hosts file on your local machine. The path would be c:\Windows\System32\Drivers\etc\hosts.

Now if you go to https://sample-app.com it should open up the sample web app as shown below.

image.png

Logging and troubleshooting

Application Gateway provides following capabilities to monitor resources.

Backend health – Application gateway provides the capability to monitor the health of individual members of the backend pools through the portal, PowerShell, and CLI.

image.png

Logging – There are different types of logs in Azure to manage and troubleshoot application gateways such as performance, firewall and access logs.

image.png

Here is a sample firewall log.

image.png

There are three different options to choose for storing your logs

  • Storage Account
  • Event Hubs
  • Log Analytics

image.png

Metrics – Application gateway currently has one metric. This metric measures the throughput of the application gateway in Bytes per second.

image.png

You can also set alert rule for application gateway based on metrics on a resource.

For example, an alert can email an administrator if the throughput of the application gateway is above, below or at a threshold for a specified period of time.

image.png

 

Summary

To summarize, we explored the option to protect your web applications against common threats such as SQL injection, cross-site scripting attacks, and session hijacks using Azure Application Gateway. We ‘ve hosted a Web App securely in an App Service Environment. This Web App isn’t publicly accessible as it is sitting in a subnet inside a Virtual Network and it isn’t exposed to the internet. The only way to access the site is through a Web Application Firewall enabled Application Gateway.

Microsoft Tech Summits 2017 and Global Integration Boot Camp

Microsoft Tech Summits kicked off today at Chicago with lots of keynotes, technical training sessions and hands-on labs to build and develop cloud skills of interested individuals.

There were also deep dive sessions covering a range of topics across Microsoft Azure and the hybrid platform including security, networking, data, storage, identity, mobile, cloud infrastructure, management, DevOps, app platform, productivity, collaboration and more.

image.png

The Microsoft Tech Summit provides Free, two-day technical training for IT professionals and developers with experts who build the cloud services across Microsoft Azure, Office 365, and Windows 10.

Here is the Agenda look like.

image.png

You can also find a city near you and Register for the event

Here’s a list of the currently published Tech Summit events around the globe:

  • Amsterdam,  March 23 – 24
  • Bangalore,  March 16 – 17
  • Birmingham, March 27 – 28
  • Chicago, January 19 – 20
  • Copenhagen, March 30 – 31
  • Frankfurt, February 9 – 10
  • Johannesburg, February 6 – 7
  • Milan, March 20 – 21
  • Seoul, April 27 – 28
  • Singapore, March 13 – 14
  • Washington D.C. March 6 – 7

.

Global Integration Bootcamp

There is another free event coming up for integration community – Global integration boot camp.

image.png

This event is driven by user groups and communities around the world, backed by Microsoft, for anyone who wants to learn more about Microsoft’s integration story. In this full-day boot camp there will deep-dive into Microsoft’s integration stack with hands-on sessions and labs, delivered to you by the experts and community leaders.

In this Boot Camp, the main focus will be on:

BizTalk 2016 –BizTalk Server 2016, what’s new, and using the new Logic Apps adapter
Logic Apps –Creating Logic Apps using commonly-used connectors
Servicebus –Build reliable and scalable cloud messaging and hybrid integration solutions
Enterprise Integration Pack –Using the Enterprise Integration Pack (EIP) with Logic Apps
API Management –How does API management help you organize your APIs and how does it increase security?
On-Premise Gateway –Connecting to on-prem resources using the On-Premise Gateway
Hybrid Integration –Hybrid integrations using BizTalk Server and Logic Apps
Microsoft Flow – Learn to compose flows with Microsoft Flow

image.png

If you are interested to be part of it or to host it on your location, you can reach out to organizers by providing your details.

Organizers