Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Azure Logic Apps
Azure SQL Database
Azure SQL Managed Instance
SQL Server on Azure Virtual Machines
Azure Database for PostgreSQL
Solution ideas
This article describes a solution idea. Your cloud architect can use this guidance to help visualize the major components for a typical implementation of this architecture. Use this article as a starting point to design a well-architected solution that aligns with your workload's specific requirements.
When you use Azure as a modern data platform, you have your choice of platform as a service (PaaS) or infrastructure as a service (IaaS). PaaS provides cloud-native options for data ingestion, transformation, and storage. IaaS gives you greater control over your hybrid infrastructure, starting with the size and type of virtual machines (VMs) you choose. With either approach, you can take advantage of a variety of fully managed relational, NoSQL, and in-memory databases, storage solutions, and analytics offerings that span proprietary and open-source engines. This example architecture shows both approaches.
Architecture
Download a Visio file of this architecture.
Workflow
IBM MQ is the middleware that acts as a loosely coupled link between a mainframe or midrange system and Azure services. Messages are received and sent according to application requirements to communicate with the mainframe application layer.
In a cloud-native approach, Azure Logic Apps uses the MQ connector to exchange messages with IBM MQ. The Scheduler feature orchestrates the Azure workflow, sending and receiving messages at recurring intervals of one second.
The MQ connector can send the messages that it reads directly to storage through a connector or send them to be transformed first. Logic Apps includes several options for data transformation, such as an inline JavaScript runtime that you can use to run simple JavaScript code snippets for data transformation, and data operations that perform transformations on JSON, CSV, and HTML table data. You can also create serverless, single-task functions by using Azure Functions.
Data is loaded into storage. Azure offers many managed data storage solutions, each providing different features and capabilities.
In an IaaS approach, a VM runs Microsoft Host Integration Server (HIS) with the BizTalk Adapter for WebSphere MQ. HIS exchanges messages with IBM MQ and exposes orchestration as a web service to a custom .NET application.
A .NET application persists the data using any supported Azure data store. For example, the application can mask data or use private endpoints for security.
Data is loaded into storage. Azure offers many managed data storage solutions, each providing different features and capabilities.
Components
.NET is a free, open-source development platform used for creating applications across different platforms and devices. In this architecture, .NET is used to create an application that pulls data through HIS to the data storage layer and can also access IBM WebSphere MQ servers directly through the Microsoft Client for MQ.
Host Integration Server (HIS) is a Microsoft integration platform that can serve as a message integrator through the WebSphere MQ adapter in Microsoft BizTalk Server. In this architecture, HIS runs on a VM and exchanges messages between IBM MQ and BizTalk Server. It serves as an MQ listener that polls the MQ server for messages at specified intervals in the IaaS approach.
Logic Apps is a cloud service that provides tools for data orchestration, data integration, and data transformation with hundreds of connectors for accessing data on-premises or in the cloud. In this architecture, Logic Apps uses the MQ connector to exchange messages with IBM MQ in a cloud-native PaaS approach. The scheduler feature orchestrates workflows at recurring intervals.
Logic Apps MQ connector is a connector that connects your Logic Apps workflows to an IBM MQ server on-premises or on Azure. In this architecture, the MQ connector enables workflows to receive and send messages stored in your MQ server and includes support for IBM WebSphere MQ versions 7.5, 8.0, and 9.0-9.2.
Logic Apps scheduler is a feature that provides triggers for starting and running workflows based on the interval and frequency of recurrence that you specify. In this architecture, Logic Apps scheduler orchestrates the Azure workflow by sending and receiving messages at recurring intervals of one second.
Alternatives
For the data layer, you have your choice of managed services:
Azure SQL Database. Part of the Azure SQL family, SQL Database is an intelligent and scalable relational database service that's built for the cloud. Always up to date, it includes automated features that optimize performance, durability, and scalability, so you can focus on building new applications.
Azure SQL Managed Instance. Part of the Azure SQL service family, SQL Managed Instance combines the broadest SQL Server engine compatibility with all the benefits of a fully managed PaaS.
Azure SQL on Azure Virtual Machines. Part of the Azure SQL family, this cost-effective option is designed for lifting and shifting SQL Server workloads to Azure. It combines the performance, security, and analytics of SQL Server with the flexibility and hybrid connectivity of Azure. It also provides 100% code compatibility and includes SQL Server 2019 images.
Azure Database for PostgreSQL. This fully managed relational database service is based on the community edition of the open-source PostgreSQL database engine. You can focus on application innovation instead of database management and easily scale your workloads.
Azure Database for MySQL. This fully managed relational database service is based on the community edition of the open-source MySQL database engine.
Azure Cosmos DB. A globally distributed, multi-model database, Azure Cosmos DB provides throughput and storage that scales elastically and independently across any number of geographic regions. It's a fully managed NoSQL database service that guarantees single-digit-millisecond latencies at the 99th percentile anywhere in the world.
Fabric Data Warehouse. This enterprise analytics service speeds up time to insight across data warehouses and big data systems.
For the storage layer, create an enterprise data lake using Azure Data Lake Storage.
For the data layer, create a big data analytics platform using Microsoft Fabric.
Scenario details
A popular approach in digital transformation scenarios is to see whether existing applications and middleware tiers can run as-is in a hybrid setup where Microsoft Azure serves as the scalable, distributed data platform. This example describes a data-first approach to middleware integration that enables IBM message queues (MQs) running on mainframe or midrange systems to work with Azure services so you can find the best data platform for your workload.
In this scenario, Azure can serve as either a PaaS or an IaaS. PaaS enables cloud-native options for ingestion, transformation, and storage. IaaS provides greater control over hybrid infrastructure through customizable VMs. Both approaches let you take advantage of fully managed databases, storage, and analytics offerings across proprietary and open-source engines.
The following architecture illustrates how these options work in practice:
Cloud-native PaaS. Azure Logic Apps exchanges messages with IBM MQ through the MQ connector. Additional connectors provide quick access to events, data, and actions across other apps, services, systems, protocols, and platforms. Logic Apps also includes tools for transforming data from the queue if you need to modify the data format, structure, or values before storing it on Azure or sending it to the application layer.
VM-based IaaS. By running Microsoft Host Integration Server (HIS) on a VM, you can use a messaging integration component that connects to IBM MQ. You control the data transformation process by creating a .NET application to read and write messages. The application can persist data in the Azure data store of your choice, and you can choose the MQ server's polling interval.
A hybrid datacenter configuration makes sense for organizations that are developing their cloud strategies. Connecting to Azure can help bridge the gaps in your datacenter, enhance performance, improve business continuity, and expand your reach globally.
For example, applications on-premises can communicate with a modern data platform on Azure to take advantage of big data analytics or machine learning. If you need a cost-effective storage solution, you can replicate mainframe data, store it on Azure, and keep the data in sync. Azure can also add the scale needed to support online transaction processing (OLTP), batch, and data ingestion systems.
Potential use cases
Either of these approaches can be used to:
Enable loosely coupled applications that communicate through messaging systems to use the Azure data platform.
Sync or replicate data incrementally between a mainframe or midrange system and Azure.
Flow event messages between mainframe or midrange systems and Azure.
Stream IBM MQ messages into Azure to enable near real‑time processing, analytics, and decoupled downstream consumption.
Data loading
You can use Logic Apps connectors to send messages directly to Azure Storage and Data Lake Storage. For example, Logic Apps includes the Azure Blob Storage connector, as the following diagram shows. The connector makes it easy to store massive amounts of unstructured data in Blob Storage. Your data becomes reachable from anywhere in the world via HTTP or HTTPS.
Download a Visio file of this architecture.
Blob Storage also supports Data Lake Storage, a big data analytics solution for the cloud. You can load data into storage by using AzCopy, Azure Data Factory, or another solution that can connect to storage.
The PaaS and IaaS architecture options both support many popular managed database services. You can load data by using a custom-built loader, a vendor solution, or a managed service like Data Factory.
Contributors
Microsoft maintains this article. The following contributors wrote this article.
Principal authors:
- Nithish Aruldoss | Engineering Architect
- Ashish Khandelwal | Principal Engineering Architecture Manager
To see nonpublic LinkedIn profiles, sign in to LinkedIn.