Skip to content

Commit fded294

Browse files
authored
Merge pull request #36810 from bgavrilovicMS/eq_shard_end_of_support
End of support note for elastic query horizontal partitioning
2 parents 6a81491 + 055fb0f commit fded294

21 files changed

Lines changed: 161 additions & 17 deletions

azure-sql/database/elastic-convert-to-use-elastic-tools.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ monikerRange: "=azuresql || =azuresql-db "
1616

1717
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1818

19+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
20+
1921
Easily manage your existing scaled-out sharded databases using tools (such as the [Building scalable cloud databases](elastic-database-client-library.md)). First convert an existing set of databases to use the [shard map manager](elastic-scale-shard-map-management.md).
2022

2123
## Overview

azure-sql/database/elastic-database-client-library.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ monikerRange: "=azuresql || =azuresql-db "
1616

1717
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1818

19+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
20+
1921
Scaling out databases can be easily accomplished using scalable tools and features for Azure SQL Database. In particular, you can use the **Elastic Database client library** to create and manage scaled-out databases. This feature lets you easily develop sharded applications using thousands databases in Azure SQL Database.
2022

2123
**To download:**

azure-sql/database/elastic-database-perf-counters.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ monikerRange: "=azuresql || =azuresql-db "
1717

1818
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1919

20+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
21+
2022
Performance counters are used to track the performance of [data dependent routing](elastic-scale-data-dependent-routing.md) operations. These counters are accessible in the **Performance Monitor**, in the "Elastic Database: Shard Management" category.
2123

2224
You can capture the performance of a [shard map manager](elastic-scale-shard-map-management.md), especially when using [data dependent routing](elastic-scale-data-dependent-routing.md). Counters are created with methods of the `Microsoft.Azure.SqlDatabase.ElasticScale.Client` class.

azure-sql/database/elastic-database-recovery-manager.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ ms.custom:
1717

1818
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1919

20+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
21+
2022
The [RecoveryManager](/dotnet/api/microsoft.azure.sqldatabase.elasticscale.shardmanagement.recovery.recoverymanager) class provides ADO.NET applications the ability to easily detect and correct any inconsistencies between the global shard map (GSM) and the local shard map (LSM) in a sharded database environment.
2123

2224
The GSM and LSM track the mapping of each database in a sharded environment. Occasionally, a break occurs between the GSM and the LSM. In that case, use the `RecoveryManager` class to detect and repair the break.

azure-sql/database/elastic-query-getting-started.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ monikerRange: "=azuresql || =azuresql-db"
1616

1717
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1818

19+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
20+
1921
You can create reports from multiple databases from a single connection point using an [elastic query](elastic-query-overview.md). The databases must be horizontally partitioned (also known as "sharded").
2022

2123
If you have an existing database, see [Migrate existing databases to scale out](elastic-convert-to-use-elastic-tools.md).
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: Migration guide from elastic query shard map manager mode
3+
description: Learn about migration options for elastic query with EXTERNAL DATA SOURCE type SHARD_MAP_MANAGER, which is reaching end of support on March 31, 2027.
4+
author: WilliamDAssafMSFT
5+
ms.author: wiassaf
6+
ms.reviewer: bgavrilovic
7+
ms.date: 03/18/2026
8+
ms.service: azure-sql-database
9+
ms.subservice: scale-out
10+
ms.topic: conceptual
11+
---
12+
13+
# Migration from elastic query shard map manager mode
14+
15+
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
16+
17+
Elastic query in shard map manager mode (horizontal partitioning), using `EXTERNAL DATA SOURCE` type `SHARD_MAP_MANAGER`, is reaching end of support on March 31, 2027. After this date, existing workloads will continue to function but will no longer receive support, and creation of new external data sources of type `SHARD_MAP_MANAGER` will no longer be possible. This article contains options to migration from elastic query shared map manager mode.
18+
19+
For customers using elastic query with `EXTERNAL DATA SOURCE` type `SHARD_MAP_MANAGER`, the best alternative depends on the use case for using elastic query and on the overall scenario and architecture.
20+
21+
This article describes possible alternatives to elastic query shard map manager mode, and key considerations for each.
22+
23+
## Microsoft Fabric
24+
25+
**Best for:** OLAP (online analytical processing) and reporting scenarios.
26+
27+
Microsoft Fabric offers robust capabilities for large-scale analytics and reporting, enabling seamless data integration and advanced analytics workloads. However, migration might require rearchitecting existing solutions and retraining teams on new tools. Evaluate the cost implications and compatibility with your existing data sources.
28+
29+
For more information, see [Microsoft Fabric documentation](/fabric/fundamentals/microsoft-fabric-overview).
30+
31+
## SQL mirroring to Fabric
32+
33+
**Best for:** Reporting and analytics on centralized data.
34+
35+
Mirroring SQL databases to Fabric can simplify reporting and analytics on centralized data. Consider the latency and synchronization requirements between source and mirrored data. Also, assess the complexity involved in setting up mirroring and the impact on ongoing operations.
36+
37+
For more information, see [Microsoft Fabric mirrored databases from Azure SQL Database](/fabric/database/mirrored-database/azure-sql-database).
38+
39+
## ETL-based approach (for example, using Azure Data Factory)
40+
41+
**Best for:** Batch processing and scheduled data movement.
42+
43+
Using ETL pipelines like Azure Data Factory (ADF) provides flexible, scheduled data movement and transformation. This approach works well for batch processing but might introduce delays in data freshness. Consider the maintenance overhead and the scalability of ETL jobs as data volume grows.
44+
45+
For more information, see [Azure Data Factory documentation](/azure/data-factory/) or [Data Factory in Microsoft Fabric](/fabric/data-factory/data-factory-overview).
46+
47+
## Fully migrate data plane to Fabric OneLake
48+
49+
**Best for:** Centralized data management with unified analytics.
50+
51+
When you migrate entirely to Fabric OneLake, you centralize data management and take advantage of unified analytics features. This migration might require significant effort, possible downtime, and refactoring applications. Weigh the long-term benefits against short-term migration challenges, and make sure it fits your business requirements.
52+
53+
For more information, see [OneLake documentation](/fabric/onelake/onelake-overview).
54+
55+
## Azure SQL Database Hyperscale
56+
57+
**Best for:** Workloads where sharding was implemented due to storage limitations that Hyperscale overcomes today.
58+
59+
Azure SQL Database Hyperscale works well for workloads that need high performance, scalability, and rapid growth in data volume. It supports databases of any size, with fast backup and restore, and high concurrency. If you originally implemented sharding because of storage limitations, you can rearchitect your system from a sharded topology to a monolithic database. Migration involves consolidating databases and adapting your application logic for centralized storage. Consider cost, performance, and operational implications, especially if you currently rely on distributed query capabilities.
60+
61+
For more information, see [Azure SQL Database Hyperscale service tier](service-tier-hyperscale.md).
62+
63+
## Elastic jobs
64+
65+
**Best for:** Running queries on individual databases or shards without result aggregation.
66+
67+
If you use elastic query to run queries on individual databases or shards in the fleet without result aggregation, elastic jobs provide the same capability. Elastic jobs are ideal for automating and managing operations across multiple databases. They don't support result aggregation, so they're best suited for scenarios where independent queries suffice. Review the job scheduling features and integration with monitoring tools to ensure operational efficiency.
68+
69+
For more information, see [Elastic jobs in Azure SQL Database](elastic-jobs-overview.md).
70+
71+
## Azure SQL Managed Instance
72+
73+
**Best for:** Point-to-point cross-database queries using three-part names or linked servers.
74+
75+
Azure SQL Managed Instance natively supports three-part name queries and linked servers. If you use elastic query for point-to-point queries, a SQL managed instance is a natural migration target. Consider networking setup, security configurations, and licensing costs. Ensure that your workload fits within the SQL managed instance's performance and scalability limits.
76+
77+
For more information, see [What is Azure SQL Managed Instance?](/azure/azure-sql/managed-instance/sql-managed-instance-paas-overview)
78+
79+
## Custom fanout query and result aggregation layer
80+
81+
**Best for:** Preserving existing Azure SQL Database architecture with maximum flexibility.
82+
83+
Building a custom fanout and aggregation layer preserves your existing architecture and provides maximum flexibility. The fanout query runs on top of the SQL Database architecture by using a customer-built layer. This approach requires development effort, ongoing maintenance, and robust error handling. Evaluate the complexity of building and supporting this solution, as well as its scalability and reliability for your needs.
84+
85+
## Related content
86+
87+
- [Elastic query overview](elastic-query-overview.md)
88+
- [Reporting across scaled-out cloud databases](elastic-query-horizontal-partitioning.md)
89+
- [Getting started with elastic query for horizontal partitioning (sharding)](elastic-query-getting-started.md)

azure-sql/database/elastic-query-horizontal-partitioning.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
22
title: Reporting Across Scaled-Out Cloud Databases
3-
description: how to set up elastic queries over horizontal partitions
3+
description: How to set up elastic queries over horizontal partitions.
44
author: WilliamDAssafMSFT
55
ms.author: wiassaf
66
ms.reviewer: bgavrilovic, mathoma
7-
ms.date: 06/13/2025
7+
ms.date: 03/17/2026
88
ms.service: azure-sql-database
99
ms.subservice: scale-out
1010
ms.topic: how-to
@@ -13,9 +13,10 @@ ms.custom:
1313
monikerRange: "=azuresql || =azuresql-db"
1414
---
1515
# Reporting across scaled-out cloud databases (preview)
16-
1716
[!INCLUDE[appliesto-sqldb](../includes/appliesto-sqldb.md)]
1817

18+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
19+
1920
Sharded databases distribute rows across a scaled out data tier. The schema is identical on all participating databases, also known as horizontal partitioning. Using an elastic query, you can create reports that span all databases in a sharded database.
2021

2122
:::image type="content" source="media/elastic-query-horizontal-partitioning/horizontal-partitioning.png" alt-text="Diagram of how queries work across shards." lightbox="media/elastic-query-horizontal-partitioning/horizontal-partitioning.png":::

azure-sql/database/elastic-query-overview.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Elastic query enables you to run a Transact-SQL query that spans mu
44
author: WilliamDAssafMSFT
55
ms.author: wiassaf
66
ms.reviewer: drskwier, bgavrilovic, mathoma, randolphwest
7-
ms.date: 01/30/2025
7+
ms.date: 03/17/2026
88
ms.service: azure-sql-database
99
ms.subservice: scale-out
1010
ms.topic: overview
@@ -16,7 +16,7 @@ ms.custom:
1616

1717
[!INCLUDE [appliesto-sqldb](../includes/appliesto-sqldb.md)]
1818

19-
The elastic query feature (in preview) enables you to run a Transact-SQL (T-SQL) query that spans multiple databases in Azure SQL Database. It allows you to perform cross-database queries to access remote tables, and to connect Microsoft and third-party tools (Excel, Power BI, Tableau, etc.) to query across data tiers with multiple databases. Using this feature, you can scale out queries to large data tiers and visualize the results in business intelligence (BI) reports.
19+
The elastic query feature (in preview) enables you to run a Transact-SQL (T-SQL) query that spans multiple databases in Azure SQL Database. It allows you to perform cross-database queries to access remote tables, and to connect Microsoft and non-Microsoft tools (Excel, Power BI, Tableau, etc.) to query across data tiers with multiple databases. Using this feature, you can scale out queries to large data tiers and visualize the results in business intelligence (BI) reports.
2020

2121
## Why use elastic queries
2222

@@ -93,6 +93,8 @@ After running the DDL statements, you can access the remote table `mytable` as t
9393

9494
## Horizontal partitioning - sharding
9595

96+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
97+
9698
Using elastic query to perform reporting tasks over a sharded, that is, horizontally partitioned, data tier requires an [elastic database shard map](elastic-scale-shard-map-management.md) to represent the databases of the data tier. Typically, only a single shard map is used in this scenario and a dedicated database with elastic query capabilities (head node) serves as the entry point for reporting queries. Only this dedicated database needs access to the shard map. Figure 4 illustrates this topology and its configuration with the elastic query database and shard map. For more information about the elastic database client library and creating shard maps, see [Shard map management](elastic-scale-shard-map-management.md).
9799

98100
**Figure 4** Horizontal partitioning - Using elastic query for reporting over sharded data tiers

azure-sql/database/elastic-scale-data-dependent-routing.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ ms.custom:
2121

2222
The application does not need to track various connection strings or DB locations associated with different slices of data in the sharded environment. Instead, the [Scale out databases with the shard map manager](elastic-scale-shard-map-management.md) opens connections to the correct databases when needed, based on the data in the shard map and the value of the sharding key that is the target of the application's request. The key is typically the *customer_id*, *tenant_id*, *date_key*, or some other specific identifier that is a fundamental parameter of the database request.
2323

24+
[!INCLUDE [elastic-query-shard-map-manager-mode-end-of-support](includes/elastic-query-shard-map-manager-mode-end-of-support.md)]
25+
2426
For more information, see [Scaling Out SQL Server with Data-Dependent Routing](/previous-versions/sql/sql-server-2005/administrator/cc966448(v=technet.10)).
2527

2628
## Download the client library

azure-sql/database/elastic-scale-faq.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ summary: |
1919
sections:
2020
- name: Ignored
2121
questions:
22+
- question: |
23+
Is elastic queries in shard map manager mode supported?
24+
answer: |
25+
Elastic query in shard map manager mode (horizontal partitioning), using `EXTERNAL DATA SOURCE` type `SHARD_MAP_MANAGER`, is reaching end of support on March 31, 2027. After this date, existing workloads will continue to function but will no longer receive support, and creation of new external data sources of type `SHARD_MAP_MANAGER` will no longer be possible. For migration options, see [Migration guide from elastic query shard map manager mode](elastic-query-horizontal-partitioning-migration.md).
26+
2227
- question: |
2328
If I have a single-tenant per shard and no sharding key, how do I populate the sharding key for the schema info?
2429
answer: |

0 commit comments

Comments
 (0)