Table of Contents
Airflow 2.0 introduced the concept of providers. Providers are packages that contain integrations with external systems. They are meant to extend capabilities of the core "Apache Airflow". Thus they are part of the vision of Airflow-as-a-Platform - where the Airflow Core provides basic data-workflow scheduling and management capabilities and can be extended by implementing Open APIs Airflow supports, adding Plugins that can add new features to the Core, and adding Providers that allow to interact with external systems.
The providers are released separately from the core Airflow and they are versioned independently. The ways how providers can extend the Airflow Core, including the types of providers, can be found at the Providers page. You can also find out there how you can create your own provider.
Providers can be maintained and released by the Airflow community or by 3rd-party teams. In any case - whether community-managed, or 3rd-party managed - they are released independently of the Airflow Core package.
When community releases the Airflow Core, it is released together with constraints, those constraints use the latest released version of providers, and our published convenience images contain a subset of most popular community providers. However, our users are free to upgrade and downgrade providers independently of the Airflow Core version as they see fit, as long as it does not cause conflicting dependencies.
You can read more about it in the Installation and upgrade scenarios chapter of our user documentation.
List of all available community providers is available at the Providers index.
When providers are accepted by the community, the process of managing and releasing them must follow the Apache Software Foundation rules and policies. This is especially, about accepting contributions and releasing new versions of the providers. This means that the code changes in the providers must be reviewed by Airflow committers and merged when they are accepted by them. Also, we must have sufficient test coverage and documentation that allow us to maintain the providers, and our users to use them.
The providers - their latest version in "main" branch of Airflow repository - are installed and tested together with other community providers and one of the key properties of the community providers is that the latest version of providers contribute their dependencies to constraints of Airflow, published when Airflow Core is released. This means that when users are using constraints published by Airflow, they can install all the providers together and they are more likely to not interfere with each other, especially they should be able to be installed together, without conflicting dependencies. This allows to add an optional "extra" to Airflow for each provider, so that the providers can be installed together with Airflow by specifying the "extra" in the installation command.
Because of the constraint and potential conflicting dependencies, the community providers have to be regularly updated and the community might decide to suspend releases of a provider if we find out that we have trouble with updating the dependencies, or if we find out that the provider is not compatible with other more popular providers and when the popular providers are limited by the constraints of the less popular ones. See Suspending releases for providers below for more details.
The following documents provide detailed information about specific aspects of provider management:
| Provider Governance | Governance framework, stewardship model, lifecycle stages (incubation, production, attic/deprecation), health metrics, and periodic reviews |
| 3rd-Party Providers | Relation to community providers, system test dashboards, and mixed governance model |
| Accepting New Providers | Prerequisites, approval process, and historical examples for proposing new community providers |
| Provider Releases and Versioning | Release process, SEMVER versioning, provider distribution states, and minimum Airflow version policy |
| Suspending and Removing Providers | Criteria and process for suspending or removing community providers |
| Managing Provider's Lifecycle | Technical how-to for creating, suspending, resuming, and removing providers |
This document describes the complete life-cycle of community providers - from inception and approval to Airflow main branch to being decommissioned and removed from the main branch in Airflow repository.
For full details, see Provider Governance.
See Provider Governance: Governance framework.
See Accepting New Community Providers.
See Provider Releases and Versioning.
See Provider Releases: Release process.
See Provider Releases: Versioning scheme.
See Provider Releases: Provider distribution states.
See Provider Releases: Upgrading minimum supported version of Airflow.
See Suspending and Removing Providers: Suspending releases.
See Suspending and Removing Providers: Removing providers.
See 3rd-Party Providers.