Provide an option to opt-in Beta (unstable) library

[JeffLuoo]

What problem do you want to solve?

Context

Following the OpenTelemetry Stable by Default proposal, distributions are expected to only enable stable components by default. This prevents users from unknowingly relying on experimental features that might break or change behavior unexpectedly.

In OpenTelemetry Python, we currently have OTEL_PYTHON_DISABLED_INSTRUMENTATIONS to opt-out of specific instrumentations. However, we lack a clear, standard way to opt-in to unstable (beta/alpha) instrumentations once the "stable by default" policy is enforced.

Describe the solution you'd like

Proposed solution

This feature request proposes to introduce a new environment variable and declarative configuration option to allow users to selectively enable unstable instrumentations.

OTEL_PYTHON_ENALBED_UNSTABLE_INSTRUMENTATIONS

1. Fine-Grained Opt-In

Introduce OTEL_PYTHON_ENABLED_UNSTABLE_INSTRUMENTATIONS.

Type: Comma-separated list of strings (similar to OTEL_PYTHON_DISABLED_INSTRUMENTATIONS).
Behavior: Only the specified unstable instrumentations will be loaded in addition to all stable ones.
Example: OTEL_PYTHON_ENABLED_UNSTABLE_INSTRUMENTATIONS="requests,django"

2. Interaction with Disable List

If an instrumentation is present in both OTEL_PYTHON_ENABLED_UNSTABLE_INSTRUMENTATIONS and OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, the disable list should take precedence. This ensures that users can easily disable a library even if it was enabled as unstable.

Describe alternatives you've considered

Alternative options

A. Coarse-Grained Opt-In (Boolean Flag)
Set the value of OTEL_PYTHON_ENABLED_UNSTABLE_INSTRUMENTATIONS (or a similar variable) to true/false.

Pros: Simple for users who want to test all available unstable instrumentations.
Cons: Violates the principle of least surprise and doesn't allow fine-grained control. If one unstable library is causing issues, the user has to disable all unstable libraries or use the disable list (which might get messy).

B. Minimum Stability Level Threshold
Instead of listing libraries, allow users to set a minimum stability level to load.

Variable: OTEL_PYTHON_MINIMUM_STABILITY_LEVEL
Values: stable (default), beta, alpha.
Behavior: If set to beta, both stable and beta libraries are loaded.

Note: This aligns with the suggestion in the stability proposal blog post to "select a desired minimum stability level".

Additional Context

Please let me know which option you prefer.

cc: @aabmass

Would you like to implement a fix?

None

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[herin049]

Generally, I'd prefer to still have some mechanism to enable all unstable instrumentation libraries. My preference would be to add both an OTEL_PYTHON_MINIMUM_STABILITY_LEVEL and an OTEL_PYTHON_UNSTABLE_INSTRUMENTATIONS environment variable to handle the common cases of enabling all unstable instrumentation and opting in to a few unstable instrumentations.

Even currently, I don't see the addition of these environment variables providing too much benefit to users because you can always just opt to not install the unstable instrumentation packages to begin with. The only scenarios I see where you'd need to rely on OTEL_PYTHON_MINIMUM_STABILITY_LEVEL is if additional packages are being injecting into your Python environment without your control (e.g. AWS Lambda layers)

A related change that should be made is to update the opentelemetry-contrib-instrumentations to only include stable packages by default, and potentially add a "beta" or "unable" extra that allows users to install all beta packages.

[JeffLuoo]

Also ref: open-telemetry/opentelemetry-specification#4813