Skip to content

Feature/cursor pagination task instances UI#64953

Open
pierrejeambrun wants to merge 5 commits intoapache:mainfrom
astronomer:feature/cursor-pagination-task-instances-ui
Open

Feature/cursor pagination task instances UI#64953
pierrejeambrun wants to merge 5 commits intoapache:mainfrom
astronomer:feature/cursor-pagination-task-instances-ui

Conversation

@pierrejeambrun
Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun commented Apr 9, 2026

Replace offset-based pagination with cursor-based pagination for the TaskInstances listing page, leveraging the new cursor API endpoint. Pagination now shows only previous/next buttons without page numbers or total count, which eliminates the expensive COUNT(*) query for large datasets.

Add generic cursor pagination support to DataTable via an optional cursorPagination prop so other tables can adopt it.

Building on top of #64845 to plug the UI on this new cursor based pagination. Default table view is much faster.

Responses are much faster. Currenrly it would take ~60 on main to load.

After

Screenshot 2026-04-09 at 11 45 12 Screenshot 2026-04-17 at 15 06 14
Was generative AI tooling used to co-author this PR?
  • [Cursor] Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@pierrejeambrun
Copy link
Copy Markdown
Member Author

Merge backend first.

@pierrejeambrun pierrejeambrun marked this pull request as draft April 9, 2026 12:08
@kaxil kaxil requested a review from Copilot April 10, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds cursor-based pagination for TaskInstances to avoid expensive COUNT(*) queries and speeds up the default listing UI, while keeping offset pagination for existing endpoints/actions.

Changes:

  • Introduces cursor pagination helpers + API support (cursor, next_cursor, previous_cursor) and response model discrimination via pagination.
  • Updates TaskInstances UI to use cursor pagination and extends DataTable with an optional cursorPagination control.
  • Regenerates OpenAPI client types/services and updates unit tests for the new pagination mode.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
airflow-ctl/src/airflowctl/api/datamodels/generated.py Adds separate cursor/offset task instance collection response models in generated client datamodels.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_task_instances.py Adds cursor pagination tests and updates existing assertions to include pagination: "offset".
airflow-core/tests/unit/api_fastapi/common/test_parameters.py Reformats imports to multi-line for readability.
airflow-core/tests/unit/api_fastapi/common/test_cursors.py Adds unit tests for cursor encode/decode and keyset filter behavior.
airflow-core/src/airflow/ui/src/queries/useClearTaskInstances.ts Updates types to use the offset response model for clear operations.
airflow-core/src/airflow/ui/src/queries/useClearDagRunDryRun.ts Updates dry-run clear typing to offset response model.
airflow-core/src/airflow/ui/src/queries/useBulkMarkAsDryRun.ts Updates response typing and EMPTY constant to offset model.
airflow-core/src/airflow/ui/src/queries/useBulkClearDryRun.ts Updates response typing and EMPTY constant to offset model.
airflow-core/src/airflow/ui/src/pages/TaskInstances/TaskInstances.tsx Switches listing to cursor pagination and wires next/previous into DataTable.
airflow-core/src/airflow/ui/src/pages/Task/Overview/Overview.tsx Makes failed-task count robust to unioned cursor/offset responses.
airflow-core/src/airflow/ui/src/pages/Dag/Overview/Overview.tsx Makes failed-task count robust to unioned cursor/offset responses.
airflow-core/src/airflow/ui/src/pages/Dag/Overview/FailedLogs.tsx Broadens prop type to accept unioned task instance listing response.
airflow-core/src/airflow/ui/src/components/DataTable/index.ts Re-exports cursor pagination prop type.
airflow-core/src/airflow/ui/src/components/DataTable/DataTable.tsx Adds cursorPagination support with prev/next icon buttons and hides total row count in cursor mode.
airflow-core/src/airflow/ui/src/components/ActionAccordion/ActionAccordion.tsx Updates affected tasks typing to offset response model.
airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Introduces cursor/offset response types and unions GetTaskInstancesResponse.
airflow-core/src/airflow/ui/openapi-gen/requests/services.gen.ts Adds cursor query param and updates docs/return types for updated task instance endpoints.
airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts Adds OpenAPI schemas for cursor/offset collection responses.
airflow-core/src/airflow/ui/openapi-gen/queries/suspense.ts Updates generated query hook typing/docs for task instance endpoints with cursor support.
airflow-core/src/airflow/ui/openapi-gen/queries/queries.ts Updates generated query hook typing/docs for task instance endpoints with cursor support.
airflow-core/src/airflow/ui/openapi-gen/queries/prefetch.ts Updates generated prefetch helpers for task instance endpoints with cursor support.
airflow-core/src/airflow/ui/openapi-gen/queries/ensureQueryData.ts Updates generated ensureQueryData helpers for task instance endpoints with cursor support.
airflow-core/src/airflow/ui/openapi-gen/queries/common.ts Includes cursor in task instance query keys for cache separation.
airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py Implements cursor pagination path and returns discriminated cursor/offset response models.
airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py Updates clear DAG run response to offset pagination model.
airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml Adds cursor query param + oneOf cursor/offset response schema with discriminator mapping.
airflow-core/src/airflow/api_fastapi/core_api/datamodels/task_instances.py Splits cursor/offset responses and defines discriminated union response type.
airflow-core/src/airflow/api_fastapi/core_api/datamodels/common.py Adds base models for offset/cursor paginated responses.
airflow-core/src/airflow/api_fastapi/common/parameters.py Adds cached resolution to SortParam and exposes resolved columns for cursor encoding/filtering.
airflow-core/src/airflow/api_fastapi/common/cursors.py Adds cursor token encoding/decoding and keyset WHERE clause builder.

Comment thread airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py Outdated
Comment thread airflow-core/src/airflow/api_fastapi/common/cursors.py Outdated
Comment thread airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Outdated
Comment thread airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Outdated
Comment thread airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Outdated
Comment thread airflow-core/src/airflow/ui/openapi-gen/requests/services.gen.ts Outdated
@pierrejeambrun pierrejeambrun force-pushed the feature/cursor-pagination-task-instances-ui branch from a5135d0 to a7f9fdd Compare April 17, 2026 09:16
@pierrejeambrun pierrejeambrun marked this pull request as ready for review April 17, 2026 09:17
Replace offset-based pagination with cursor-based pagination
for the TaskInstances listing page, leveraging the new cursor
API endpoint. Pagination now shows only previous/next buttons
without page numbers or total count, which eliminates the
expensive COUNT(*) query for large datasets.

Add generic cursor pagination support to DataTable via an
optional cursorPagination prop so other tables can adopt it.
@pierrejeambrun pierrejeambrun force-pushed the feature/cursor-pagination-task-instances-ui branch from a7f9fdd to c2a34e6 Compare April 17, 2026 09:52
@pierrejeambrun pierrejeambrun force-pushed the feature/cursor-pagination-task-instances-ui branch 3 times, most recently from 9a9ebb5 to 57c751e Compare April 17, 2026 13:04
@pierrejeambrun pierrejeambrun force-pushed the feature/cursor-pagination-task-instances-ui branch 2 times, most recently from 8b8e080 to c756e41 Compare April 17, 2026 13:47
@pierrejeambrun pierrejeambrun force-pushed the feature/cursor-pagination-task-instances-ui branch from c756e41 to 674dbdb Compare April 17, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:airflow-ctl area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants