Feature/cursor pagination task instances UI#64953
Open
pierrejeambrun wants to merge 5 commits intoapache:mainfrom
Open
Feature/cursor pagination task instances UI#64953pierrejeambrun wants to merge 5 commits intoapache:mainfrom
pierrejeambrun wants to merge 5 commits intoapache:mainfrom
Conversation
Member
Author
|
Merge backend first. |
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds cursor-based pagination for TaskInstances to avoid expensive COUNT(*) queries and speeds up the default listing UI, while keeping offset pagination for existing endpoints/actions.
Changes:
- Introduces cursor pagination helpers + API support (
cursor,next_cursor,previous_cursor) and response model discrimination viapagination. - Updates TaskInstances UI to use cursor pagination and extends
DataTablewith an optionalcursorPaginationcontrol. - Regenerates OpenAPI client types/services and updates unit tests for the new pagination mode.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| airflow-ctl/src/airflowctl/api/datamodels/generated.py | Adds separate cursor/offset task instance collection response models in generated client datamodels. |
| airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_task_instances.py | Adds cursor pagination tests and updates existing assertions to include pagination: "offset". |
| airflow-core/tests/unit/api_fastapi/common/test_parameters.py | Reformats imports to multi-line for readability. |
| airflow-core/tests/unit/api_fastapi/common/test_cursors.py | Adds unit tests for cursor encode/decode and keyset filter behavior. |
| airflow-core/src/airflow/ui/src/queries/useClearTaskInstances.ts | Updates types to use the offset response model for clear operations. |
| airflow-core/src/airflow/ui/src/queries/useClearDagRunDryRun.ts | Updates dry-run clear typing to offset response model. |
| airflow-core/src/airflow/ui/src/queries/useBulkMarkAsDryRun.ts | Updates response typing and EMPTY constant to offset model. |
| airflow-core/src/airflow/ui/src/queries/useBulkClearDryRun.ts | Updates response typing and EMPTY constant to offset model. |
| airflow-core/src/airflow/ui/src/pages/TaskInstances/TaskInstances.tsx | Switches listing to cursor pagination and wires next/previous into DataTable. |
| airflow-core/src/airflow/ui/src/pages/Task/Overview/Overview.tsx | Makes failed-task count robust to unioned cursor/offset responses. |
| airflow-core/src/airflow/ui/src/pages/Dag/Overview/Overview.tsx | Makes failed-task count robust to unioned cursor/offset responses. |
| airflow-core/src/airflow/ui/src/pages/Dag/Overview/FailedLogs.tsx | Broadens prop type to accept unioned task instance listing response. |
| airflow-core/src/airflow/ui/src/components/DataTable/index.ts | Re-exports cursor pagination prop type. |
| airflow-core/src/airflow/ui/src/components/DataTable/DataTable.tsx | Adds cursorPagination support with prev/next icon buttons and hides total row count in cursor mode. |
| airflow-core/src/airflow/ui/src/components/ActionAccordion/ActionAccordion.tsx | Updates affected tasks typing to offset response model. |
| airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts | Introduces cursor/offset response types and unions GetTaskInstancesResponse. |
| airflow-core/src/airflow/ui/openapi-gen/requests/services.gen.ts | Adds cursor query param and updates docs/return types for updated task instance endpoints. |
| airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts | Adds OpenAPI schemas for cursor/offset collection responses. |
| airflow-core/src/airflow/ui/openapi-gen/queries/suspense.ts | Updates generated query hook typing/docs for task instance endpoints with cursor support. |
| airflow-core/src/airflow/ui/openapi-gen/queries/queries.ts | Updates generated query hook typing/docs for task instance endpoints with cursor support. |
| airflow-core/src/airflow/ui/openapi-gen/queries/prefetch.ts | Updates generated prefetch helpers for task instance endpoints with cursor support. |
| airflow-core/src/airflow/ui/openapi-gen/queries/ensureQueryData.ts | Updates generated ensureQueryData helpers for task instance endpoints with cursor support. |
| airflow-core/src/airflow/ui/openapi-gen/queries/common.ts | Includes cursor in task instance query keys for cache separation. |
| airflow-core/src/airflow/api_fastapi/core_api/routes/public/task_instances.py | Implements cursor pagination path and returns discriminated cursor/offset response models. |
| airflow-core/src/airflow/api_fastapi/core_api/routes/public/dag_run.py | Updates clear DAG run response to offset pagination model. |
| airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml | Adds cursor query param + oneOf cursor/offset response schema with discriminator mapping. |
| airflow-core/src/airflow/api_fastapi/core_api/datamodels/task_instances.py | Splits cursor/offset responses and defines discriminated union response type. |
| airflow-core/src/airflow/api_fastapi/core_api/datamodels/common.py | Adds base models for offset/cursor paginated responses. |
| airflow-core/src/airflow/api_fastapi/common/parameters.py | Adds cached resolution to SortParam and exposes resolved columns for cursor encoding/filtering. |
| airflow-core/src/airflow/api_fastapi/common/cursors.py | Adds cursor token encoding/decoding and keyset WHERE clause builder. |
1 task
a5135d0 to
a7f9fdd
Compare
Replace offset-based pagination with cursor-based pagination for the TaskInstances listing page, leveraging the new cursor API endpoint. Pagination now shows only previous/next buttons without page numbers or total count, which eliminates the expensive COUNT(*) query for large datasets. Add generic cursor pagination support to DataTable via an optional cursorPagination prop so other tables can adopt it.
a7f9fdd to
c2a34e6
Compare
9a9ebb5 to
57c751e
Compare
8b8e080 to
c756e41
Compare
c756e41 to
674dbdb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace offset-based pagination with cursor-based pagination for the TaskInstances listing page, leveraging the new cursor API endpoint. Pagination now shows only previous/next buttons without page numbers or total count, which eliminates the expensive COUNT(*) query for large datasets.
Add generic cursor pagination support to DataTable via an optional cursorPagination prop so other tables can adopt it.
Building on top of #64845 to plug the UI on this new cursor based pagination. Default table view is much faster.
Responses are much faster. Currenrly it would take ~60 on main to load.
After
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.