Skip to content

Commit fae3714

Browse files
authored
Merge pull request #825 from atlanhq/BLDX-689
BLDX-689 | Fixed docs, generator and dependencies updates
2 parents de441ba + fa475fc commit fae3714

592 files changed

Lines changed: 15210 additions & 7968 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
---
2+
name: upgrade-deps
3+
description: Upgrade all pyatlan Python dependencies and GitHub Actions to their latest compatible versions
4+
disable-model-invocation: true
5+
allowed-tools: Read, Bash, Edit, Write, Glob, Grep, WebFetch
6+
---
7+
8+
Upgrade all dependencies for the pyatlan Python SDK. Follow these steps carefully.
9+
10+
## Context
11+
12+
- The project supports Python >=3.9, so skip any package version that requires Python >=3.10 or higher
13+
- Use `uv` for all Python package operations
14+
- Workflow files are in `.github/workflows/`
15+
- Dependencies are managed in `pyproject.toml` and locked in `uv.lock`
16+
17+
---
18+
19+
## Step 1 — Find outdated Python packages
20+
21+
Run:
22+
```
23+
uv pip list --outdated
24+
```
25+
26+
Cross-reference the output with the direct dependencies pinned in `pyproject.toml` (sections: `dependencies`, `dev`, `docs`). Ignore transitive dependencies — only update packages that are explicitly listed in `pyproject.toml`.
27+
28+
---
29+
30+
## Step 2 — Check Python version compatibility for each outdated package
31+
32+
For every package identified in Step 1, check its `requires_python` field:
33+
34+
```
35+
curl -s https://pypi.org/pypi/<package>/<new-version>/json | python3 -c "import json,sys; d=json.load(sys.stdin); print(d['info']['requires_python'])"
36+
```
37+
38+
**Rules:**
39+
- If `requires_python` is `>=3.9` or lower (or None) → safe to upgrade
40+
- If `requires_python` is `>=3.10` or higher → check if an older compatible version exists:
41+
```
42+
curl -s https://pypi.org/pypi/<package>/json | python3 -c "
43+
import json, sys
44+
d = json.load(sys.stdin)
45+
for v in sorted(d['releases'].keys(), reverse=True)[:10]:
46+
rels = d['releases'][v]
47+
if rels:
48+
rp = rels[0].get('requires_python','')
49+
print(f'{v}: {rp}')
50+
"
51+
```
52+
Pick the highest version that supports Python 3.9. If already at that version, skip it.
53+
- Packages already conditioned in pyproject.toml with `; python_version >= '3.10'` (like `filelock`) can be upgraded freely since they only install on 3.10+
54+
55+
---
56+
57+
## Step 3 — Update pyproject.toml
58+
59+
For each package that can be safely upgraded, update its version pin in `pyproject.toml`.
60+
61+
Keep the same `~=X.Y.Z` pinning style as used by other packages in the file.
62+
63+
---
64+
65+
## Step 4 — Upgrade GitHub Actions
66+
67+
Scan all workflow files in `.github/workflows/` for `uses:` lines:
68+
```
69+
grep -rh "uses:" .github/workflows/ | sort -u
70+
```
71+
72+
For each action that uses a version tag, check the latest release:
73+
```
74+
curl -s https://api.github.com/repos/<owner>/<repo>/releases/latest | python3 -c "import json,sys; print(json.load(sys.stdin)['tag_name'])"
75+
```
76+
77+
Use the major version tag style (e.g., `v7`) if the action uses that convention, or the exact version if it was pinned exactly (e.g., `0.34.2`).
78+
79+
Update all workflow files using sed in-place replacements for each changed action.
80+
81+
---
82+
83+
## Step 5 — Update pre-commit ruff rev
84+
85+
If ruff was upgraded in pyproject.toml, also update the `rev:` in `.pre-commit-config.yaml` to match:
86+
```yaml
87+
- repo: https://github.com/astral-sh/ruff-pre-commit
88+
rev: v<new-version>
89+
```
90+
91+
---
92+
93+
## Step 6 — Regenerate the lockfile
94+
95+
```
96+
uv lock --upgrade
97+
```
98+
99+
If this fails due to a Python version conflict, revisit Step 2 for the failing package.
100+
101+
---
102+
103+
## Step 7 — Install and run tests
104+
105+
```
106+
uv sync --all-groups
107+
python -m pytest tests/unit/ -x -q --tb=short
108+
```
109+
110+
Fix any test failures caused by the upgrades before proceeding.
111+
112+
---
113+
114+
## Step 8 — Run pre-commit
115+
116+
```
117+
pre-commit run --all-files
118+
```
119+
120+
The first run may auto-fix formatting. Run again to confirm all hooks pass.
121+
122+
---
123+
124+
## Step 9 — Summarise and commit
125+
126+
Present a summary table of all changes made:
127+
128+
| Package / Action | Before | After | Notes |
129+
|---|---|---|---|
130+
| pydantic | 2.12.4 | 2.12.5 | |
131+
| ... | | | |
132+
| docker/login-action | v3 | v4 | |
133+
134+
Then commit:
135+
```
136+
git add pyproject.toml uv.lock requirements.txt .pre-commit-config.yaml .github/workflows/
137+
git commit -m "chore: upgrade Python dependencies and GitHub Actions to latest versions"
138+
```
139+
140+
Ask the user before pushing.

.github/workflows/pyatlan-chainguard.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -48,24 +48,24 @@ jobs:
4848
fetch-depth: 0 # Need full history for commit hash
4949

5050
- name: Set up Docker Buildx
51-
uses: docker/setup-buildx-action@v3
51+
uses: docker/setup-buildx-action@v4
5252

5353
- name: Log in to GitHub Container Registry
54-
uses: docker/login-action@v3
54+
uses: docker/login-action@v4
5555
with:
5656
registry: ghcr.io
5757
username: ${{ github.actor }}
5858
password: ${{ secrets.GITHUB_TOKEN }}
5959

6060
- name: Log in to Atlan Harbor Registry
61-
uses: docker/login-action@v3
61+
uses: docker/login-action@v4
6262
with:
6363
registry: registry.atlan.com
6464
username: ${{ secrets.HARBOR_USERNAME }}
6565
password: ${{ secrets.HARBOR_PASSWORD }}
6666

6767
- name: Log in to Chainguard Container Registry
68-
uses: docker/login-action@v3
68+
uses: docker/login-action@v4
6969
with:
7070
registry: cgr.dev
7171
username: ${{ secrets.CHAINGUARD_USERNAME }}
@@ -172,7 +172,7 @@ jobs:
172172
echo "Generated image tag: ghcr.io/atlanhq/pyatlan-chainguard-base:$IMAGE_TAG"
173173
174174
- name: Build and push Docker image
175-
uses: docker/build-push-action@v6
175+
uses: docker/build-push-action@v7
176176
with:
177177
context: .
178178
file: ./Dockerfile
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# SPDX-License-Identifier: Apache-2.0
22
# Copyright 2023 Atlan Pte. Ltd.
3-
name: Pyatlan Sphinx Docs Build
3+
name: Pyatlan MkDocs Build
44

55
on:
66
push:
@@ -10,7 +10,7 @@ on:
1010
jobs:
1111
publish-docs:
1212
runs-on: ubuntu-latest
13-
name: "Sphinx"
13+
name: "MkDocs"
1414
steps:
1515
- name: Checkout
1616
uses: actions/checkout@v4
@@ -21,16 +21,16 @@ jobs:
2121
python-version: "3.12"
2222

2323
- name: Install uv
24-
uses: astral-sh/setup-uv@v6
24+
uses: astral-sh/setup-uv@v7
2525

2626
- name: Install dependencies
2727
run: uv sync --group docs
2828

29-
- name: Build Sphinx docs
30-
run: uv run sphinx-build -b html docs docs/_build
29+
- name: Build MkDocs
30+
run: uv run mkdocs build
3131

3232
- name: Publish to GitHub Pages
3333
uses: peaceiris/actions-gh-pages@v4
3434
with:
3535
github_token: ${{ secrets.GITHUB_TOKEN }}
36-
publish_dir: docs/_build
36+
publish_dir: site

.github/workflows/pyatlan-pr.yaml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ jobs:
6767
python-version: ${{ matrix.python-version }}
6868

6969
- name: Install uv
70-
uses: astral-sh/setup-uv@v6
70+
uses: astral-sh/setup-uv@v7
7171

7272
- uses: pypa/gh-action-pip-audit@v1.1.0
7373
with:
@@ -161,7 +161,7 @@ jobs:
161161
python-version: ${{ matrix.python-version }}
162162

163163
- name: Install uv
164-
uses: astral-sh/setup-uv@v6
164+
uses: astral-sh/setup-uv@v7
165165

166166
- name: Install dependencies
167167
run: uv sync --group dev
@@ -223,7 +223,7 @@ jobs:
223223
python-version: "3.9"
224224

225225
- name: Install uv
226-
uses: astral-sh/setup-uv@v6
226+
uses: astral-sh/setup-uv@v7
227227

228228
- name: Install dependencies
229229
run: uv sync --group dev
@@ -265,7 +265,7 @@ jobs:
265265
python-version: "3.9"
266266

267267
- name: Install uv
268-
uses: astral-sh/setup-uv@v6
268+
uses: astral-sh/setup-uv@v7
269269

270270
- name: Install dependencies
271271
run: uv sync --group dev
@@ -304,7 +304,7 @@ jobs:
304304
python-version: ${{ matrix.python-version }}
305305

306306
- name: Install uv
307-
uses: astral-sh/setup-uv@v6
307+
uses: astral-sh/setup-uv@v7
308308

309309
- name: Install dependencies
310310
run: uv sync --group dev
@@ -381,7 +381,7 @@ jobs:
381381
python-version: "3.11"
382382

383383
- name: Install uv
384-
uses: astral-sh/setup-uv@v6
384+
uses: astral-sh/setup-uv@v7
385385

386386
- name: Install dependencies
387387
run: uv sync --group dev
@@ -417,7 +417,7 @@ jobs:
417417
python-version: "3.11"
418418

419419
- name: Install uv
420-
uses: astral-sh/setup-uv@v6
420+
uses: astral-sh/setup-uv@v7
421421

422422
- name: Install dependencies
423423
run: uv sync --group dev

.github/workflows/pyatlan-publish.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ jobs:
2828
with:
2929
python-version: '3.9'
3030
- name: Install uv
31-
uses: astral-sh/setup-uv@v6
31+
uses: astral-sh/setup-uv@v7
3232
- name: Install dependencies
3333
run: uv sync --group dev
3434
- name: check tag

.github/workflows/pyatlan-scheduled-scan.yml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,17 +20,17 @@ jobs:
2020
uses: actions/checkout@v4
2121

2222
- name: Set up Docker Buildx
23-
uses: docker/setup-buildx-action@v3
23+
uses: docker/setup-buildx-action@v4
2424

2525
- name: Log in to container registry
26-
uses: docker/login-action@v3
26+
uses: docker/login-action@v4
2727
with:
2828
registry: cgr.dev
2929
username: ${{ secrets.CHAINGUARD_USERNAME }}
3030
password: ${{ secrets.CHAINGUARD_PASSWORD }}
3131

3232
- name: Build image
33-
uses: docker/build-push-action@v6
33+
uses: docker/build-push-action@v7
3434
with:
3535
context: '.'
3636
file: './Dockerfile'
@@ -41,7 +41,7 @@ jobs:
4141
# ── Image scan ──
4242

4343
- name: Trivy image scan (JSON)
44-
uses: aquasecurity/trivy-action@0.33.1
44+
uses: aquasecurity/trivy-action@0.34.2
4545
with:
4646
image-ref: 'pyatlan:trivy-scan'
4747
scanners: 'vuln'
@@ -56,7 +56,7 @@ jobs:
5656
TRIVY_JAVA_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-java-db:1
5757

5858
- name: Trivy image scan (table)
59-
uses: aquasecurity/trivy-action@0.33.1
59+
uses: aquasecurity/trivy-action@0.34.2
6060
with:
6161
image-ref: 'pyatlan:trivy-scan'
6262
scanners: 'vuln'
@@ -73,7 +73,7 @@ jobs:
7373
# ── Dependency scan ──
7474

7575
- name: Trivy dependency scan (JSON)
76-
uses: aquasecurity/trivy-action@0.33.1
76+
uses: aquasecurity/trivy-action@0.34.2
7777
with:
7878
scan-type: fs
7979
input: 'uv.lock'
@@ -88,7 +88,7 @@ jobs:
8888
TRIVY_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-db:2
8989

9090
- name: Trivy dependency scan (table)
91-
uses: aquasecurity/trivy-action@0.33.1
91+
uses: aquasecurity/trivy-action@0.34.2
9292
with:
9393
scan-type: fs
9494
input: 'uv.lock'

.github/workflows/pyatlan-test-cron.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
python-version: ${{ matrix.python-version }}
2222

2323
- name: Install uv
24-
uses: astral-sh/setup-uv@v6
24+
uses: astral-sh/setup-uv@v7
2525

2626
- name: Install dependencies
2727
run: uv sync --group dev

0 commit comments

Comments
 (0)