feat(import[gitlab]): add --with-shared and --skip-group flags (#519)

tony · web-flow · commit 13818a272e5f · 2026-02-22T19:01:18.000-06:00
## Summary - **`--with-shared`** — controls whether GitLab's `/groups/:id/projects` endpoint returns projects that have been shared into the group from other namespaces. GitLab's API default is `with_shared=true`; vcspull now sends `with_shared=false` by default so only the group's own repos appear, with `--with-shared` to opt back in. - **`--skip-group GROUP`** — client-side filter that excludes all repositories whose owner namespace path contains `GROUP` as a path segment (case-insensitive). Repeatable: `--skip-group bots --skip-group archived`. Matches `my-group/bots` and `my-group/bots/subteam` but not `my-group/robotics`. Both flags are GitLab-only; all other service importers (GitHub, Gitea, CodeCommit, Codeberg, Forgejo) are unaffected — new params carry safe defaults. ## What changed | Layer | File | Change | |-------|------|--------| | Model | `_internal/remotes/base.py` | `ImportOptions`: new `with_shared: bool = False` and `skip_groups: list[str]` fields; `filter_repo()`: segment-matching skip guard before min-stars check | | API | `_internal/remotes/gitlab.py` | `_paginate_repos()`: emit `with_shared=true/false` inside `if include_subgroups:` block (group mode only) | | CLI | `cli/import_cmd/gitlab.py` | `--with-shared` (`store_true`) and `--skip-group` (`action=append`, `default=None`) | | Plumbing | `cli/import_cmd/_common.py` | `_run_import()`: new `with_shared`/`skip_groups` kwargs; normalises `None → []` before `ImportOptions` construction | | Docs | `docs/cli/import/gitlab.md` | "Including shared repositories" and "Skipping subgroups" sections | **GitLab API reference:** - [`GET /groups/:id/projects`](https://docs.gitlab.com/api/groups/) — `with_shared` (boolean, optional, **default `true`**) - [Sharing projects and groups](https://docs.gitlab.com/user/project/members/sharing_projects_groups/) ## Test plan ### Automated (run with `uv run pytest`) All 1158 existing + new tests pass. New tests added: #### URL parameter verification (`tests/_internal/remotes/test_gitlab.py`) These tests monkeypatch `urllib.request.urlopen`, capture the actual URL sent to the GitLab API, and assert using `urllib.parse.parse_qs` (not substring search, so query-string ordering cannot cause false failures). - [ ] `test_gitlab_with_shared_false_by_default` — group mode, no flag → `parse_qs` confirms `with_shared=["false"]` in the request URL - [ ] `test_gitlab_with_shared_true_when_flag_set` — group mode, `ImportOptions(with_shared=True)` → `parse_qs` confirms `with_shared=["true"]` - [ ] `test_gitlab_with_shared_not_sent_in_user_mode` — user mode, `with_shared=True` in options (should be ignored) → `with_shared` key must be **absent** from the URL (GitLab `/users/:id/projects` does not accept this param) #### Filter integration (`tests/_internal/remotes/test_gitlab.py`) These tests inject a multi-repo mock response and assert which repos survive after the full `filter_repo()` pass inside `fetch_repos()`. - [ ] `test_gitlab_skip_groups_filters_repos` — response contains repos in `testgroup`, `testgroup/bots`, `testgroup/bots/subteam`; `skip_groups=["bots"]` → only the `testgroup` repo survives (segment match eliminates both bots tiers) - [ ] `test_gitlab_skip_groups_case_insensitive` — repo owner is `ORG/Bots`, `skip_groups=["bots"]` (lowercase) → repo is excluded (case-insensitive match) #### CLI parser roundtrip (`tests/cli/test_import_repos.py`) - [ ] `test_import_with_shared_flag_via_cli` — `parser.parse_args([..., "--with-shared"])` → `args.with_shared is True` - [ ] `test_import_skip_group_flag_via_cli` — `parser.parse_args([..., "--skip-group", "bots", "--skip-group", "archived"])` → `args.skip_groups == ["bots", "archived"]` #### End-to-end plumbing (`tests/cli/test_import_repos.py`) - [ ] `test_run_import_forwards_with_shared_and_skip_groups` — uses `CapturingMockImporter` (new helper) to record the `ImportOptions` object that `_run_import()` constructs and passes to `fetch_repos()`; asserts `opts.with_shared is True` and `opts.skip_groups == ["bots", "archived"]` ### Manual smoke tests (requires `GITLAB_TOKEN` with `api` scope) **E2E fixture setup (one-time):** `tony/external-shared-repo` (public) has been shared into `vcs-python-group-test` at Reporter (20) access. This makes `--with-shared` observably different from the default. The group now has 15 own repos + 1 externally-shared repo visible with `--with-shared`. All 5 permutations verified on 2026-02-21: | Flags | Expected | Actual | Status | |-------|----------|--------|--------| | (none) | 15 own repos | ✓ 15 | ✅ | | `--with-shared` | 15 + 1 shared | ✓ 16 | ✅ | | `--skip-group vcs-python-subgroup-test` | top-level only | ✓ 5 | ✅ | | `--with-shared --skip-group vcs-python-subgroup-test` | top-level + shared | ✓ 6 | ✅ | | `--skip-group subgroup` (segment guard) | all 15 | ✓ 15 | ✅ | The combined case (row 4) is the key proof: `skip_groups` does **not** accidentally drop `external-shared-repo` (whose namespace is `tony`, with no overlap with `vcs-python-subgroup-test`). **`--with-shared` default (shared repos excluded):** ```console $ vcspull import gl vcs-python-group-test \ --mode org \ --workspace /tmp/test-import \ --dry-run \ --yes ``` Expected: 15 repos (group's own repos only; `external-shared-repo` absent). **`--with-shared` enabled:** ```console $ vcspull import gl vcs-python-group-test \ --mode org \ --workspace /tmp/test-import \ --with-shared \ --dry-run \ --yes ``` Expected: 16 repos (`external-shared-repo` from `tony` namespace appears). **`--skip-group` skipping the subgroup:** ```console $ vcspull import gl vcs-python-group-test \ --mode org \ --workspace /tmp/test-import \ --skip-group vcs-python-subgroup-test \ --dry-run \ --yes ``` Expected: only top-level repos (5 repos). `sub-test-repo-{1-4}` and `subsub-test-repo-{1-4}` must be absent. **`--skip-group` skipping only the sub-subgroup:** ```console $ vcspull import gl vcs-python-group-test \ --mode org \ --workspace /tmp/test-import \ --skip-group vcs-python-subsubgroup-test \ --dry-run \ --yes ``` Expected: top-level + subgroup repos. `subsub-test-repo-{1-4}` must be absent. **Segment-not-substring guard:** ```console $ vcspull import gl vcs-python-group-test \ --mode org \ --workspace /tmp/test-import \ --skip-group subgroup \ --dry-run \ --yes ``` Expected: all 15 own repos returned — `vcs-python-subgroup-test` contains the string "subgroup" but the segment is `vcs-python-subgroup-test`, not `subgroup`, so no match. **`--with-shared` has no effect in user mode:** ```console $ vcspull import gl <your-gitlab-username> \ --mode user \ --workspace /tmp/test-import \ --with-shared \ --dry-run \ --yes ``` Expected: same result as without `--with-shared` (param not sent to API).
diff --git a/CHANGES b/CHANGES
@@ -33,6 +33,25 @@ $ uvx --from 'vcspull' --prerelease allow vcspull
 _Notes on upcoming releases will be added here_
 <!-- END PLACEHOLDER - ADD NEW CHANGELOG ENTRIES BELOW THIS LINE -->
 
+### Breaking changes
+
+#### `vcspull import gitlab`: `--with-shared` now defaults to excluded
+
+Prior to this release, GitLab's group API returned shared projects by default.
+vcspull now explicitly requests `with_shared=false`, so shared repositories are
+excluded unless `--with-shared` is passed.
+
+### New features
+
+#### `vcspull import gitlab`: Add `--with-shared` and `--skip-group` flags
+
+- `--with-shared`: Opt-in flag to include projects shared into a group from
+  another namespace. Only applies in `--mode org`; a warning is emitted when
+  used in user or search modes.
+- `--skip-group <name>`: Client-side filter that excludes all repositories
+  whose owner path contains the specified group name segment (case-insensitive,
+  segment-based). Repeat the flag to skip multiple groups.
+
 ## vcspull v1.56.1 (2026-02-18)
 
 ### Bug fixes
diff --git a/docs/cli/import/gitlab.md b/docs/cli/import/gitlab.md
@@ -61,6 +61,50 @@ Given a group tree `my-group → sub → leaf`, importing from `~/code/`:
 When the target is already the deepest group (a leaf), `--flatten-groups` has
 no effect — all repositories already land in the base workspace.
 
+## Including shared repositories
+
+By default, repositories shared into a group from another namespace are
+excluded. Use `--with-shared` to include them when importing in `--mode org`:
+
+```console
+$ vcspull import gl my-group \
+    --mode org \
+    --workspace ~/code/ \
+    --with-shared
+```
+
+`--with-shared` has no effect in user mode or search mode; a warning is
+emitted if the flag is passed in those contexts.
+
+## Skipping subgroups
+
+Use `--skip-group` to exclude all repositories whose owner path contains a
+specific group name segment. Matching is case-insensitive and segment-based:
+
+```console
+$ vcspull import gl my-group \
+    --mode org \
+    --workspace ~/code/ \
+    --skip-group bots
+```
+
+Repeat the flag to skip multiple groups:
+
+```console
+$ vcspull import gl my-group \
+    --mode org \
+    --workspace ~/code/ \
+    --skip-group bots \
+    --skip-group archived
+```
+
+The flag matches any path segment: `--skip-group bots` skips repos owned by
+`my-group/bots` or `my-group/bots/subteam` but not `my-group/robotics`.
+
+> **Note**: Passing the root group name itself (`--skip-group my-org` when
+> importing `my-org`) will silently exclude every repository, since all repo
+> paths include the root group as a path segment.
+
 ## Authentication
 
 - **Env vars**: `GITLAB_TOKEN` (primary), `GL_TOKEN` (fallback)
diff --git a/src/vcspull/_internal/remotes/base.py b/src/vcspull/_internal/remotes/base.py
@@ -232,6 +232,15 @@ class ImportOptions:
         Minimum star count (for search mode)
     limit : int
         Maximum number of repositories to return
+    with_shared : bool
+        Include projects shared into a group from other namespaces
+        (GitLab group mode only; default: False)
+    skip_groups : list[str]
+        Exclude repos whose owner path contains any of these group name
+        segments (case-insensitive segment matching). For flat-namespace
+        hosts (e.g. GitHub orgs, which have a single path segment), passing
+        the org name itself as a skip value would exclude every repository
+        under that org.
     """
 
     mode: ImportMode = ImportMode.USER
@@ -244,6 +253,8 @@ class ImportOptions:
     topics: list[str] = dataclasses.field(default_factory=list)
     min_stars: int = 0
     limit: int = 100
+    with_shared: bool = False
+    skip_groups: list[str] = dataclasses.field(default_factory=list)
 
     def __post_init__(self) -> None:
         """Validate options after initialization.
@@ -263,6 +274,18 @@ def __post_init__(self) -> None:
         Traceback (most recent call last):
             ...
         ValueError: limit must be >= 0, got -1
+
+        >>> opts = ImportOptions(with_shared=True)
+        >>> opts.with_shared
+        True
+
+        >>> opts = ImportOptions(skip_groups=["bots", "archived"])
+        >>> opts.skip_groups
+        ['bots', 'archived']
+
+        >>> opts = ImportOptions()
+        >>> opts.skip_groups
+        []
         """
         if self.limit < 0:
             msg = f"limit must be >= 0, got {self.limit}"
@@ -606,6 +629,37 @@ def filter_repo(
     >>> options = ImportOptions(language="JavaScript")
     >>> filter_repo(repo, options)
     False
+
+    >>> options = ImportOptions(skip_groups=["user"])
+    >>> filter_repo(repo, options)
+    False
+
+    >>> options = ImportOptions(skip_groups=["other-group"])
+    >>> filter_repo(repo, options)
+    True
+
+    >>> repo_sub = RemoteRepo(
+    ...     name="sub-project",
+    ...     clone_url="https://github.com/org/sub/sub-project.git",
+    ...     ssh_url="git@github.com:org/sub/sub-project.git",
+    ...     html_url="https://github.com/org/sub/sub-project",
+    ...     description=None,
+    ...     language=None,
+    ...     topics=(),
+    ...     stars=0,
+    ...     is_fork=False,
+    ...     is_archived=False,
+    ...     default_branch="main",
+    ...     owner="org/sub",
+    ... )
+    >>> filter_repo(repo_sub, ImportOptions(skip_groups=["sub"]))
+    False
+    >>> filter_repo(repo_sub, ImportOptions(skip_groups=["org"]))
+    False
+    >>> filter_repo(repo_sub, ImportOptions(skip_groups=["unrelated"]))
+    True
+    >>> filter_repo(repo_sub, ImportOptions(skip_groups=["SUB"]))
+    False
     """
     # Check fork filter
     if repo.is_fork and not options.include_forks:
@@ -628,5 +682,12 @@ def filter_repo(
         if not required_topics_lower.issubset(repo_topics_lower):
             return False
 
+    # Check skip_groups: exclude if any owner path segment matches
+    if options.skip_groups:
+        owner_segments = {s.lower() for s in repo.owner.split("/") if s}
+        skip_set = {g.lower() for g in options.skip_groups if g}
+        if owner_segments & skip_set:
+            return False
+
     # Check minimum stars
     return not (options.min_stars > 0 and repo.stars < options.min_stars)
diff --git a/src/vcspull/_internal/remotes/gitlab.py b/src/vcspull/_internal/remotes/gitlab.py
@@ -241,7 +241,18 @@ def _fetch_search(self, options: ImportOptions) -> t.Iterator[RemoteRepo]:
             page += 1
 
         # Warn if results were truncated by --limit
-        self._warn_truncation(count, options.limit, total_available, last_x_next_page)
+        self._warn_truncation(
+            count,
+            options.limit,
+            total_available,
+            last_x_next_page,
+            has_client_filters=bool(
+                options.skip_groups
+                or options.language
+                or options.topics
+                or options.min_stars > 0
+            ),
+        )
 
     def _paginate_repos(
         self,
@@ -283,6 +294,7 @@ def _paginate_repos(
 
             if include_subgroups:
                 params["include_subgroups"] = "true"
+                params["with_shared"] = "true" if options.with_shared else "false"
 
             if not options.include_archived:
                 params["archived"] = "false"
@@ -324,37 +336,71 @@ def _paginate_repos(
             page += 1
 
         # Warn if results were truncated by --limit
-        self._warn_truncation(count, options.limit, total_available, last_x_next_page)
+        self._warn_truncation(
+            count,
+            options.limit,
+            total_available,
+            last_x_next_page,
+            has_client_filters=bool(
+                options.skip_groups
+                or options.language
+                or options.topics
+                or options.min_stars > 0
+            ),
+        )
 
     def _warn_truncation(
         self,
         count: int,
         limit: int,
         total_available: int | None,
         x_next_page: str | None,
+        *,
+        has_client_filters: bool = False,
     ) -> None:
         """Warn if results were truncated by the --limit option.
 
         Parameters
         ----------
         count : int
-            Number of repositories actually returned
+            Number of repositories returned after client-side filtering
         limit : int
             The configured limit
         total_available : int | None
-            Value of x-total header (None if absent)
+            Value of x-total header (None if absent). Reflects the unfiltered
+            server-side total; may overstate the count of repos matching
+            client-side filters (skip_groups, language, topics).
         x_next_page : str | None
             Value of x-next-page header (None if absent/empty)
+        has_client_filters : bool
+            True when any client-side filter (skip_groups, language, topics,
+            min_stars) is active; suppresses the early ``count < limit`` return
+            so the qualified warning can fire even when count < limit.
         """
-        if count < limit:
+        # Without client-side filters, count < limit means we consumed all
+        # server results without hitting the limit — no truncation to warn about.
+        # With active filters, skip the early return: count < limit may simply
+        # reflect that filters excluded repos, while total_available still
+        # exceeds count and is worth surfacing to the user.
+        if count < limit and not has_client_filters:
             return
 
         if total_available is not None and total_available > count:
-            log.warning(
-                "Showing %d of %d repositories (use --limit 0 to fetch all)",
-                count,
-                total_available,
-            )
+            if has_client_filters:
+                log.warning(
+                    "Showing %d repositories; %d total on server "
+                    "(note: server total includes projects excluded by "
+                    "--skip-group / --language / --topics / --min-stars; "
+                    "use --limit 0 to fetch all matching your filters)",
+                    count,
+                    total_available,
+                )
+            else:
+                log.warning(
+                    "Showing %d of %d repositories (use --limit 0 to fetch all)",
+                    count,
+                    total_available,
+                )
         elif x_next_page is not None:
             log.warning(
                 "Showing %d repositories; more are available "
diff --git a/src/vcspull/cli/import_cmd/_common.py b/src/vcspull/cli/import_cmd/_common.py
@@ -281,6 +281,8 @@ def _run_import(
     color: str,
     use_https: bool = False,
     flatten_groups: bool = False,
+    with_shared: bool = False,
+    skip_groups: list[str] | None = None,
 ) -> int:
     """Run the import workflow for a single service.
 
@@ -329,6 +331,10 @@ def _run_import(
         Use HTTPS clone URLs instead of SSH (default: False, i.e., SSH)
     flatten_groups : bool
         For GitLab org imports, flatten subgroup paths into base workspace
+    with_shared : bool
+        Include projects shared into a group from other namespaces (GitLab only)
+    skip_groups : list[str] | None
+        Exclude repos whose owner path contains any of these group name segments
 
     Returns
     -------
@@ -347,6 +353,21 @@ def _run_import(
         else []
     )
 
+    skip_groups_list: list[str] = skip_groups or []
+
+    # Validate skip_groups: each value must be a single path segment.
+    # A slash inside a value like "bots/subteam" would never match any
+    # owner segment (filter_repo splits repo.owner on "/" and compares
+    # individual parts), so it silently has no effect.
+    for group in skip_groups_list:
+        if "/" in group:
+            log.error(
+                "--skip-group values must be single path segments; "
+                "'/' is not allowed: %r",
+                group,
+            )
+            return 1
+
     try:
         options = ImportOptions(
             mode=import_mode,
@@ -357,6 +378,8 @@ def _run_import(
             topics=topic_list,
             min_stars=min_stars,
             limit=limit,
+            with_shared=with_shared,
+            skip_groups=skip_groups_list,
         )
     except ValueError as exc_:
         log.error("%s %s", colors.error("✗"), exc_)  # noqa: TRY400
@@ -384,6 +407,16 @@ def _run_import(
             colors.warning("!"),
             importer.service_name,
         )
+    if (
+        options.with_shared
+        and service_name == "gitlab"
+        and options.mode != ImportMode.ORG
+    ):
+        log.warning(
+            "%s --with-shared has no effect outside org mode; "
+            "shared projects are only available via the GitLab group API",
+            colors.warning("!"),
+        )
 
     # Resolve workspace path
     workspace_path = pathlib.Path(workspace).expanduser().resolve()
diff --git a/src/vcspull/cli/import_cmd/gitlab.py b/src/vcspull/cli/import_cmd/gitlab.py
@@ -54,6 +54,27 @@ def create_gitlab_subparser(
             "workspace instead of preserving subgroup paths"
         ),
     )
+    parser.add_argument(
+        "--with-shared",
+        action="store_true",
+        dest="with_shared",
+        default=False,
+        help=(
+            "Include projects shared into the group from other namespaces "
+            "(default: exclude shared projects)"
+        ),
+    )
+    parser.add_argument(
+        "--skip-group",
+        action="append",
+        dest="skip_groups",
+        metavar="GROUP",
+        default=None,
+        help=(
+            "Exclude repos whose namespace contains GROUP as a path segment "
+            "(repeatable: --skip-group bots --skip-group archived)"
+        ),
+    )
     parser.set_defaults(import_handler=handle_gitlab)
 
 
@@ -98,4 +119,6 @@ def handle_gitlab(args: argparse.Namespace) -> int:
         color=getattr(args, "color", "auto"),
         use_https=getattr(args, "use_https", False),
         flatten_groups=getattr(args, "flatten_groups", False),
+        with_shared=getattr(args, "with_shared", False),
+        skip_groups=getattr(args, "skip_groups", None),
     )
diff --git a/tests/_internal/remotes/test_gitlab.py b/tests/_internal/remotes/test_gitlab.py
diff --git a/tests/cli/test_import_repos.py b/tests/cli/test_import_repos.py
