Installing pants fails when a dir called `venv` exists and it is a python module

[xkortex]

Describe the bug

Pants fails with a rather confusing traceback if python -m venv cannot run, for example if you happen to have venv/__init__.py for whatever reason, or python environment is a superfund site in general.

Reproduction:

1. Install pants bootstrap with either get-pants.sh or brew install pantsbuild/tap/pants
2. Remove all pants caches
3. Create a blank dir and cd into it
4. Create venv/__init__.py to create a python module named venv
5. Create a basic pants.toml
6. Run pants

rm -rf ~/Library/Caches/nce
mkdir /tmp/repro_pants && cd /tmp/repro_pants 
mkdir -p venv
touch venv/__init__.py
printf '[GLOBAL]\npants_version = "2.31.0"\n' > pants.toml
pants --version

This happens because the command is effectively python -m venv, which sees the local dir named venv with init, tries to execute that (in fact if you put tracing code in the init you can see it is being invoked), which of course fails to create a real venv, and everything else explodes. If I run that by itself, I get the error /usr/local/bin/python: No module named venv.__main__; 'venv' is a package and cannot be directly executed, but this error is never surfaced.

It took me a while and some LLM rubber ducky to figure out, and only clicked when it asked me to run /root/.cache/nce/110...58/cpython-3.9.23+20250612-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9 -m venv and I saw the above error and realized I had ended up in a weird state with my directories (likely due to Pycharm wanting to create __init__.py during refactors). I know that is a weird corner case, but I think at the very least, surfacing the the error message caused by _run_code would be helpful.

Actually this looks like it might in fact be a Pex bug, so I think I'll open an issue over there.
Fails in the vicinity of this:
https://github.com/pex-tool/pex/blob/4028787e177c1ea1ebcfbd86e801e38eb03c63f5/pex/venv/virtualenv.py#L317

Traceback (see details)

Details

Traceback (most recent call last): File "/root/.cache/nce/110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358/cpython-3.9.23+20250612-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/root/.cache/nce/110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358/cpython-3.9.23+20250612-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/__main__.py", line 233, in result, should_exit, is_globals = boot( File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/__main__.py", line 225, in boot result = bootstrap_pex( File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/pex_bootstrapper.py", line 673, in bootstrap_pex venv_pex = _bootstrap_venv(entry_point, interpreter=target) File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/pex_bootstrapper.py", line 740, in _bootstrap_venv return ensure_venv(pex.PEX(entry_point, interpreter=interpreter)) File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/pex_bootstrapper.py", line 533, in ensure_venv virtualenv = Virtualenv.create_atomic( File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/venv/virtualenv.py", line 349, in create_atomic virtualenv = cls.create( File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/venv/virtualenv.py", line 315, in create interpreter.execute(args=args, env=env) File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/interpreter.py", line 1576, in execute stdout, stderr = Executor.execute(cmd, stdin_payload=stdin_payload, env=env, **kwargs) File "/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/unzipped_pexes/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/.bootstrap/pex/executor.py", line 99, in execute raise cls.NonZeroExit(cmd, process.returncode, stdout, stderr) pex.executor.Executor.NonZeroExit: received exit code 1 during execution of `['/root/.cache/nce/110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358/cpython-3.9.23+20250612-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9', '-s', '-E', '-m', 'venv', '/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/venvs/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/34c61b4506f36dad9fe0599deea04c7d08cde9bd.lck.work', '--without-pip', '--prompt', 'tools.pex']` while trying to execute `['/root/.cache/nce/110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358/cpython-3.9.23+20250612-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9', '-s', '-E', '-m', 'venv', '/root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/bindings/pex_root/venvs/1/9f456f34c6ccd1199ac98e22b12277de7423dc45/34c61b4506f36dad9fe0599deea04c7d08cde9bd.lck.work', '--without-pip', '--prompt', 'tools.pex']` Error: Failed to establish atomic directory /root/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/locks/configure-c7288ffc0c7678c86ec9650e348176440f82ec41e968ee7b0f9fd6f2875eaf5e. Population of work directory failed: Boot binding command failed: exit status: 1

Isolates your Pants from the elements.

Please select from the following boot commands:

(when SCIE_BOOT is not set in the environment) Detects the current Pants installation and launches it.
bootstrap-tools Introspection tools for the Pants bootstrap process.
update Update scie-pants.

You can select a boot command by setting the SCIE_BOOT environment variable.

Pants version

Tested on 2.27.0-2.31.0

OS

Mac (M3 Pro) and Linux (Container, ubuntu)

Additional info
n/a

[xkortex]

I raised this over here since it in fact seems rooted in a Pex issue, so feel free to close this if it's more relevant to Pex: pex-tool/pex#3107

[jsirois]

I'll add a note here when pex-tool/pex#3109 is out in Pex 2.90.2 and Pants can probably close this issue, but over in scie-pants (which is what actually experiences the issue), by upgrading its Pex 2.30.0 to >=2.90.2:
https://github.com/pantsbuild/scie-pants/blob/3203faa30d46800dadba485edd1167246c1dde3a/package/pbt.toml#L20-L24

[jsirois]

The Pex fix is released in https://github.com/pex-tool/pex/releases/tag/v2.90.2

[jsirois]

@cburroughs is scie-pants fixed? This is about scie-pants, not Pants. Confusingly (for a maintainer) the bug was filed under Pants.

[cburroughs]

Ah I thought we had and issue in all of pex, pants, and scie-pants, but I see there are just two. Will reopen this one.