Skip to content

fix: caseless match for arm64 and proper check for win32#97

Open
vinovo wants to merge 1 commit into
mlc-ai:mainfrom
vinovo:fix/paul/cmake
Open

fix: caseless match for arm64 and proper check for win32#97
vinovo wants to merge 1 commit into
mlc-ai:mainfrom
vinovo:fix/paul/cmake

Conversation

@vinovo
Copy link
Copy Markdown

@vinovo vinovo commented May 16, 2026

Summary

Two small CMake fixes that make tokenizers-cpp build robustly on Windows
across different compilers, generators, and toolchain files. The previous
logic happened to work for the common MSVC + Visual Studio generator case
but was not robust to other valid Windows configurations.

Changes

1. Pick .lib vs .a based on the OS, not the host compiler

rustc on Windows always emits tokenizers_c.lib regardless of which
C/C++ compiler CMake is configured with — the file name is chosen by Rust,
not by the host compiler. The previous gate if(MSVC) is only true for
cl.exe, so any other Windows compiler (where MSVC is FALSE) was
expecting libtokenizers_c.a. The custom command's OUTPUT was never
produced and the post-build copy step failed.

Switching the gate to if(WIN32) matches what Rust actually produces and
makes the rule depend on the target OS rather than on which host compiler
happens to be in use.

2. Case-insensitive CMAKE_SYSTEM_PROCESSOR match on Windows

The Windows branch only accepted the exact spellings ARM64 (uppercase)
or aarch64 (lowercase). The uppercase form is filled in by some
generators (e.g. Visual Studio invoked with -A ARM64); other valid
configurations such as Ninja + a toolchain file may use mixed/lowercase
arm64, which silently fell through to the else branch and selected
x86_64-pc-windows-msvc. Rust then cross-built the wrong architecture
and the resulting .lib could not link against the rest of the arm64
build.

Normalizing CMAKE_SYSTEM_PROCESSOR with string(TOUPPER ...) before
comparing makes the detection robust to whichever spelling the generator
or toolchain file happens to use.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The changes in CMakeLists.txt improve the robustness of the Windows build configuration. Specifically, the processor check for ARM64 is now case-insensitive by converting the processor string to uppercase, and the library extension logic has been updated to use WIN32 instead of MSVC to ensure correct naming across different Windows compilers. I have no feedback to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant