Refactor tests: use TAEF metadata for WSL version filtering (#40129)

Replace runtime WSL2_TEST_ONLY()/WSL1_TEST_ONLY() skip macros with
TAEF metadata-based test selection. Tests that don't apply to the
current WSL version are now excluded by /select: queries at selection
time rather than skipped at runtime, eliminating hundreds of 'skipped'
results from test output.

Changes:
- Add WSL2_TEST_METHOD, WSL1_TEST_METHOD, WSLC_TEST_METHOD macros
  in Common.h that tag tests with WSLVersion metadata property
- Convert ~430 test methods across 26 files to use new macros
- Update run-tests.ps1 to auto-add /select: version filter
- Update CloudTest XML configs with version selection queries
- Remove WSL2_TEST_ONLY() from composite macros in NetworkTests.cpp
- Update test README with new macro documentation

Co-authored-by: Ben Hillis <benhill@ntdev.microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: 3 people

cloudtest/TestGroup-wslc.xml.in

@@ -22,6 +22,6 @@
   </Setup>
 
   <TestJob Name="CloudTest.Taef" TimeoutMins="120">
-    <Execution Type="TAEF" Path="[WorkingDirectory]\wsltests.dll" Args="/p:bugReportDirectory=[LoggingDirectory]\BugReportOutput /errorOnCrash /testmode:etwlogger /EtwLogger:WPRProfileFile=[WorkingDirectory]\wsl.wprp /EtwLogger:WPRProfile=WSL /EtwLogger:SavePoint=ExecutionComplete /EtwLogger:RecordingScope=Execution /p:SetupScript=.\test-setup.ps1 /p:Package=[WorkingDirectory]\Microsoft.WSL_${PACKAGE_VERSION}_x64_ARM64.msixbundle /p:Version=2 /p:AllowUnsigned=${ALLOW_UNSIGNED_PACKAGE} /p:UnitTestsPath=[WorkingDirectory]\unit_tests /p:DistroPath=[WorkingDirectory]\test_distro.tar.xz /p:TestDataPath=[WorkingDirectory]\test_data /p:DistroName=test_distro /logOutput:High /p:RedirectStdout=[LoggingDirectory]\stdout.txt /p:RedirectStderr=[LoggingDirectory]\stderr.txt  /p:KernelLogs=[LoggingDirectory]\dmesg.txt /p:DumpFolder=[LoggingDirectory] /p:WerReport /p:LogDmesg /p:PipelineBuildId=${PIPELINE_BUILD_ID} /p:DumpTool=DumpTool.exe /select:&quot;@TestCategory='WSLC'&quot;" />
+    <Execution Type="TAEF" Path="[WorkingDirectory]\wsltests.dll" Args="/p:bugReportDirectory=[LoggingDirectory]\BugReportOutput /errorOnCrash /testmode:etwlogger /EtwLogger:WPRProfileFile=[WorkingDirectory]\wsl.wprp /EtwLogger:WPRProfile=WSL /EtwLogger:SavePoint=ExecutionComplete /EtwLogger:RecordingScope=Execution /p:SetupScript=.\test-setup.ps1 /p:Package=[WorkingDirectory]\Microsoft.WSL_${PACKAGE_VERSION}_x64_ARM64.msixbundle /p:Version=2 /p:AllowUnsigned=${ALLOW_UNSIGNED_PACKAGE} /p:UnitTestsPath=[WorkingDirectory]\unit_tests /p:DistroPath=[WorkingDirectory]\test_distro.tar.xz /p:TestDataPath=[WorkingDirectory]\test_data /p:DistroName=test_distro /logOutput:High /p:RedirectStdout=[LoggingDirectory]\stdout.txt /p:RedirectStderr=[LoggingDirectory]\stderr.txt  /p:KernelLogs=[LoggingDirectory]\dmesg.txt /p:DumpFolder=[LoggingDirectory] /p:WerReport /p:LogDmesg /p:PipelineBuildId=${PIPELINE_BUILD_ID} /p:DumpTool=DumpTool.exe /select:&quot;@TestCategory='WSLC' and (@WSLVersion='2' or not(@WSLVersion='*'))&quot;" />
   </TestJob>
 </TestJobGroup>

cloudtest/TestGroup.xml.in

@@ -22,6 +22,6 @@
   </Setup>
 
   <TestJob Name="CloudTest.Taef" TimeoutMins="240">
-    <Execution Type="TAEF" Path="[WorkingDirectory]\wsltests.dll" Args="/p:bugReportDirectory=[LoggingDirectory]\BugReportOutput /errorOnCrash /testmode:etwlogger /EtwLogger:WPRProfileFile=[WorkingDirectory]\wsl.wprp /EtwLogger:WPRProfile=WSL /EtwLogger:SavePoint=ExecutionComplete /EtwLogger:RecordingScope=Execution /p:SetupScript=.\test-setup.ps1 /p:Package=[WorkingDirectory]\Microsoft.WSL_${PACKAGE_VERSION}_x64_ARM64.msixbundle /p:Version=${version} /p:AllowUnsigned=${ALLOW_UNSIGNED_PACKAGE} /p:UnitTestsPath=[WorkingDirectory]\unit_tests /p:DistroPath=[WorkingDirectory]\test_distro.tar.xz /p:TestDataPath=[WorkingDirectory]\test_data /p:DistroName=test_distro /logOutput:High /p:RedirectStdout=[LoggingDirectory]\stdout.txt /p:RedirectStderr=[LoggingDirectory]\stderr.txt  /p:KernelLogs=[LoggingDirectory]\dmesg.txt /p:DumpFolder=[LoggingDirectory] /p:WerReport /p:LogDmesg /p:PipelineBuildId=${PIPELINE_BUILD_ID} /p:DumpTool=DumpTool.exe /select:&quot;not(@TestCategory='WSLC')&quot;" />
+    <Execution Type="TAEF" Path="[WorkingDirectory]\wsltests.dll" Args="/p:bugReportDirectory=[LoggingDirectory]\BugReportOutput /errorOnCrash /testmode:etwlogger /EtwLogger:WPRProfileFile=[WorkingDirectory]\wsl.wprp /EtwLogger:WPRProfile=WSL /EtwLogger:SavePoint=ExecutionComplete /EtwLogger:RecordingScope=Execution /p:SetupScript=.\test-setup.ps1 /p:Package=[WorkingDirectory]\Microsoft.WSL_${PACKAGE_VERSION}_x64_ARM64.msixbundle /p:Version=${version} /p:AllowUnsigned=${ALLOW_UNSIGNED_PACKAGE} /p:UnitTestsPath=[WorkingDirectory]\unit_tests /p:DistroPath=[WorkingDirectory]\test_distro.tar.xz /p:TestDataPath=[WorkingDirectory]\test_data /p:DistroName=test_distro /logOutput:High /p:RedirectStdout=[LoggingDirectory]\stdout.txt /p:RedirectStderr=[LoggingDirectory]\stderr.txt  /p:KernelLogs=[LoggingDirectory]\dmesg.txt /p:DumpFolder=[LoggingDirectory] /p:WerReport /p:LogDmesg /p:PipelineBuildId=${PIPELINE_BUILD_ID} /p:DumpTool=DumpTool.exe /select:&quot;not(@TestCategory='WSLC') and (@WSLVersion='${version}' or not(@WSLVersion='*'))&quot;" />
   </TestJob>
 </TestJobGroup>

test/README.md

@@ -81,6 +81,18 @@ Below is a brief overview:
 
 ### Writing the Test
 
+For tests that only apply to a specific WSL version, use the version-specific test method macros instead of `TEST_METHOD`:
+
+- `WSL2_TEST_METHOD(Name)` — test only runs on WSL2
+- `WSL1_TEST_METHOD(Name)` — test only runs on WSL1
+- `WSLC_TEST_METHOD(Name)` — test only runs on WSL2 (for use in WSLC test classes)
+- `TEST_METHOD(Name)` — test runs on both WSL1 and WSL2
+
+These macros use TAEF metadata properties to tag tests with their required WSL version.
+When tests are run via `run-tests.ps1` or CloudTest, a `/select:` query automatically
+filters out tests that don't match the target version—so they don't appear in results at all
+(no "skipped" noise).
+
 For example, consider the file below, named `ExampleTest.cpp`:
 
 ```cpp
@@ -96,13 +108,19 @@ For example, consider the file below, named `ExampleTest.cpp`:
         {
             TEST_CLASS(ExampleTest) // define this as a test class
 
-            // add tests via test methods of the test class
-            TEST_METHOD(HelloWorldTest) // ExampleTest::ExampleTest::HelloWorldTest
+            // runs on both WSL1 and WSL2
+            TEST_METHOD(HelloWorldTest)
             {
                 std::wstring outputExpected = L"Linux on Windows Rocks!\n";
                 auto [output, __] = LxsstuLaunchWslAndCaptureOutput(L"echo Linux on Windows Rocks!"); // from /test/Common.h
                 VERIFY_ARE_EQUAL(output, outputExpected); // TAEF test method that passes if both are equal, and fails otherwise.
             }
+
+            // only runs on WSL2
+            WSL2_TEST_METHOD(Wsl2OnlyTest)
+            {
+                // ...
+            }
         };
     } //namespace ExampleTest
 ```

test/windows/Common.h

@@ -38,19 +38,29 @@ using namespace std::chrono_literals;
 
 #define LXSST_REMOVE_DISTRO_CONF_COMMAND_LINE L"-u root -e rm /etc/wsl.conf"
 
-#define WSL1_TEST_ONLY() \
-    if (LxsstuVmMode()) \
-    { \
-        LogSkipped("This test is only applicable to WSL1"); \
-        return; \
-    }
-
-#define WSL2_TEST_ONLY() \
-    if (!LxsstuVmMode()) \
-    { \
-        LogSkipped("This test is only applicable to WSL2"); \
-        return; \
-    }
+//
+// Test method declaration macros that tag tests with TAEF metadata for version-based selection.
+// Use these instead of TEST_METHOD() for tests that only apply to a specific WSL version.
+// When run via run-tests.ps1 or CloudTest, inapplicable tests are excluded from the run
+// entirely (no "skipped" noise) via TAEF /select: queries.
+//
+#define WSL1_TEST_METHOD(_name) \
+    TAEF_BEGIN_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE(_name) \
+    TEST_METHOD_PROPERTY(L"WSLVersion", L"1") \
+    TAEF_END_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE() \
+    TEST_METHOD(_name)
+
+#define WSL2_TEST_METHOD(_name) \
+    TAEF_BEGIN_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE(_name) \
+    TEST_METHOD_PROPERTY(L"WSLVersion", L"2") \
+    TAEF_END_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE() \
+    TEST_METHOD(_name)
+
+#define WSLC_TEST_METHOD(_name) \
+    TAEF_BEGIN_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE(_name) \
+    TEST_METHOD_PROPERTY(L"WSLVersion", L"2") \
+    TAEF_END_TEST_METHOD_PROPERTIES_IN_CLASS_SCOPE() \
+    TEST_METHOD(_name)
 
 // macro for skipping tests that are currently failing due to not yet being fully implemented
 #define SKIP_TEST_NOT_IMPL() \

test/windows/DrvFsTests.cpp

@@ -258,7 +258,6 @@ class DrvFsTests
 
     void DrvfsMountElevated(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -270,7 +269,6 @@ class DrvFsTests
 
     void DrvfsMountElevatedDifferentConsole(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -282,7 +280,6 @@ class DrvFsTests
 
     void DrvfsMountNonElevated(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -296,7 +293,6 @@ class DrvFsTests
 
     void DrvfsMountNonElevatedDifferentConsole(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -310,7 +306,6 @@ class DrvFsTests
 
     void DrvfsMountElevatedSystemDistroEnabled(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -322,7 +317,6 @@ class DrvFsTests
 
     void DrvfsMountNonElevatedSystemDistroEnabled(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
         WINDOWS_11_TEST_ONLY(); // TODO: Enable on Windows 10 when virtio support is added
         SKIP_TEST_ARM64();
 
@@ -383,8 +377,6 @@ class DrvFsTests
 
     void DrvFsMountUnicodePath(DrvFsMode Mode)
     {
-        WSL2_TEST_ONLY();
-
         // Create a Windows directory with unicode characters
         constexpr auto unicodeDir = L"C:\\drvfs-测试-テスト";
         auto cleanup = wil::scope_exit_log(WI_DIAGNOSTICS_INFO, [&]() { std::filesystem::remove_all(unicodeDir); });
@@ -1149,63 +1141,53 @@ class WSL1 : public DrvFsTests
         return true;
     }
 
-    TEST_METHOD(DrvFsDisableQueryByName)
+    WSL1_TEST_METHOD(DrvFsDisableQueryByName)
     {
-        WSL1_TEST_ONLY();
         VERIFY_NO_THROW(DrvFsCommon(LX_DRVFS_DISABLE_QUERY_BY_NAME));
     }
 
-    TEST_METHOD(DrvFsDisableQueryByNameAndStatInfo)
+    WSL1_TEST_METHOD(DrvFsDisableQueryByNameAndStatInfo)
     {
-        WSL1_TEST_ONLY();
         VERIFY_NO_THROW(DrvFsCommon(LX_DRVFS_DISABLE_QUERY_BY_NAME_AND_STAT_INFO));
     }
 
-    TEST_METHOD(VfsAccessDrvFs)
+    WSL1_TEST_METHOD(VfsAccessDrvFs)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::VfsAccessDrvFs();
     }
 
-    TEST_METHOD(FsCommonDrvFs)
+    WSL1_TEST_METHOD(FsCommonDrvFs)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::FsCommonDrvFs();
     }
 
-    TEST_METHOD(DrvFs)
+    WSL1_TEST_METHOD(DrvFs)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::DrvFs(DrvFsMode::WSL1);
     }
 
-    TEST_METHOD(DrvFsFat)
+    WSL1_TEST_METHOD(DrvFsFat)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::DrvFsFat(DrvFsMode::WSL1);
     }
 
-    TEST_METHOD(DrvFsSmb)
+    WSL1_TEST_METHOD(DrvFsSmb)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::DrvFsSmb(DrvFsMode::WSL1);
     }
 
-    TEST_METHOD(DrvFsMetadata)
+    WSL1_TEST_METHOD(DrvFsMetadata)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::DrvFsMetadata(DrvFsMode::WSL1);
     }
 
-    TEST_METHOD(XattrDrvFs)
+    WSL1_TEST_METHOD(XattrDrvFs)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::XattrDrvFs(DrvFsMode::WSL1);
     }
 
-    TEST_METHOD(WslPath)
+    WSL1_TEST_METHOD(WslPath)
     {
-        WSL1_TEST_ONLY();
         DrvFsTests::WslPath(DrvFsMode::WSL1);
     }
 };
@@ -1242,99 +1224,83 @@ class WSL1 : public DrvFsTests
             return true; \
         } \
 \
-        TEST_METHOD(VfsAccessDrvFs) \
+        WSL2_TEST_METHOD(VfsAccessDrvFs) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::VfsAccessDrvFs(); \
         } \
 \
-        TEST_METHOD(FsCommonDrvFs) \
+        WSL2_TEST_METHOD(FsCommonDrvFs) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::FsCommonDrvFs(); \
         } \
 \
-        TEST_METHOD(DrvFs) \
+        WSL2_TEST_METHOD(DrvFs) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFs(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvFsFat) \
+        WSL2_TEST_METHOD(DrvFsFat) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFsFat(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvFsSmb) \
+        WSL2_TEST_METHOD(DrvFsSmb) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFsSmb(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvFsMetadata) \
+        WSL2_TEST_METHOD(DrvFsMetadata) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFsMetadata(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountElevated) \
+        WSL2_TEST_METHOD(DrvfsMountElevated) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountElevated(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountElevatedDifferentConsole) \
+        WSL2_TEST_METHOD(DrvfsMountElevatedDifferentConsole) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountElevatedDifferentConsole(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountNonElevated) \
+        WSL2_TEST_METHOD(DrvfsMountNonElevated) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountNonElevated(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountNonElevatedDifferentConsole) \
+        WSL2_TEST_METHOD(DrvfsMountNonElevatedDifferentConsole) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountNonElevatedDifferentConsole(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountElevatedSystemDistroEnabled) \
+        WSL2_TEST_METHOD(DrvfsMountElevatedSystemDistroEnabled) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountElevatedSystemDistroEnabled(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvfsMountNonElevatedSystemDistroEnabled) \
+        WSL2_TEST_METHOD(DrvfsMountNonElevatedSystemDistroEnabled) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvfsMountNonElevatedSystemDistroEnabled(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(XattrDrvFs) \
+        WSL2_TEST_METHOD(XattrDrvFs) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::XattrDrvFs(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvFsReFs) \
+        WSL2_TEST_METHOD(DrvFsReFs) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFsReFs(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(WslPath) \
+        WSL2_TEST_METHOD(WslPath) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::WslPath(DrvFsMode::##_mode##); \
         } \
 \
-        TEST_METHOD(DrvFsMountUnicodePath) \
+        WSL2_TEST_METHOD(DrvFsMountUnicodePath) \
         { \
-            WSL2_TEST_ONLY(); \
             DrvFsTests::DrvFsMountUnicodePath(DrvFsMode::##_mode##); \
         } \
     }