Revert "test matplotlib in free threading"

This reverts commit 686f7ac.

Author: jcapriot

.ci/environment_test_bare.yml

@@ -9,7 +9,6 @@ dependencies:
   - sympy
   - pytest
   - pytest-cov
-  - matplotlib
 
   # Building
   - pip