Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test discovery problem when single quote is used in F# method name #4313

Closed
VasilSirakov opened this issue Feb 17, 2023 · 1 comment · Fixed by #4969
Closed

Test discovery problem when single quote is used in F# method name #4313

VasilSirakov opened this issue Feb 17, 2023 · 1 comment · Fixed by #4969

Comments

@VasilSirakov
Copy link

VasilSirakov commented Feb 17, 2023

Description

When specifying a name for a test method with the double backtick syntax, adding a single quote is allowed by the compiler. This, however, causes some issue with the test discovery and it prevents not only the particular test from being discovered, but also all other tests in the test file defined below it.

Running dotnet test outputs the following warning:

[MSTest][Discovery][C:\path\to\solution\SingleQuoteTestName\bin\Debug\net7.0\SingleQuoteTestName.dll] MSTestAdapter failed to discover tests in class 'SingleQuoteTestName.TestClass' of assembly 'C:\path\to\solution\SingleQuoteTestName\bin\Debug\net7.0\SingleQuoteTestName.dll' because Whitespace is not valid in a ManagedName (pos: 19).

Given this warning it is hard to narrow it down immediately to the single quote being the problem. If you have many other tests, it is easy to miss this as the test run will not fail, rather it will just run the tests it managed to discover and succeed if those pass.

In my initial attempts to reproduce this issue, it seemed like having an odd number of single quotes was causing the issue whereas an even number was OK. After some further experiments, I unfortunately couldn't really narrow it down to some fixed pattern for when it happens and when it doesn't happen.

Steps to reproduce

Simply create a blank F# test project with no other dependencies and paste this in the Tests.fs file.

namespace SingleQuoteTestName

open System
open Microsoft.VisualStudio.TestTools.UnitTesting

[<TestClass>]
type TestClass () =

    [<TestMethod>]
    member _.``test that doesn't pass`` () =
        Assert.IsTrue(false);

    [<TestMethod>]
    member _.``test that passes`` () =
        Assert.IsTrue(true);

Then, run dotnet test and observe the output.

Expected behavior

All tests with names accepted by the compiler are discovered and executed.

Actual behavior

The single quote in the name of the first test method is preventing it from being discovered as well as all other tests in this file.

Environment

Replicated this both on .NET 6 (6.0.13) & .NET 7 (7.0.2).

AB#1819417

@microsoft-github-policy-service microsoft-github-policy-service bot added the needs-triage This item should be discussed in the next triage meeting. label Feb 17, 2023
@nohwnd
Copy link
Member

nohwnd commented Feb 27, 2023

I can repro, it is not just the presence of ', (e.g. ' works, 'hello' works.
But 'a does not work, or ' , or ' '.

@nohwnd nohwnd added bug needs-attention and removed needs-triage This item should be discussed in the next triage meeting. labels Feb 27, 2023
@Evangelink Evangelink self-assigned this May 15, 2023
@Evangelink Evangelink assigned nohwnd and unassigned Evangelink Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants