Resources

Microsoft Testing index Microsoft Testing Integration test NUnit in 1 hour(youtube) NUnit and XUnit are different things Integration test BDD and specFlow BDD in vs

Comparision

Functionality NUnit 3.x MSTest v2.x. xUnit.net 2.x
Marks a test method. [Test] [TestMethod] [Fact]
Marks a test class. [TestFixture] [TestClass] n/a
Triggered before every test case. [SetUp] [TestInitialize] Constructor
Triggered after every test case. [TearDown] [TestCleanup] IDisposable.Dispose
One-time triggered method before test cases start. [OneTimeSetUp] [ClassInitialize] IClassFixture< T >
One-time triggered method after test cases end. [OneTimeTearDown] [ClassCleanup] IClassFixture< T >
Ignores a test case. [Ignore("reason")] [Ignore] [Fact(Skip="reason")]
Sets arbitrary metadata on a test. [Property] [TestProperty] [Trait]
Configures a data-driven test. [Theory] [DataRow] [Theory]
Categorizes the test cases or classes. [Category("")] [TestCategory("")] [Trait("Category", "")]

NUnit

Installation: Install-Package NUnit Install-Package NUnit.TestAdapter Install-Package Microsoft.NET.Test.Sdk Work flow and syntaxes:

using NUnit.Framework;
namespace NUnitUnitTests
{
 // A class that contains NUnit unit tests. (Required)
 [TestFixture]
 public class NonBellatrixTests
 {
 [OneTimeSetUp]
 public void ClassInit()
 {
 // Executes once for the test class. (Optional)
 }
 [SetUp]
 public void TestInit()
 {
 // Runs before each test. (Optional)
 }
 [Test]
 public void TestMethod()
 {
 }
 [TearDown]
 public void TestCleanup()
 {
 // Runs after each test. (Optional)
 }
 [OneTimeTearDown]
 public void ClassCleanup()
 {
 // Runs once after all tests in this class are executed. (Optional)
 // Not guaranteed that it executes instantly after all tests from the class.
 }
 }
}
// A SetUpFixture outside of any namespace provides SetUp and TearDown for the entire assembly.
[SetUpFixture]
public class MySetUpClass
{
 [OneTimeSetUp]
 public void RunBeforeAnyTests()
 {
 // Executes once before the test run. (Optional)
 }
 [OneTimeTearDown]
 public void RunAfterAnyTests()
 {
 // Executes once after the test run. (Optional)
 }
}

NUnit Assertion

Classic:

Assert.AreEqual(28, _actualFuel); // Tests whether the specified values are equal. 
Assert.AreNotEqual(28, _actualFuel); // Tests whether the specified values are unequal. Same as AreEqual for numeric values.
Assert.AreSame(_expectedRocket, _actualRocket); // Tests whether the specified objects both refer to the same object
Assert.AreNotSame(_expectedRocket, _actualRocket); // Tests whether the specified objects refer to different objects
Assert.IsTrue(_isThereEnoughFuel); // Tests whether the specified condition is true
Assert.IsFalse(_isThereEnoughFuel); // Tests whether the specified condition is false
Assert.IsNull(_actualRocket); // Tests whether the specified object is null
Assert.IsNotNull(_actualRocket); // Tests whether the specified object is non-null
Assert.IsInstanceOf(_actualRocket, typeof(Falcon9Rocket)); // Tests whether the specified object is an instance of the expected type
Assert.IsNotInstanceOf(_actualRocket, typeof(Falcon9Rocket)); // Tests whether the specified object is not an instance of type
StringAssert.AreEqualIgnoringCase(_expectedBellatrixTitle, "Bellatrix"); // Tests whether the specified strings are equal ignoring their casing
StringAssert.Contains(_expectedBellatrixTitle, "Bellatrix"); // Tests whether the specified string contains the specified substring
StringAssert.DoesNotContain(_expectedBellatrixTitle, "Bellatrix"); // Tests whether the specified string doesn't contain the specified substring
StringAssert.StartsWith(_expectedBellatrixTitle, "Bellatrix"); // Tests whether the specified string begins with the specified substring
StringAssert.StartsWith(_expectedBellatrixTitle, "Bellatrix"); // Tests whether the specified string begins with the specified substring
StringAssert.IsMatch("(281)388-0388", @"(?d{3})?-? *d{3}-? *-?d{4}"); // Tests whether the specified string matches a regular expression
StringAssert.DoesNotMatch("281)388-0388", @"(?d{3})?-? *d{3}-? *-?d{4}"); // Tests whether the specified string does not match a regular expression
CollectionAssert.AreEqual(_expectedRockets, _actualRockets); // Tests whether the specified collections have the same elements in the same order and quantity.
CollectionAssert.AreNotEqual(_expectedRockets, _actualRockets); // Tests whether the specified collections does not have the same elements or the elements are in a different order and quantity.
CollectionAssert.AreEquivalent(_expectedRockets, _actualRockets); // Tests whether two collections contain the same elements.
CollectionAssert.AreNotEquivalent(_expectedRockets, _actualRockets); // Tests whether two collections contain different elements.
CollectionAssert.AllItemsAreInstancesOfType(_expectedRockets, _actualRockets); // Tests whether all elements in the specified collection are instances of the expected type
CollectionAssert.AllItemsAreNotNull(_expectedRockets); // Tests whether all items in the specified collection are non-null
CollectionAssert.AllItemsAreUnique(_expectedRockets); // Tests whether all items in the specified collection are unique
CollectionAssert.Contains(_actualRockets, falcon9); // Tests whether the specified collection contains the specified element
CollectionAssert.DoesNotContain(_actualRockets, falcon9); // Tests whether the specified collection does not contain the specified element
CollectionAssert.IsSubsetOf(_expectedRockets, _actualRockets); // Tests whether one collection is a subset of another collection
CollectionAssert.IsNotSubsetOf(_expectedRockets, _actualRockets); // Tests whether one collection is not a subset of another collection
Assert.Throws(() => new Regex(null)); // Tests whether the code specified by delegate throws exact given exception of type T
Constraint Model:

Assert.That(28, Is.EqualTo(_actualFuel)); // Tests whether the specified values are equal. 
Assert.That(28, Is.Not.EqualTo(_actualFuel)); // Tests whether the specified values are unequal. Same as AreEqual for numeric values.
Assert.That(_expectedRocket, Is.SameAs(_actualRocket)); // Tests whether the specified objects both refer to the same object
Assert.That(_expectedRocket, Is.Not.SameAs(_actualRocket)); // Tests whether the specified objects refer to different objects
Assert.That(_isThereEnoughFuel, Is.True); // Tests whether the specified condition is true
Assert.That(_isThereEnoughFuel, Is.False); // Tests whether the specified condition is false
Assert.That(_actualRocket, Is.Null); // Tests whether the specified object is null
Assert.That(_actualRocket, Is.Not.Null); // Tests whether the specified object is non-null
Assert.That(_actualRocket, Is.InstanceOf()); // Tests whether the specified object is an instance of the expected type
Assert.That(_actualRocket, Is.Not.InstanceOf()); // Tests whether the specified object is not an instance of type
Assert.That(_actualFuel, Is.GreaterThan(20)); // Tests whether the specified object greater than the specified value
Assert.That(28, Is.EqualTo(_actualFuel).Within(0.50));
// Tests whether the specified values are nearly equal within the specified tolerance.
Assert.That(28, Is.EqualTo(_actualFuel).Within(2).Percent);
// Tests whether the specified values are nearly equal within the specified % tolerance.
Assert.That(_actualRocketParts, Has.Exactly(10).Items);
// Tests whether the specified collection has exactly the stated number of items in it.
Assert.That(_actualRocketParts, Is.Unique);
// Tests whether the items in the specified collections are unique.
Assert.That(_actualRocketParts, Does.Contain(_expectedRocketPart));
// Tests whether a given items is present in the specified list of items.
Assert.That(_actualRocketParts, Has.Exactly(1).Matches(part => part.Name == "Door" && part.Height == "200"));
// Tests whether the specified collection has exactly the stated item in it.

Advanced attributes

//RepeatAttribute is used on a test method to specify that it should be executed multiple times. 
//If any repetition fails, the remaining ones are not run and a failure is reported.

[Test]
[Repeat(10)]
public void RocketFuelMeassuredCorrectly_When_Flying() { /* ... */ }


// The CombinatorialAttribute is used on a test to specify that 
// NUnit should generate test cases for all possible combinations of the individual data items
// provided for the parameters of a test.
[Test, Combinatorial]
public void CorrectFuelMeassured_When_X_Site([Values(1,2,3)] int x, [Values("A","B")] string s)
{
 ...
}
//Generated tests:
//CorrectFuelMeassured_When_X_Site(1, "A")
//CorrectFuelMeassured_When_X_Site(1, "B")
//CorrectFuelMeassured_When_X_Site(2, "A")

//The PairwiseAttribute is used on a test to specify that NUnit should generate test cases 
//in such a way that all possible pairs of values are used.
[Test, Pairwise]
public void ValidateLandingSiteOfRover_When_GoingToMars
 ([Values("a", "b", "c")] string a, [Values("+", "-")] string b, [Values("x", "y")] string c)
{
 Debug.WriteLine("{0} {1} {2}", a, b, c);
}

//The RandomAttribute is used to specify a set of random values to be provided for an individual numeric parameter of a parameterized test method.
[Test]
public void GenerateRandomLandingSiteOnMoon([Values(1,2,3)] int x, [Random(-1.0, 1.0, 5)] double d)
{
 ...
}

//The RangeAttribute is used to specify a range of values to be provided for an individual parameter of a parameterized test method. NUnit creates test cases from all possible combinations of the provided on parameters - the combinatorial approach.

[Test]
public void CalculateJupiterBaseLandingPoint([Values(1,2,3)] int x, [Range(0.2,0.6)] double y)
{
 //...
}
//Generated tests:
//CalculateJupiterBaseLandingPoint(1, 0.2)
//CalculateJupiterBaseLandingPoint(1, 0.4)
//CalculateJupiterBaseLandingPoint(1, 0.6)
//...


//RetryAttribute is used on a test method to specify that it should be rerun if it fails,
//up to a maximum number of times.

[Test]
[Retry(3)]
public void CalculateJupiterBaseLandingPoint([Values(1,2,3)] int x, [Range(0.2,0.6)] double y)
{
 //...
}

// The TimeoutAttribute is used to specify a timeout value in milliseconds for a test case. 
//If the test case runs longer than the time specified it is immediately cancelled and reported as a failure, 
//with a message indicating that the timeout was exceeded.

[Test, Timeout(2000)]
public void FireRocketToProximaCentauri()
{
 ...
}

NUnit Examples


using NUnit.Framework;
using Prime.Services;

namespace Prime.UnitTests.Services
{
    [TestFixture]
    public class PrimeService_IsPrimeShould
    {
        private PrimeService _primeService;

        [SetUp]
        public void SetUp()
        {
            _primeService = new PrimeService();
        }

        [Test]
        public void IsPrime_InputIs1_ReturnFalse()
        {
            var result = _primeService.IsPrime(1);

            Assert.IsFalse(result, "1 should not be prime");
        }
    }
}

Test Methods ( MSTest projects )

It's decorated with the [TestMethod] attribute. It returns void. It cannot have parameters.

requirements for a test class

.The [TestClass] attribute is required on any class that contains unit test methods that you want to run in Test Explorer. .Each test method that you want Test Explorer to recognize must have the [TestMethod] attribute.

A sample test method


[TestMethod]
public void Debit_WithValidAmount_UpdatesBalance()
{
    // Arrange
    double beginningBalance = 11.99;
    double debitAmount = 4.55;
    double expected = 7.44;
    BankAccount account = new BankAccount("Mr. Bryan Walton", beginningBalance);

    // Act
    account.Debit(debitAmount);

    // Assert
    double actual = account.Balance;
    Assert.AreEqual(expected, actual, 0.001, "Account not debited correctly");
}    

Build and run the test

On the Build menu, choose Build Solution (or press Ctrl + SHIFT + B). If Test Explorer is not open, open it by choosing Test > Windows > Test Explorer from the top menu bar (or press Ctrl + E, T). Choose Run All to run the test (or press Ctrl + R, V).

Exception


[TestMethod]
public void Debit_WhenAmountIsLessThanZero_ShouldThrowArgumentOutOfRange()
{
    // Arrange
    double beginningBalance = 11.99;
    double debitAmount = -100.00;
    BankAccount account = new BankAccount("Mr. Bryan Walton", beginningBalance);

    // Act and assert
    Assert.ThrowsException(() => account.Debit(debitAmount));
}

Microsoft fake

Microsoft Fakes helps you isolate the code you're testing by replacing other parts of the application with stubs or shims. The stubs and shims are small pieces of code that are under the control of your tests. By isolating your code for testing, you know that if the test fails, the cause is there and not somewhere else. Stubs and shims also let you test your code even if other parts of your application aren't working yet.

test with stubs


        [TestClass]
        class TestStockAnalyzer
        {
            [TestMethod]
            public void TestContosoStockPrice()
            {
              // Arrange:
        
                // Create the fake stockFeed:
                IStockFeed stockFeed =
                     new StockAnalysis.Fakes.StubIStockFeed() // Generated by Fakes.
                         {
                             // Define each method:
                             // Name is original name + parameter types:
                             GetSharePriceString = (company) => { return 1234; }
                         };
        
                // In the completed application, stockFeed would be a real one:
                var componentUnderTest = new StockAnalyzer(stockFeed);
        
                // Act:
                int actualValue = componentUnderTest.GetContosoPrice();
        
                // Assert:
                Assert.AreEqual(1234, actualValue);
            }
            ...
        }       
    

Verify parameter values

You can verify that when your component makes a call to another component, it passes the correct values. You can either place an assertion in the stub, or you can store the value and verify it in the main body of the test. For example:


    [TestClass]
    class TestMyComponent
    {
        [TestMethod]
        public void TestVariableContosoPrice()
        {
            // Arrange:
            int priceToReturn = 345;
            string companyCodeUsed = "";
            var componentUnderTest = new StockAnalyzer(new StockAnalysis.Fakes.StubIStockFeed()
                {
                   GetSharePriceString = (company) =>
                      {
                         // Store the parameter value:
                         companyCodeUsed = company;
                         // Return the value prescribed by this test:
                         return priceToReturn;
                      };
                };
    
            // Act:
            int actualResult = componentUnderTest.GetContosoPrice();
    
            // Assert:
            // Verify the correct result in the usual way:
            Assert.AreEqual(priceToReturn, actualResult);
    
            // Verify that the component made the correct call:
            Assert.AreEqual("COOO", companyCodeUsed);
        }
    ...
    }

Stubs for different kinds of type members

Methods

As described in the example, methods can be stubbed by attaching a delegate to an instance of the stub class. The name of the stub type is derived from the names of the method and parameters. For example, given the following IMyInterface interface and method MyMethod:



    // application under test
    interface IMyInterface
    {
        int MyMethod(string value);
    }
    We attach a stub to MyMethod that always returns 1:
    

    // unit test code
    var stub = new StubIMyInterface ();
    stub.MyMethodString = (value) => 1;
    

If you do not provide a stub for a function, Fakes generates a function that returns the default value of the return type. For numbers, the default value is 0 and for class types it is null (C#) or Nothing (Visual Basic). Properties Property getters and setters are exposed as separate delegates and can be stubbed separately. For example, consider the Value property of IMyInterface:


    // code under test
    interface IMyInterface
    {
        int Value { get; set; }
    }
    We attach delegates to the getter and setter of Value to simulate an auto-property:

    
    // unit test code
    int i = 5;
    var stub = new StubIMyInterface();
    stub.ValueGet = () => i;
    stub.ValueSet = (value) => i = value;
  

If you do not provide stub methods for either the setter or the getter of a property, Fakes generates a stub that stores values so that the stub property works like a simple variable. Events Events are exposed as delegate fields. As a result, any stubbed event can be raised simply by invoking the event backing field. Let's consider the following interface to stub:


    // code under test
    interface IWithEvents
    {
        event EventHandler Changed;
    }
    To raise the Changed event, we simply invoke the backing delegate:
    
    // unit test code
      var withEvents = new StubIWithEvents();
      // raising Changed
      withEvents.ChangedEvent(withEvents, EventArgs.Empty);
   

Generic methods

It's possible to stub generic methods by providing a delegate for each desired instantiation of the method. For example, given the following interface containing a generic method:


    // code under test
    interface IGenericMethod
    {
        T GetValue();
    }
    You could write a test that stubs the GetValue instantiation:
    
    // unit test code
    [TestMethod]
    public void TestGetValue()
    {
        var stub = new StubIGenericMethod();
        stub.GetValueOf1(() => 5);
    
        IGenericMethod target = stub;
        Assert.AreEqual(5, target.GetValue());
    }
 
If the code were to call GetValue with any other instantiation, the stub would simply

Shims

Shim types are one of two technologies that the Microsoft Fakes Framework uses to let you isolate components under test from the environment. Shims divert calls to specific methods to code that you write as part of your test. Many methods return different results dependent on external conditions, but a shim is under the control of your test and can return consistent results at every call. This makes it easier to write the tests. Use shims to isolate your code from assemblies that are not part of your solution. To isolate components of your solution from each other, use stubs.

How to use shims

First, add a Fakes assembly: In Solution Explorer, For an older .NET Framework Project (non-SDK style), expand your unit test project's References node. For an SDK-style project targeting .NET Framework, .NET Core, or .NET 5.0 or later, expand the Dependencies node to find the assembly you would like to fake under Assemblies, Projects, or Packages. If you're working in Visual Basic, select Show All Files in the Solution Explorer toolbar to see the References node. Select the assembly that contains the class definitions for which you want to create shims. For example, if you want to shim DateTime, select System.dll. On the shortcut menu, choose Add Fakes Assembly.

Use ShimsContext

When using shim types in a unit test framework, wrap the test code in a ShimsContext to control the lifetime of your shims. Otherwise, the shims would last until the AppDomain shut down. The easiest way to create a ShimsContext is by using the static Create() method as shown in the following code:


//unit test code
[Test]
public void Y2kCheckerTest() {
  using(ShimsContext.Create()) {
    ...
  } // clear all shims
}    

It's critical to properly dispose each shim context. As a rule of thumb, call the ShimsContext. Create inside of a using statement to ensure proper clearing of the registered shims. For example, you might register a shim for a test method that replaces the DateTime. Now method with a delegate that always returns the first of January 2000. If you forget to clear the registered shim in the test method, the rest of the test run would always return the first of January 2000 as the DateTime.Now value. This might be surprising and confusing.

Write a test with shims


 [TestClass]
public class TestClass1
{
        [TestMethod]
        public void TestCurrentYear()
        {
            int fixedYear = 2000;

            using (ShimsContext.Create())
            {
              // Arrange:
                // Detour DateTime.Now to return a fixed date:
                System.Fakes.ShimDateTime.NowGet =
                () =>
                { return new DateTime(fixedYear, 1, 1); };

                // Instantiate the component under test:
                var componentUnderTest = new MyComponent();

              // Act:
                int year = componentUnderTest.GetTheCurrentYear();

              // Assert:
                // This will always be true if the component is working:
                Assert.AreEqual(fixedYear, year);
            }
        }
}   

Shim class names are made up by prefixing Fakes.Shim to the original type name. Shims work by inserting detours into the code of the application under test. Wherever a call to the original method occurs, the Fakes system performs a detour, so that instead of calling the real method, your shim code is called. Notice that detours are created and deleted at run time. You must always create a detour within the life of a ShimsContext. When it is disposed, any shims you created while it was active are removed. The best way to do this is inside a using statement. You might see a build error stating that the Fakes namespace does not exist. This error sometimes appears when there are other compilation errors. Fix the other errors and it will vanish.

shims advanced

https://docs.microsoft.com/en-us/visualstudio/test/using-shims-to-isolate-your-application-from-other-assemblies-for-unit-testing?view=vs-2022&tabs=csharp

data-driven unit test

set up a unit test method to retrieve values from a data source. The method is run successively for each row in the data source, which makes it easy to test a variety of input by using a single method. Creating a data-driven unit test involves the following steps: Create a data source that contains the values that you use in the test method. The data source can be any type that is registered on the machine that runs the test. Add a private TestContext field and a public TestContext property to the test class. Create a unit test method and add a DataSourceAttribute attribute to it. Use the DataRow indexer property to retrieve the values that you use in a test.

Add a TestContext to the test class

The unit test framework creates a TestContext object to store the data source information for a data-driven test. The framework then sets this object as the value of the TestContext property that you create.


private TestContext testContextInstance;
public TestContext TestContext
{
    get { return testContextInstance; }
    set { testContextInstance = value; }
}      

Write the test method

The test method for AddIntegers is fairly simple. For each row in the data source, call AddIntegers with the FirstNumber and SecondNumber column values as parameters, and verify the return value against Sum column value:


   [DataSource(@"Provider=Microsoft.SqlServerCe.Client.4.0; Data Source=C:\Data\MathsData.sdf;", "Numbers")]
    [TestMethod()]
    public void AddIntegers_FromDataSourceTest()
    {
        var target = new Maths();
    
        // Access the data
        int x = Convert.ToInt32(TestContext.DataRow["FirstNumber"]);
        int y = Convert.ToInt32(TestContext.DataRow["SecondNumber"]);
        int expected = Convert.ToInt32(TestContext.DataRow["Sum"]);
        int actual = target.IntegerMethod(x, y);
        Assert.AreEqual(expected, actual,
            "x:<{0}> y:<{1}>",
            new object[] {x, y});
    }    

The Assert method includes a message that displays the x and y values of a failed iteration. By default, the asserted values - expected and actual - are already included in failed test details.

Run an inline data-driven test

For inline tests, MSTest uses DataRow to retrieve values from a data source. The test in this example runs successively for each data row.


[DataTestMethod]
[DataRow(1, 1, 2)]
[DataRow(2, 2, 4)]
[DataRow(3, 3, 6)]
[DataRow(0, 0, 1)] // The test run with this row fails
public void AddTests(int x, int y, int expected)
{
  Assert.AreEqual(expected, x + y);
}