Sunday 26 November 2017

.net core and entity framework integration tests with in memory database

    Integration testing is really useful, and any tool or framework which makes it easier will be welcomed. For .net core we are given the TestServer class and the entity framework's in memory database setup. I've built a small example, with a useful base class, which I'll present in this post.

Here's the whole code sample, including a small API in .net core 2,  and integration tests prepared using xUnit:
https://github.com/simonkatanski/coreintegrationtests

In the following I'm not going into basic knowledge of Entity Framework, or the setup of .net Core API. There's plenty articles about it elsewhere. This post shouldn't also be taken as an example of good design, I'm trying to make is as small and self-contained as possible.

I'll start with showing a simple DbContext which I want to "mock" with an "in-memory" version in my tests. A small db containing some exemplary city data.

public class CityContext : DbContextICityContext
{
    public DbSet<City> Cities { getset; }
    
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder.UseSqlite("Data Source=Cities.db");
}

It is used directly in our controller (not a good thing IRL ;)) like this:

[Route("api")]
public class CityController : Controller
{
    private readonly ICityContext _cityContext;

    public CityController(ICityContext cityContext)
    {
        _cityContext = cityContext;
    }
    
    [HttpGet("cities")]
    public IEnumerable<string> Get()
    {
        var cities = _cityContext.Cities.ToList();
        if (cities.Any())
        {
            return cities.Select(c => c.CityName);
        }
        return Enumerable.Empty<string>();
    }

    [HttpPost("cities")]
    public void Post([FromBody]City city)
    {
        _cityContext.Cities.Add(city);
        _cityContext.SaveChanges();
    }
}

As you can see it is minimal. Having created our small API we do want to create integration tests for the implemented actions. Now after creation of the integration tests project, we can create our "in-memory" version of CityContext.

public class InMemoryCityContext : CityContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder.UseInMemoryDatabase(Guid.NewGuid().ToString());
}

We derive from the context and override the OnConfiguring method. Bear in mind this might work slightly differently, if your DbContext is configured in a different way. Now to make it possible to swap our CityContext for the InMemoryCityContext implementation we need to prepare by using appropriate container registration.

In our case (we are using .net core built-in Dependency Injection IOC framework) we will use the following registration method, it will not register another ICityContext implementation, if there's already one existing in the container:

services.TryAddScoped<ICityContextCityContext>();

This means we need to register "in-memory" version before the actual API registration takes place. Now other containers are much more robust and performing a registration overriding is much easier using them. I'm going to focus on the built-in container.

Microsoft provides the following package for integration testing: "Microsoft.AspNetCore.TestHost". It contains the TestServer class, which can use our API's Startup class and set up an in-memory API for testing.

I've built a wrapper for it, to make it easier to manage the mocked DB context.

public class ApiTestServer : IDisposable
{
    private readonly TestServer _server;
    public InMemoryCityContext CityContext { get; }

    /// <summary>
    ///     A wrapper around the TestServer, which also contains the 
    ///     EF contexts used in the API.
    /// </summary>
    public ApiTestServer()
    {
        _server = new TestServer(new WebHostBuilder()
            .UseStartup<Startup>()
            .ConfigureServices(RegisterDependencies));

        CityContext = new InMemoryCityContext();
    }

    public RequestBuilder CreateRequest(string path)
    {
        return _server.CreateRequest(path);
    }
    
    /// <summary>
    ///     Register dependencies, which differ from the ordinary setup of the API. 
    ///     For the registration here to work, you need to use the TryAdd* versions
    ///     of container registration methods.
    /// </summary>
    private void RegisterDependencies(IServiceCollection service)
    {
        service.AddSingleton<ICityContextInMemoryCityContext>(serviceProvider => CityContext);
    }

    public void Dispose()
    {
        CityContext?.Dispose();
        _server?.Dispose();
    }
}

As you can see above the class exposes the Context outside, so that we can both set our test data in the context, and validate the changes introduced by the API. It also exposes the RequestBuilder, which allows us to send http requests. ApiTestServer registers the context as a singleton, the context is later used thanks to the TryAddScoped registration of the original base context.

An example created with xUnit:

[Fact]
public async Task GivenNonEmptyDb_ThenExpectCityToBeAdded()
{
    using (var server = new ApiTestServer())
    {
        //Arrange
        server.CityContext.Add(new City { CityName = "Bristol", Population = 100000, Id = 0 });
        server.CityContext.SaveChanges();

        var cityToAdd = new City { CityName = "Berlin", Population = 100000 };
        var stringData = JsonConvert.SerializeObject(cityToAdd);
        var request = server.CreateRequest("/api/cities")
            .And(c => c.Content = new StringContent(stringData, Encoding.UTF8, "application/json"))

        //Act
        var response = await request.PostAsync();

        //Assert
        Assert.True(response.StatusCode == HttpStatusCode.OK);
        Assert.True(server.CityContext.Cities.Count() == 2);
    }
}

In the example we:

  • create our test server
  • setup the context (state of the db before the API call)
  • prepare the request
  • call the API with the request
  • assert the context state

This way, in a real-life case, by adding all our contexts into the ApiTestServer class we can prepare a complete setup for our DB and our integration tests.

UPDATE:
I've explicitly set the ID to 0 for the entity added to the context prior to the http call. Obviously that's the default value for that parameter - but in case you were using some special data generator like Fizzware's NBuilder, you might forget about setting it to 0 explicitly and struggle to find the reason why EF is not assigning a new value to the newly added entity. For the Entity Framework's primary key's integer value generator to generate a new value it has to have 0 assigned, otherwise it'll be omitted from the generation.

Friday 17 November 2017

.net core dependency injection container and decorator pattern registration

I've spent some time with Dependency Injection container, which is the default out of the box IOC implementation, which comes with .net core. It's fast and lightweight.
You can see the performance comparison here:
http://www.palmmedia.de/blog/2011/8/30/ioc-container-benchmark-performance-comparison

It is however not very extensible. It's pretty easy to switch to something else, but if you really want to stay with the built in IOC you have to invest some time into extending it. The two classes which are the cornerstone of this IOC ServiceProvider and ServiceCollection. One of the issues I've had with it, was implementation of a decorator pattern.
http://www.dofactory.com/net/decorator-design-pattern

The following implementation, which I've had created is small and lightwaight. But it is also quite slow. I've omitted the null checks and most of the validation to make it more concise. At the end I'll note what could be done to make it faster and potentially more robust.

servicesCollection.AddTransient<IDataSourceDataSource>();
servicesCollection.AddDecorator<IDataSourceDataSource2>();

AddTransient is just an ordinary type registration method of the .net core IOC. AddDecorator looks as follows:

public static void AddDecorator<TServiceTDecorator>(this IServiceCollection services,
    ServiceLifetime lifetime = ServiceLifetime.Scoped)
    where TService : class
    where TDecorator : class
{
    var serviceDescriptor = new ServiceDescriptor(
        typeof(TService),
        provider => Construct<TServiceTDecorator>(provider, services), lifetime);
    services.Add(serviceDescriptor);
}

One of the ways of extending how the types are resolved is by adding a factory based registration as above. All the container's methods are based on service descriptors, and we are using on the available ones. Unfortunately, factory registrations are not generic, and hence they don't store the resolved type in the service collection - it would be much easier if that was not the case.

private static TDecorator Construct<TServiceTDecorator>(IServiceProvider serviceProvider,
    IServiceCollection services)
    where TDecorator : class
    where TService : class
{
    var type = GetDecoratedType<TService>(services);
    var decoratedConstructor = GetConstructor(type);
    var decoratorConstructor = GetConstructor(typeof(TDecorator));
    var docoratedDependencies = serviceProvider.ResolveConstructorDependencies(
        decoratedConstructor.GetParameters());
    var decoratedService = decoratedConstructor.Invoke(docoratedDependencies.ToArray()) 
        as TService;
    var decoratorDependencies = serviceProvider.ResolveConstructorDependencies(
        decoratedService, 
        decoratorConstructor.GetParameters());
    return decoratorConstructor.Invoke(decoratorDependencies.ToArray()) as TDecorator;
}

The factory method takes in the generic type TDecorator of the decorator service and the TService type of the interface it implements.
First, we need to find the decorated type, which is already registered in the container's service collection.

private static Type GetDecoratedType<TService>(IServiceCollection services)
{
    if (services.Count(p => 
        p.ServiceType == typeof(TService&& 
        p.ImplementationFactory == null> 1)
    {
        throw new InvalidOperationException(
            $"Only one decorated service for interface {nameof(TService)} allowed");
    }

    var nonFactoryDescriptor = services.FirstOrDefault(p => 
        p.ServiceType == typeof(TService&& 
        p.ImplementationFactory == null);
    return nonFactoryDescriptor?.ImplementationType;
}

To find the correct implementation we need to find the one which implement the interface TService and which doesn't have a factory based registration (because only the decorator class can have such).
Then we return the type.

private static ConstructorInfo GetConstructor(Type type)
{
    var availableConstructors = type
        .GetConstructors()
        .Where(c => c.IsPublic)
        .ToList();

    if (availableConstructors.Count!= 1)
    {
        throw new InvalidOperationException("Only single constructor types are supported");
    }
    return availableConstructors.First();
}

Then we create constructors for both the decorator and the decorated services. We only support single public constructor classes. With the constructor info we have access to the number and types of all constructor parameters for these 2 classes. We resolve these dependencies using the ServiceProvider. In decorator's case we inject the constructed decorated service's instance. Here is the example of the extension methods for the ServiceProvider.

public static List<object> ResolveConstructorDependencies<TService>(
    this IServiceProvider serviceProvider,
    TService decorated,
    IEnumerable<ParameterInfo> constructorParameters)
{
    var depencenciesList = new List<object>();
    foreach (var parameter in constructorParameters)
    {
        if (parameter.ParameterType == typeof(TService))
        {
            depencenciesList.Add(decorated);
        }
        else
        {
            var resolvedDependency = serviceProvider.GetService(parameter.ParameterType);
            depencenciesList.Add(resolvedDependency);
        }
    }
    return depencenciesList;
}

public static List<object> ResolveConstructorDependencies(
    this IServiceProvider serviceProvider,
    IEnumerable<ParameterInfo> constructorParameters)
{
    var depencenciesList = new List<object>();
    foreach (var parameter in constructorParameters)
    {
        var resolvedDependency = serviceProvider.GetService(parameter.ParameterType);
        depencenciesList.Add(resolvedDependency);
    }
    return depencenciesList;
}

Having materialized a collection of dependencies, we can create an instance of the Decorator class with previously injected Decorated instance, using the Invoke method of the constructor.

decoratorConstructor.Invoke(decoratorDependencies.ToArray()); 

The exact sources are available in the following github repository:
https://github.com/simonkatanski/dependencyinjection.extensions

The things to improve:
- null checks
- more validation
- constructor lookup (instead of creating constructor each time we could just add it to some sort of dictionary lookup during registration and then just reuse it on each call
- support for multiple constructors (this container has it by default, if you venture into its code with reflection)
- implement registrations out of order
- find a way to have factory based decorator registration