A response time benchmark that compares the frameworks ASP.NET 4.8 and ASP.NET CORE 3.0 ensures what I just suspected.
When I started developing with ASP.NET CORE, I realized some response time performance changes comparing with ASP.NET Framework.
I thought to myself I need to perform a benchmark about it.
I won’t write some complex benchmark with memory and processor data, but some simple stuff, easy to understand.
It’s important to say that if you are a new developer or never seen ASP.NET CORE before, I suggest you read my article .NET CORE for .NET developers first.
I didn’t use the BenchmarDotNet library!
I have talked to a friend about this article, and He suggested me to use the BenchmarkDotNet library as the main focus for my benchmark.
About the BenchmarkDotNet library, you can install it from NuGet and, when you put some attributes in methods, properties, classes, you will be able to run it.
It returns lots of information about CLR/Framework, memory, processor, and other stuff. If you want to know more about it, check their website.
The BenchMarkDotNet library seems very interesting but is not the focus of my article.
The guidelines for the benchmark are:
– Use the developer’s machine.
– Use the default Visual Studio template.
– The application must have only an API.
– Focus on response time only.
– All NuGet packages must be updated.
– Install Newtonsoft JSON from NuGet.
– Install Dapper from NuGet.
– Use IIS Express and debug mode.
– Use Postman for requests.
– No source code changes permitted.
– All SQL Server tables have more than one million records.
For this benchmark, I followed those steps:
1 – Start debugging the application.
2 – Open Postman.
3 – Send the request.
4 – Get the response time (in milliseconds).
5 – Repeat these steps three times.
I also apply this benchmark using ASP.NET CORE 2.2 too.
Source code
// ASP.NET 4.8
public sealed class AddressSqlRepository
{
public async Task<Address> GetAddressAsync(
string zipCode)
{
using (var connection = CreateConnection())
{
await connection.OpenAsync();
var parameters = new
{
zipCode
};
var sql = @"SELECT a.cod_postal AS ZipCode,
b.des_cidade AS City,
c.des_sigla AS State,
(t.des_tipo_logradouro + ' ' + d.des_logradouro) AS Street,
r.des_bairro AS Neighborhood
FROM dbo.tb_cep AS a
LEFT OUTER JOIN dbo.tb_cidade AS b ON a.cod_cidade = b.cod_cidade
LEFT OUTER JOIN dbo.tb_estado AS c ON a.cod_estado = c.cod_estado
LEFT OUTER JOIN dbo.tb_logradouro AS d ON a.cod_logradouro = d.cod_logradouro
LEFT OUTER JOIN dbo.tb_tipo_logradouro AS t ON a.cod_tipo_logradouro = t.cod_tipo_logradouro
LEFT OUTER JOIN dbo.tb_bairro AS r ON a.cod_bairro = r.cod_bairro
WHERE a.cod_postal = @zipCode";
var data = await connection.QueryFirstOrDefaultAsync<Address>(
sql,
parameters);
connection.Close();
return data;
};
}
public async Task<IEnumerable<Address>> GetAddressRangeAsync(
string start,
string end)
{
using (var connection = CreateConnection())
{
await connection.OpenAsync();
var parameters = new
{
start,
end
};
var sql = @"SELECT a.cod_postal AS ZipCode,
b.des_cidade AS City,
c.des_sigla AS State,
(t.des_tipo_logradouro + ' ' + d.des_logradouro) AS Street,
r.des_bairro AS Neighborhood
FROM dbo.tb_cep AS a
LEFT OUTER JOIN dbo.tb_cidade AS b ON a.cod_cidade = b.cod_cidade
LEFT OUTER JOIN dbo.tb_estado AS c ON a.cod_estado = c.cod_estado
LEFT OUTER JOIN dbo.tb_logradouro AS d ON a.cod_logradouro = d.cod_logradouro
LEFT OUTER JOIN dbo.tb_tipo_logradouro AS t ON a.cod_tipo_logradouro = t.cod_tipo_logradouro
LEFT OUTER JOIN dbo.tb_bairro AS r ON a.cod_bairro = r.cod_bairro
WHERE a.cod_postal BETWEEN @start AND @end";
var data = await connection.QueryAsync<Address>(
sql,
parameters);
connection.Close();
return data;
};
}
private SqlConnection CreateConnection()
{
return new SqlConnection(ConfigurationManager.ConnectionStrings["Default"].ConnectionString);
}
}
// ASP.NET CORE
public sealed class AddressSqlRepository
{
private readonly IConfiguration _configuration;
public AddressSqlRepository(
IConfiguration configuration)
{
_configuration = configuration;
}
public async Task<Address> GetAddressAsync(
string zipCode)
{
using (var connection = CreateConnection())
{
await connection.OpenAsync();
var parameters = new
{
zipCode
};
var sql = @"SELECT a.cod_postal AS ZipCode,
b.des_cidade AS City,
c.des_sigla AS State,
(t.des_tipo_logradouro + ' ' + d.des_logradouro) AS Street,
r.des_bairro AS Neighborhood
FROM dbo.tb_cep AS a
LEFT OUTER JOIN dbo.tb_cidade AS b ON a.cod_cidade = b.cod_cidade
LEFT OUTER JOIN dbo.tb_estado AS c ON a.cod_estado = c.cod_estado
LEFT OUTER JOIN dbo.tb_logradouro AS d ON a.cod_logradouro = d.cod_logradouro
LEFT OUTER JOIN dbo.tb_tipo_logradouro AS t ON a.cod_tipo_logradouro = t.cod_tipo_logradouro
LEFT OUTER JOIN dbo.tb_bairro AS r ON a.cod_bairro = r.cod_bairro
WHERE a.cod_postal = @zipCode";
var data = await connection.QueryFirstOrDefaultAsync<Address>(
sql,
parameters);
await connection.CloseAsync();
return data;
};
}
public async Task<IEnumerable<Address>> GetAddressRangeAsync(
string start,
string end)
{
using (var connection = CreateConnection())
{
await connection.OpenAsync();
var parameters = new
{
start,
end
};
var sql = @"SELECT a.cod_postal AS ZipCode,
b.des_cidade AS City,
c.des_sigla AS State,
(t.des_tipo_logradouro + ' ' + d.des_logradouro) AS Street,
r.des_bairro AS Neighborhood
FROM dbo.tb_cep AS a
LEFT OUTER JOIN dbo.tb_cidade AS b ON a.cod_cidade = b.cod_cidade
LEFT OUTER JOIN dbo.tb_estado AS c ON a.cod_estado = c.cod_estado
LEFT OUTER JOIN dbo.tb_logradouro AS d ON a.cod_logradouro = d.cod_logradouro
LEFT OUTER JOIN dbo.tb_tipo_logradouro AS t ON a.cod_tipo_logradouro = t.cod_tipo_logradouro
LEFT OUTER JOIN dbo.tb_bairro AS r ON a.cod_bairro = r.cod_bairro
WHERE a.cod_postal BETWEEN @start AND @end";
var data = await connection.QueryAsync<Address>(
sql,
parameters);
await connection.CloseAsync();
return data;
};
}
private SqlConnection CreateConnection()
{
return new SqlConnection(_configuration.GetConnectionString("Default"));
}
}
#1 Benchmark – Address Collection
Perform a query bringing twenty records from an Address SQL Server table using Dapper.
Winner: ASP.NET 4.8 (19 ms)
#2 Benchmark – One Address
Perform a query bringing one record from an Address SQL Server table using Dapper.
Winner: ASP.NET 4.8 (14 ms)
#3 Benchmark – 97.996 addresses in TXT
Perform a query bringing 97.996 records from an Address SQL Server table using Dapper, and transform them in TXT format.
I have used some methods like String Builder, Reflection, and LINQ.
Winner: ASP.NET 4.8 and ASP.NET CORE 3.0 (1.172 ms)
#4 Benchmark – 477.397 addresses in TXT
Perform the same test as before, but bringing half 477.397 records.
Winner: ASP.NET CORE 3.0 (5.344 ms)
public async Task<string> GetRangeTxtAsync(
string start,
string end)
{
var addresses = await _addressRepository.GetAddressRangeAsync(
start,
end);
string txt;
if (addresses == null || addresses.Count() == 0)
{
txt = "addresses NULL or EMPTY";
}
else
{
var sb = new StringBuilder();
sb.Append(GetColumns<Address>());
sb.Append("\r\n");
foreach (var address in addresses)
{
sb.Append(GetColumns(address));
sb.Append("\r\n");
}
txt = sb.ToString();
}
return txt;
}
private string GetColumns<T>(
T data = default)
{
var sb = new StringBuilder();
var type = typeof(T);
var properties = type.GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo property in properties)
{
if (sb.Length > 0)
{
sb.Append(";");
}
if (data == null)
{
sb.AppendFormat(
"{0}_{1}",
type.Name,
property.Name);
}
else
{
var val = property.GetValue(data);
sb.Append(val == null ? "" : $" {val.ToString()}");
}
}
sb.Append(";");
return sb.ToString();
}
#5 Benchmark – 97.996 addresses in JSON
Perform a query bringing 97.996 records from an Address SQL Server table using Dapper and serializes it in JSON format.
Winner: ASP.NET CORE 2.2 (1.082 ms)
#6 Benchmark – 477.397 addresses in JSON
Perform the same test as before, but bringing half 477.397 records.
Winner: ASP.NET CORE 3.0 (5.023 ms)
#7 Benchmark – Read a 7MB PDF file
I put a 7MB PDF file in the App_Data folder and returned as a download response.
Winner: ASP.NET CORE 2.2 (84 ms)
// ASP.NET 4.8
public HttpResponseMessage GetFileSystemDownload()
{
var path = System.Web.Hosting.HostingEnvironment.MapPath("~/App_Data/Cartilha_do_Idoso.pdf");
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StreamContent(new FileStream(path, FileMode.Open, FileAccess.Read))
};
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = Path.GetFileName(path)
};
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return response;
}
// ASP.NET CORE
public IActionResult GetFileSystemDownload()
{
var path = $@"{_env.ContentRootPath}\App_Data\Cartilha_do_Idoso.pdf";
return new FileStreamResult(new FileStream(path, FileMode.Open, FileAccess.Read), "application/pdf");
}
#8 Benchmark – Read all files located at c:\windows
Perform a query reading all files located at c:\windows folder.
Winner: ASP.NET 4.8 (11 ms)
Remarks
I have repeated all tests thee times em different days and different hours and, all results were the same.
I also performed all tests again in two other developers’ machines and, all results were the same.
An interesting thing about the results of my benchmark is, in some cases, the ASP.NET 4.8 is better than ASP.NET CORE (3.0/2.2) and vice-versa.
The question is, which framework do you think is better using the guidelines for this article?
Thank you for reading 🙂
Articles about ASP.NET CORE:
AppSettings: 6 Ways to Read the Config in ASP.NET CORE 3.0
.NET Core for .NET Developers
IIS: How to Host a .NET Core Application in 10 Steps
Benchmark Index
ASP.NET Core: Saturating 10GbE at 7+ million request/s
Performance Improvements in .NET Core 3.0
Performance Tests / Benchmarking for ASP.NET Core 2.2 Endpoints
Dicas de performance para APIs REST no ASP.NET Core
Teste de performance de aplicações .NET Core com BenchMarkDotNet
Swift vs .NET Core — Benchmark
Dapper vs EF Core Query Performance Benchmarking
Lightweight .NET Core benchmarking with BenchmarkDotNet and dotnet-script
.NET Serialization Benchmark 2019 Roundup
Benchmarking .NET code
Profiling .NET Code with PerfView and visualizing it with speedscope.app
Benchmarking Your .NET Core Code With BenchmarkDotNet
Performance benchmark: gRPC vs. REST in .NET Core 3 Preview 8
The Battle of C# to JSON Serializers in .NET Core 3
C# .NET Core versus Java fastest programs
gRPC performance benchmark in ASP.NET Core 3
