Request and Response Logging — Structured Logs for API Observability

Structured logging transforms API observability. Instead of searching free-text log lines for “user 42 created post 7 in 120ms,” structured logging produces machine-parseable events: { "event": "PostCreated", "userId": "42", "postId": 7, "elapsedMs": 120 }. Log management platforms (Application Insights, Seq, Elastic) can then filter, aggregate, and alert on these fields. Serilog is the standard structured logging library for .NET — it integrates with ASP.NET Core’s ILogger abstraction and adds powerful sinks and enrichers.

Serilog Configuration

// dotnet add package Serilog.AspNetCore
// dotnet add package Serilog.Sinks.Console
// dotnet add package Serilog.Sinks.ApplicationInsights (for production)

// ── Program.cs — configure Serilog ────────────────────────────────────────
Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(builder.Configuration)   // from appsettings
    .Enrich.FromLogContext()                          // picks up LogContext.PushProperty values
    .Enrich.WithMachineName()
    .Enrich.WithEnvironmentName()
    .Enrich.WithProperty("Application", "BlogApp.Api")
    .WriteTo.Console(outputTemplate:
        "[{Timestamp:HH:mm:ss} {Level:u3}] [{CorrelationId}] {Message:lj}{NewLine}{Exception}")
    .CreateLogger();

builder.Host.UseSerilog();

// ── Serilog HTTP request logging (replaces default ASP.NET Core request log) ──
app.UseSerilogRequestLogging(options =>
{
    options.MessageTemplate = "HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000}ms";
    options.EnrichDiagnosticContext = (diagnosticContext, httpContext) =>
    {
        diagnosticContext.Set("UserId",        httpContext.User.FindFirstValue(ClaimTypes.NameIdentifier));
        diagnosticContext.Set("CorrelationId", httpContext.Items["CorrelationId"]?.ToString());
        diagnosticContext.Set("UserAgent",     httpContext.Request.Headers.UserAgent.ToString());
        diagnosticContext.Set("RemoteIp",      httpContext.Connection.RemoteIpAddress?.ToString());
    };
    // Exclude health check endpoints from request logs (too noisy)
    options.GetLevel = (ctx, elapsed, ex) =>
        ctx.Request.Path.StartsWithSegments("/health") ? LogEventLevel.Verbose
        : ex is not null || ctx.Response.StatusCode >= 500 ? LogEventLevel.Error
        : ctx.Response.StatusCode >= 400 ? LogEventLevel.Warning
        : LogEventLevel.Information;
});
Note: UseSerilogRequestLogging() should be placed early in the middleware pipeline — before routing, authentication, and controllers — so it captures all requests including those rejected by authentication. Place it after correlation ID middleware so the correlation ID is in the log context when the request log entry is written at the end of the request. Serilog’s request logging fires once per request after the response is complete, producing a single clean log entry per request rather than multiple partial entries.
Tip: Use ILogger with structured parameters everywhere in services and controllers: _logger.LogInformation("Post {PostId} published by user {UserId} in {ElapsedMs}ms", post.Id, userId, elapsed). Serilog captures PostId, UserId, and ElapsedMs as separate structured fields. Log management platforms can then query: “show me all slow publish operations by user 42 in the last hour” — this is only possible with structured logging. Plain string interpolation ($"Post {post.Id} published") produces flat text that cannot be queried by field.
Warning: Never log request bodies or response bodies in production logging without sanitisation. Requests may contain passwords, tokens, or personal data; responses may contain sensitive user information. If you need request body logging for debugging, implement it only in Development environment with explicit field masking for known sensitive fields. Enable it via a feature flag that can be turned on temporarily in production for specific investigation, not always-on.

appsettings.json Serilog Configuration

// ── appsettings.json ────────────────────────────────────────────────────
// {
//   "Serilog": {
//     "MinimumLevel": {
//       "Default": "Information",
//       "Override": {
//         "Microsoft": "Warning",
//         "Microsoft.EntityFrameworkCore.Database.Command": "Information",
//         "System": "Warning"
//       }
//     },
//     "WriteTo": [
//       { "Name": "Console" }
//     ],
//     "Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"]
//   }
// }

// ── appsettings.Production.json ──────────────────────────────────────────
// {
//   "Serilog": {
//     "WriteTo": [
//       {
//         "Name": "ApplicationInsights",
//         "Args": {
//           "connectionString": "InstrumentationKey=...",
//           "telemetryConverter": "Serilog.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
//         }
//       }
//     ]
//   }
// }

Common Mistakes

Mistake 1 — Using string interpolation in log messages (breaks structured logging)

❌ Wrong — structured fields lost:

logger.LogInformation($"Post {postId} created");   // postId is embedded in the string!

✅ Correct — logger.LogInformation("Post {PostId} created", postId) — Serilog captures PostId as a searchable field.

Mistake 2 — Logging health check requests (fills logs with noise)

❌ Wrong — Kubernetes probes hit /health every 10 seconds; 6 log entries per minute of noise.

✅ Correct — configure GetLevel to return Verbose for /health paths so they are not logged at Information level.

🧠 Test Yourself

Why does logger.LogInformation("Post {PostId} created by {UserId}", postId, userId) produce better observability than logger.LogInformation($"Post {postId} created by {userId}")?