Structured Logging

Why `console.log("User " + id + " failed")` is impossible to search at scale.

The idea

When investigating a bug, you search your log database (like Datadog or Splunk) to see what happened. If you write logs as raw text strings, searching is a nightmare. Finding all failed payments for "User 42" requires writing complex, fragile Regular Expressions. Instead, modern systems use Structured Logging. They output logs as machine-readable JSON objects. Instead of searching a string, you query the database exactly like SQL: WHERE user_id = 42 AND action = 'payment'. It is instantly searchable, indexable, and graphable.

Step 1: Unstructured Logging. The app combines variables into a long, messy string.

How it works (JSON Logs)

Never concatenate variables into strings. Use a logging library (like Winston or Pino in Node, or Logrus in Go) that takes a message string and a separate JSON object containing the variables. The library automatically adds standard fields (timestamp, server ID, log level) and formats it as JSON before sending it to the log aggregator.

// BAD: Unstructured text
console.log(`Payment failed for user ${user.id} on item ${item.sku}. Reason: ${err}`);
// Outputs: "Payment failed for user 99 on item ABC. Reason: Insufficient funds"

// GOOD: Structured JSON
logger.error("Payment failed", {
    user_id: user.id,
    sku: item.sku,
    error: err.message,
    action: "checkout"
});
// Outputs: {"level":"error", "msg":"Payment failed", "user_id":99, "sku":"ABC", "error":"Insufficient funds"}

Cost

JSON logs use significantly more disk space than raw strings because they repeat the keys ("user_id") on every single line. This increases your logging bill. However, the engineering time saved during an active outage by being able to click a button to graph "Errors grouped by SKU" massively outweighs the cost of the extra disk space.

Watch out for