Logs and Cache Signals, Version One
1. Decision
The site did not need a heavyweight observability design before making progress. The proposed first step was a one-page implementation boundary followed by development. The immediate problem was being able to understand, replay, and locate online failures. A full platform combining logs, metrics, alerts, and tracing was beyond the needs of this first version.
2. Goals
The first version aimed to give the web app and scripts one logging entry point, keep recent logs for three days, make production errors easier to locate, and expose whether server-side caches were being hit. Its purpose was practical debugging and collaboration on bug fixes, not enterprise-scale observability.
3. Included and Excluded Work
Included: a shared logger in place of scattered console.log and console.error, structured JSON-like output, basic web-request context, script names and run IDs, processing summaries, uncaught exceptions and rejected promises, cache events (hit, miss, set, stale), and three-day retention.
Deferred: Sentry, Datadog, OpenTelemetry, Prometheus and Grafana, distributed tracing, alert-platform integration, and complex dashboards.
4. Log Record Design
The web app and scripts should use the same logger wrapper. Records should be machine-readable so they can later be filtered and aggregated. Every record should include timestamp, level, message, module, and env. Error records should additionally include error_name, error_message, and stack.
Useful web context includes path, locale, slug, duration_ms, and cache_namespace. Script context includes script, run_id, job, target, processed_count, success_count, and failed_count. The proposed levels were debug, info, warn, and error: local development could show debug logs, while production would normally start at info.
5. Request Logs
A first version did not require a complex middleware framework. It did need a request-start record with method, path, a summary of query parameters, and request ID. The request-end record should include method, path, status code, duration, and cache-hit status. Rendered pages could also include page type, entity type, and entity slug.
6. Error Logs
The shared error path should cover API-handler exceptions, server component or server-action exceptions, uncaught top-level script errors, and promise rejections. One-line strings are insufficient: the record needs the error type, message, stack, module, and request or job context. Otherwise operators may know that a request failed without being able to locate the cause.
7. Cache Events
The first version could log cache behavior as structured events without building a separate metrics stack. Each hit, miss, set, or stale event should carry cache_namespace, cache_key, cache_event, ttl_seconds, stale_at, and regenerated where relevant.
Those events should answer practical questions: Was a slow page caused by a cache miss? Is a key always missing? Is a cache rebuilding too often? Does stale handling work as expected?
8. Implementation Path
A shared lib/logger could expose logger.debug, .info, .warn, and .error. Cache instrumentation belongs in the common server-cache wrapper rather than scattered page code, reducing missed cases and making a later metrics implementation easier. Import, sync, and repair scripts should create a run_id at startup and attach it throughout the run.
9. Storage
Local or server-side JSON Lines files were sufficient for the first version, rolled daily and retained for three days. In Docker, stdout and stderr could be collected, but structured records remained useful.
10. Why This Scope Was Enough
The immediate need was to know what a request did, what a script did, and whether a cache worked. The recommendation was a shared logger, consistent request and error records, cache events in one wrapper, run IDs for scripts, and short local retention. These were proposed implementation boundaries, not a claim that every field and retention rule was already deployed.