<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Slo on Elastocera</title>
    <link>https://elastocera.com/tags/slo/</link>
    <description>Recent content in Slo on Elastocera</description>
    <image>
      <title>Elastocera</title>
      <url>https://elastocera.com/images/forest-og.jpg</url>
      <link>https://elastocera.com/images/forest-og.jpg</link>
    </image>
    <generator>Hugo -- 0.157.0</generator>
    <language>en</language>
    <lastBuildDate>Fri, 05 Jun 2026 10:00:00 -0300</lastBuildDate>
    <atom:link href="https://elastocera.com/tags/slo/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>The Dashboard Illusion</title>
      <link>https://elastocera.com/posts/dashboard-illusion-comprehension-ceiling/</link>
      <pubDate>Fri, 05 Jun 2026 10:00:00 -0300</pubDate>
      <guid>https://elastocera.com/posts/dashboard-illusion-comprehension-ceiling/</guid>
      <description>Observability has solved detection. It has not solved understanding. The gap between the two has a structure, and a calculable ceiling above which more dashboards produce less clarity.</description>
        <enclosure url="https://elastocera.com/images/mantis-shrimp-og.jpg" length="0" type="image/jpeg"/>
      <content:encoded><![CDATA[<p>Observability is described as understanding the system.</p>
<p>It is detection.</p>
<p>The distinction is not academic. It is the difference between knowing that a signal exists and knowing what the signal means about the platform that produced it. Detection has been industrialized over the past decade. Understanding has not. Most of the friction during incidents lives in the gap between them.</p>
<p>This article is not an argument against observability. The detection capability the industry has built is real and valuable. The argument is that detection has been mistaken for comprehension, and that the conflation has a measurable cost.</p>
<hr>
<h3 id="the-detection-achievement">The Detection Achievement</h3>
<p>What observability has solved is significant.</p>
<p>Distributed tracing, standardized through <span class="tooltip-term" data-tooltip="OpenTelemetry: a vendor-neutral framework and set of APIs, SDKs, and tools for instrumenting, generating, collecting, and exporting telemetry data (metrics, logs, and traces). It became a CNCF graduated project in 2024 and is the de facto standard for cross-service observability."> OpenTelemetry </span>, makes request paths visible across services that were opaque to operators a decade ago. Structured logging turns unsearchable text into queryable data. High-cardinality metrics let operators slice system behavior by attributes that previously required manual correlation across multiple tools. Real-time aggregation pipelines deliver signals in seconds rather than minutes.</p>
<p>Each of these is a real engineering achievement. None should be dismissed.</p>
<p>The <span class="tooltip-term" data-tooltip="SLO/SLI framework: a methodology for measuring service reliability, formalized in Google&#39;s Site Reliability Engineering book (Beyer, Jones, Petoff, Murphy, 2016). SLI (Service Level Indicator) is the metric. SLO (Service Level Objective) is the target. The framework converts raw telemetry into operational targets that map to user experience."> SLO/SLI framework </span> popularized by the Google SRE book has given platform teams a vocabulary for converting raw telemetry into operational targets. Vendors have built mature commercial products around it. Open-source alternatives have caught up. CNCF currently lists more than one hundred observability projects in its landscape.</p>
<p>This abundance is the achievement. It is also where the second problem begins.</p>
<hr>
<h3 id="the-comprehension-gap">The Comprehension Gap</h3>
<p>Detection produces signals. Comprehension produces a model of the system that the signals are about. The two are different cognitive operations and require different inputs.</p>
<p>A dashboard showing rising latency on a service is a signal. The understanding that the latency is rising because a certificate rotation triggered a connection pool reset, which is exhausting capacity on a downstream service that was already under load from a batch job, is a model. The dashboard does not produce the model. It produces the input from which a model can, with effort, be constructed.</p>
<p>Industry surveys consistently confirm that the friction is in the second step. The Honeycomb State of Observability surveys, published annually since 2021, repeatedly find that organizations have between five and fifteen distinct observability tools. The New Relic Observability Forecast finds that despite increased investment in tooling, mean time to resolution has not improved at the rate the investment would predict. The pattern is consistent across vendors, geographies, and industries: more telemetry has not produced proportional gains in operational understanding (<a href="https://elastocera.com/field-notes/abstractions-simplify-usage-not-operation/" class="fn-ref" title="Abstractions Simplify Usage, Not Operation">FN-0006</a>).</p>
<p>The reason is not the tooling. It is that detection scales technically. Comprehension scales cognitively. The two scale at different rates, and they reach different limits.</p>
<blockquote>
<p>Detection is a property of the system. Comprehension is a property of the operator.</p>
</blockquote>
<hr>
<h3 id="the-comprehension-ceiling">The Comprehension Ceiling</h3>
<p>There is a point above which adding more telemetry does not increase understanding. Past that point, each additional signal degrades the operator&rsquo;s ability to construct a coherent model of the system. The point is cognitive, not technical. It is where the operator&rsquo;s capacity to integrate signals reaches its limit.</p>
<p>This point is the <strong>Comprehension Ceiling</strong>, and it is calculable from three inputs:</p>
<ul>
<li>
<p><strong>Signal cardinality</strong>: the number of distinct signals the operator must consider. This includes metrics, logs, traces, alerts, and dashboards across every tool the team uses. A single service exposed through multiple tools counts more than once, because each tool requires separate cognitive effort to interpret.</p>
</li>
<li>
<p><strong>Cognitive load per signal</strong>: the mental work required to interpret one signal in isolation. A signal that maps directly to user impact (an SLO burn rate) has low load. A signal that requires translation through multiple layers of context (a Kubernetes pod restart count without service mapping) has high load.</p>
</li>
<li>
<p><strong>Integration capacity</strong>: how many signals the operator can hold in working memory simultaneously while reasoning about their relationships. This is bounded by human cognition, not by tooling. Foundational research in cognitive load theory (Sweller, 1988) places working memory capacity at four to seven items for novel information under stress.</p>
</li>
</ul>
<p>Below the Comprehension Ceiling, additional signals add value. Each is interpretable. Integration is feasible. The operator builds a model that matches the system&rsquo;s actual behavior.</p>
<p>At the ceiling, signals plateau in usefulness. Adding more does not improve the model. The operator is already at capacity.</p>
<p>Above the ceiling, additional signals degrade comprehension. Cognitive load per signal increases as the operator tries to disambiguate similar signals from different tools. Integration breaks down because too many signals are competing for too little working memory. Misclassification rates rise. Alert fatigue, well-documented in both medical and operational literature, becomes structural rather than incidental.</p>
<p>The ceiling is not a fixed number. It varies by operator experience, signal design, and incident pressure. A senior engineer who designed the system has a higher ceiling than a junior engineer on first-night oncall. A signal designed to map cleanly to user impact contributes less load than a raw infrastructure metric. An operator under acute incident stress has a lower ceiling than the same operator in steady-state monitoring.</p>
<blockquote>
<p>The Comprehension Ceiling is where signal abundance becomes signal interference.</p>
</blockquote>
<p><strong>Executive implication:</strong> Ask the platform team how many distinct observability tools the on-call rotation must consult during an incident. If the answer is more than three, the team is operating near or above the Comprehension Ceiling for most of its members. The investment required to push above the ceiling does not come from more tools. It comes from designing for fewer signals with higher meaning per signal.</p>
<hr>
<h3 id="why-more-tools-compound-the-problem">Why More Tools Compound the Problem</h3>
<p>Tooling sprawl is a structural contributor to the Comprehension Ceiling.</p>
<p>Each observability tool brings its own vocabulary, its own naming conventions, its own thresholds, its own visual conventions. An operator working across five tools is not just consulting five sources. They are translating between five ontologies. The translation cost is paid in cognitive load per signal, and it is paid most heavily during incidents, when the cognitive budget is already constrained (<a href="https://elastocera.com/field-notes/operational-knowledge-fragmentation/" class="fn-ref" title="Operational Knowledge Fragmentation">FN-0008</a>).</p>
<p>The translation is invisible in tooling reports. It does not appear as a metric on a dashboard. It manifests as misdiagnoses, missed correlations, and time spent reconstructing context that the tools already had but presented in incompatible forms.</p>
<p>This is one of the few places where consolidation, despite its risks documented in <a href="/posts/cost-optimization-risk-concentration-hosted-control-planes/">Cost Optimization vs Risk Concentration in Hosted Control Planes</a>, has a clear comprehension benefit. Reducing the number of distinct tools an operator must consult lowers cognitive load per signal. The reduction is not free, and the consolidation has structural risk implications, but the cognitive math is straightforward.</p>
<blockquote>
<p>Tools that share a vocabulary share their comprehension budget. Tools that do not, compete for it.</p>
</blockquote>
<hr>
<h3 id="designing-for-comprehension-not-just-detection">Designing for Comprehension, Not Just Detection</h3>
<p>Detection is now a solved problem in most organizations. Comprehension is a design problem that has not received the same attention.</p>
<p>A small set of practices distinguishes platforms designed for understanding from those designed only for detection.</p>
<p><strong>Reduce cardinality where it costs less than it gives.</strong> Not every metric collected needs to be displayed. Not every dashboard built needs to be consulted. A platform team that audits its observability surface and removes signals that do not consistently inform action is reducing cognitive load without reducing detection.</p>
<p><strong>Build narratives, not just dashboards.</strong> A dashboard shows signals. A narrative shows what those signals mean about a specific aspect of the system. Golden path documentation, named queries that capture diagnostic patterns, and runbooks that tie symptoms to causes are all narratives. They pre-compute parts of the integration that the operator would otherwise do under stress.</p>
<p><strong>Pre-compute integration where possible.</strong> The SLO/SLI framework is a pre-computed integration: it converts many raw signals into a single operational target. SLO burn rate alerts, error budget dashboards, and named composite queries all do similar work. They lift signals up the abstraction stack before the operator engages with them.</p>
<p><strong>Treat dashboards as artifacts, not as comprehension.</strong> A dashboard is a tool for thinking, not the thinking itself. Teams that confuse dashboard quantity for comprehension quality build elaborate detection layers and atrophy in their model-building capacity. The artifact is necessary. It is not sufficient.</p>
<p><strong>Train comprehension explicitly.</strong> Incident drills, game days, and chaos engineering exercises are not only resilience tests. They are deliberate practice for the cognitive operation of building a system model under pressure (<a href="https://elastocera.com/field-notes/the-first-incident-test/" class="fn-ref" title="The First Incident Test">FN-0015</a>). Teams that train comprehension lift the Comprehension Ceiling for individual operators. Teams that do not, depend on whoever happens to be on call having seen the failure mode before.</p>
<hr>
<h3 id="from-visibility-to-understanding">From Visibility to Understanding</h3>
<p>The structural shift required is small in description and large in practice.</p>
<p>Observability adoption is not the same as comprehension capability. The first is technical and well-instrumented. The second is cognitive and rarely tracked. An organization that measures the first without measuring the second is reporting on detection while assuming understanding follows. It usually does not.</p>
<p>The investment that addresses the gap is not larger toolsets. It is fewer signals with higher meaning per signal, narratives that pre-compute integration, and explicit training in the cognitive work of building a model from telemetry. None of this requires net-new technology. All of it requires recognizing that comprehension is a separate engineering discipline that has been hidden inside observability budgets.</p>
<p><strong>Executive implication:</strong> When the next observability budget is reviewed, separate the question of &ldquo;do we detect&rdquo; from the question of &ldquo;do we understand&rdquo;. The first is answered by tooling. The second is answered by what the team can do with the tooling under pressure. The two budgets are not the same line item, even if they share an invoice.</p>
<hr>
<h3 id="architectural-continuity">Architectural Continuity</h3>
<p>Observability has industrialized detection. It has not industrialized understanding. The conflation between the two has a measurable cost, and the cost is paid most heavily at the moment when comprehension matters most: during incidents, where cognitive capacity is already strained.</p>
<blockquote>
<p>Detection produces signals.
Understanding produces a model.
The Comprehension Ceiling is where the two stop matching.</p>
</blockquote>
<p>The mantis shrimp has sixteen types of photoreceptors. Humans have three. Decades of research, including the foundational work from Justin Marshall&rsquo;s lab at the University of Queensland, have shown that despite this sensory abundance, mantis shrimps discriminate colors less precisely than humans. The additional sensors do not produce finer comprehension. They produce faster detection. The platform team that adds dashboards in the name of understanding is making a similar trade without recognizing it. Designing for understanding is a different problem from designing for detection. The first requires engineering the operator&rsquo;s model, not only the system&rsquo;s signals.</p>
<hr>
<h3 id="references">References</h3>
<ol>
<li>
<p>Sweller, John. <a href="https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1202_4">&ldquo;Cognitive Load During Problem Solving: Effects on Learning&rdquo;</a>, Cognitive Science, Volume 12, Issue 2, 1988.</p>
</li>
<li>
<p>Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall Richard. <a href="https://sre.google/sre-book/table-of-contents/">Site Reliability Engineering: How Google Runs Production Systems</a>, O&rsquo;Reilly Media, 2016.</p>
</li>
<li>
<p>Honeycomb, <a href="https://www.honeycomb.io/state-of-observability">&ldquo;State of Observability&rdquo;</a>, annual industry survey.</p>
</li>
<li>
<p>New Relic, <a href="https://newrelic.com/resources/report/observability-forecast/2024">&ldquo;Observability Forecast&rdquo;</a>, 2024.</p>
</li>
<li>
<p>Thoen, Hanne H.; How, Martin J.; Chiou, Tsyr-Huei; Marshall, Justin. <a href="https://www.science.org/doi/10.1126/science.1245824">&ldquo;A Different Form of Color Vision in Mantis Shrimp&rdquo;</a>, Science, Volume 343, 2014.</p>
</li>
</ol>
]]></content:encoded>
    </item>
  </channel>
</rss>
