A Name That Does Not Resolve Is Not Transient

A retry exists to give a transient failure another chance. A hostname that does not exist will not start existing, so retrying it is not resilience — it is three guaranteed failures, three log lines and a delayed answer.

What was measured

The control instance, 2026-09-17 (#4613). Two outbound fetches — an agent reading a company's website through the WebSearch plugin's FetchWebPage tool — failed like this:

fail: Polly[3]
  Execution attempt. Source: '-standard//Standard-Retry', Result: 'Name or service not known
  (www.boss-software.ch:443)', Handled: 'True', Attempt: '3'
  System.Net.Http.HttpRequestException: Name or service not known (www.boss-software.ch:443)
   ---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known

Attempt: '3' is the whole story: the standard predicate handles every HttpRequestException, so each URL was tried three times.

Whose defect this is — measured, not assumed

The issue offered three environmental explanations (cluster DNS, an egress NetworkPolicy, a pod with no route). All three are ruled out from outside the cluster. Resolved from a laptop on the public internet, 2026-09-17:

Neither apex domain exists, so nothing inside the cluster is broken and "confirm whether those hostnames resolve from inside the cluster" never needed asking. Their half is a URL in somebody's data that points at a hostname that does not exist; no code change fixes that, and neither hostname appears anywhere in MeshWeaver, MeshWeaver.Plugins or Memex, so nothing we ship put it there.

Our half, and why it is worth fixing

Three attempts, each logged at Error under category Polly — and Error lines are what the fleet's log watcher folds into a LogIncident and a GitHub ticket. So a bad URL in customer data manufactured a platform defect report. The incident pipeline could not tell "our software is broken" from "somebody typed a hostname that does not exist", and the second is not ours to fix.

Memex.Portal.ServiceDefaults therefore excludes a name-resolution failure from the default pipeline's retry predicate and from its circuit breaker:

• Retry — no attempt can make a name resolve, so the second and third are pure cost.
• Breaker — the default pipeline is shared by every client that does not name its own, so counting a dead hostname as a failure lets one bad URL push the breaker toward open for calls that have nothing to do with it. A name that does not resolve says nothing about the health of any endpoint.

Excluding it from ShouldHandle also drops the log level: Polly logs an unhandled outcome far below Error, so the incident stops being manufactured at its source rather than being filtered downstream.

🚨 Exactly one socket error, and the control matters more than the case

SocketError.HostNotFound only.

SocketError.TryAgain (EAI_AGAIN) is a nameserver that failed to answer — genuinely transient, and excluding it would turn a DNS hiccup into a hard failure, which is the opposite defect and a worse one. ConnectionRefused is a host that exists and is not listening: about the service. ConnectionReset, TimedOut and every TLS failure other than a rejected certificate (below) are about the network or the peer. All of them keep being retried exactly as before.

ServiceDefaults.NameDoesNotResolve walks the inner-exception chain, because HttpClient wraps the resolver's SocketException in an HttpRequestException and a handler pipeline can wrap that again. It is pure, and NonexistentHostIsNotRetriedTest asserts it directly — a predicate that can only be exercised through a real failed connection is a predicate nobody checks.

A rejected certificate is the second deterministic failure

#5910: an agent fetch of a site whose certificate had expired (NotTimeValid) was attempted three times. No retry makes an expired or untrusted certificate valid, and one site's certificate says nothing about any other host the client calls, so it is excluded from the retry and the breaker on the same two grounds as a dead name.

🚨 The signal is raised where the verdict is made, never parsed afterwards. A bare AuthenticationException is not it: SslStream raises other handshake failures in exactly that shape — the peer closing the transport stream mid-handshake is one — and those are transient. A first attempt keyed on "an AuthenticationException with no inner exception" was withdrawn for that reason. Instead the defaults install a certificate validation callback on every primary handler that has none of its own. It keeps the runtime's verdict (accept exactly when SslPolicyErrors is None) and reports a rejection by throwing RemoteCertificateRejectedException (in MeshWeaver.Mesh.Contract), which reaches the caller as the inner exception of the HttpRequestException and carries the policy errors and the chain status. ServiceDefaults.IsDeterministicTransportFailure is the one predicate both the retry and the breaker read.

A client that installs its own validation callback or its own primary handler keeps it, and gets no marker — its rejections are retried as before. That holds whether the client is registered before or after the defaults: the defaults' action runs on the handler the factory built for them, and a client's own primary-handler factory replaces that handler afterwards, so the marker never reaches the client's handler (pinned in both orders). RejectedCertificateIsNotRetriedTest runs the real defaults against a loopback TLS listener: an expired certificate is contacted once and arrives typed, and the control — a server that drops the handshake — is still retried.

A remote site giving up is not a platform incident

The two carve-outs above stop retrying what no retry can fix. They leave a third shape untouched on purpose: a TLS handshake reset, a refused connect, an EAGAIN at connect. Those ARE transient-shaped, so the pipeline keeps retrying them, and when the last retry fails Polly reports the event at Error.

fail: Polly[3]
  Execution attempt. Source: '-standard/(unnamed)/Standard-Retry', Result: 'The SSL connection could
  not be established, see inner exception.', Handled: 'True', Attempt: '3'

The fleet's log watcher folds every Error line into a LogIncident. For a client that fetches whatever URL an agent or a document hands it, every remote site's bad day became a ticket against the platform: the one log site of #7085 fanned out into 46 fingerprints.

The rule (policy arbitrary-url-fetch-failure-is-warning): a named client that fetches arbitrary, user-supplied URLs logs the pipeline's give-up at Warning. Every other client keeps Error, because a platform or service client giving up is an incident.

A client declares itself; the host keeps no list. The module that registers the client says what it fetches:

services.AddHttpClient(WebSearchPlugin.HttpClientName);
services.DeclareArbitraryUrlFetcher(WebSearchPlugin.HttpClientName);

DeclareArbitraryUrlFetcher (in MeshWeaver.Mesh.Contract) registers an IArbitraryUrlFetcher. The host's resilience defaults install a Polly TelemetryOptions.SeverityProvider that reads the sending client's name off the request (the stamp ClientNameOf already reads, #4528) and lowers an Error to Warning only when a declaration names that client. Three details:

• Only Error is lowered. Critical passes unchanged, and so does every severity of an undeclared client.
• The default, unnamed client never qualifies. Nothing can declare it, so a fetcher that still rides the default client keeps Error. Giving a fetcher its own named client is the first step.
• The declarations are read when the event fires. A module swapped live contributes its declaration through a forwarder, so nothing is captured at boot.

The level is a cost decision, not a cosmetic one. An Error line costs a person: an incident, triage and a ticket. A Warning still ships to Loki and still counts in a rate, so a site that fails constantly stays visible, but it no longer claims that the platform is broken. The caller still reports the failure in words the model can act on (the per-URL answers in MeshWeaver.AI.WebSearch).

AnArbitraryUrlFetcherGivingUpIsAWarningTest (core, MeshWeaver.Hosting.Test) builds the real defaults and reads the level off the Polly line. A declared client's final attempt is Warning. The negative control is an undeclared platform client and the unnamed client failing the same way: both are still Error.

The half this does not reach

The caller is still not told in words it can act on. FetchWebPageCore already draws the right line for an HTTP status — a 404 is reported to the model as an answer rather than raised as a fault — but a dead hostname still arrives as a raw Name or service not known, and the person who owns the data that holds the URL learns nothing at all. That is a separate change in MeshWeaver.AI.WebSearch, and the principle it should follow is the one already written at that call site: a fact about the URL is an answer, not a malfunction.

Related

• Error Propagation & Wedges — what a fault must surface as, and where swallowing one costs a wedge.
• An Unreachable Store Is Not a Refusal — the same distinction one layer in: an availability failure is not a verdict.