Composer attachments — take a photo, attach a file

What the person sees

Every composer has two buttons beside the microphone, photo first: Take photo and Attach file. This holds for Blazor, the React portal, and the React Native app on iOS, Android, Expo web and the Mac app. Taking a photo or picking a file adds a chip above the input. The chip shows a thumbnail for an image, an icon for any other file, and a spinner while the file uploads. Each chip can be removed. A failed upload stays on its chip with the reason, and the send waits until that chip is removed. A message may be sent with attachments and no text, in which case the file names stand in as the text. The chips clear once the send is confirmed. In Blazor, which shows a failure as one dismissible line rather than a failed chip, the send waits until that line is dismissed, and a failed submission puts the uploaded chips back.

One config, one model

The declared config is COMPOSER_ATTACHMENTS in clients/react/src/controls/composerAttachments.ts. It holds the action pair (photo/camera/chat.takePhoto, then file/file/chat.attachFile), the 2048 px long edge, JPEG quality 0.85, the 20 MB cap and at most 10 attachments per message. Blazor mirrors the same constants in src/MeshWeaver.Blazor.Chat/ComposerAttachments.cs, and ComposerAttachmentsTest.TheConfig_MirrorsTheSharedTypeScriptSource fails when the two drift.

useComposerAttachments(ops, platform) is the model every React leaf uses. It covers pick, prepare, upload, chip, references and clear. A leaf contributes only its platform, which opens the camera or the picker, and its rendering of the buttons and chips.

Client Camera File Downscale
iOS / Android (rnAttachments.ts) expo-image-picker launchCameraAsync expo-document-picker expo-image-manipulator
Expo web, the Mac app (WKWebView) getUserMedia capture sheet (RN Modal + <video>) <input type=file> canvas (webAttachments.ts)
React portal (threadChat.tsx) the same getUserMedia sheet (Fluent Dialog) <input type=file> canvas (webAttachments.ts)
Blazor (ThreadChatView) getUserMedia sheet (FluentDialog + ThreadChatView.razor.js) FluentInputFile the photo is scaled in the browser; a picked file is sent as it is

An image is re-encoded as JPEG only when it is larger than 2048 px on the long edge, heavier than 1.5 MB, or in a format a model cannot read, such as HEIC. A GIF or an SVG is never flattened.

Where the bytes go, and what the agent receives

upload  {home}/content/attachments/{yyyy-MM}/{yyyyMMdd-HHmmss}-{nonce}-{name}
send    ThreadSubmitOptions.attachments = ["{home}/content:attachments/…"]

{home} is the viewer's own partition (ops.userId, or the circuit user in Blazor). It is the one place every signed-in person may write, so a photo taken on a read-only page can still be attached, and no one else gains read access to it. The React clients upload through POST /api/mesh/upload (ops.uploadContent). Blazor uploads through hub.SaveContentFile(home) as the circuit's viewer. In both cases the owning hub's write gate decides whether the upload is allowed.

The attachments travel on the user message (ThreadMessage.Attachments) to AgentChatClient.AppendContextAndAttachmentsAsync. A {node}/content:{file} reference with an image or PDF extension becomes a DataContent that the model sees as the picture or document. Any other file is read as text through the mesh Get.

🚨 The bytes are read from the node that OWNS them (AttachmentContentReader, through ContentFileResolver). The round runs on the thread's hub, and before this change it read every content: attachment from that hub's own collection. It parsed the node prefix and then ignored it, so a photo kept in the viewer's home reached the model as nothing, and no log line said so. A binary attachment that resolves to no bytes now logs a warning.

Platform permissions