|
| 1 | +# MMS Attachment Support — Design Spec |
| 2 | + |
| 3 | +## Problem |
| 4 | + |
| 5 | +The Android app now forwards MMS attachments (as base64-encoded data) when receiving MMS messages via `HttpSmsApiService.receive()`. The API server needs to: |
| 6 | + |
| 7 | +1. Accept attachment data in the receive endpoint |
| 8 | +2. Upload attachments to cloud storage (GCS or in-memory) |
| 9 | +3. Store download URLs in the Message entity |
| 10 | +4. Serve a download endpoint for retrieving attachments |
| 11 | +5. Include attachment URLs in webhook event payloads |
| 12 | + |
| 13 | +## Approach |
| 14 | + |
| 15 | +**Approach A: Storage Interface + Minimal New Code** — Add an `AttachmentStorage` interface with GCS and memory implementations. Upload logic lives in the existing `MessageService.ReceiveMessage()` flow (synchronous). A new `AttachmentHandler` serves downloads. No new database tables — content type is encoded in the URL file extension. |
| 16 | + |
| 17 | +## Design |
| 18 | + |
| 19 | +### 1. Storage Interface |
| 20 | + |
| 21 | +**New file: `pkg/repositories/attachment_storage.go`** |
| 22 | + |
| 23 | +```go |
| 24 | +type AttachmentStorage interface { |
| 25 | + Upload(ctx context.Context, path string, data []byte) error |
| 26 | + Download(ctx context.Context, path string) ([]byte, error) |
| 27 | + Delete(ctx context.Context, path string) error |
| 28 | +} |
| 29 | +``` |
| 30 | + |
| 31 | +**GCS Implementation** (`pkg/repositories/gcs_attachment_storage.go`): |
| 32 | + |
| 33 | +- Uses `cloud.google.com/go/storage` SDK |
| 34 | +- Configured with bucket name from `GCS_BUCKET_NAME` env var |
| 35 | +- Stores objects at: `attachments/{userID}/{messageID}/{index}/{name}.{ext}` |
| 36 | +- Extension derived from content type (e.g., `image/jpeg` → `.jpg`); falls back to `.bin` only when no mapping exists |
| 37 | + |
| 38 | +**Memory Implementation** (`pkg/repositories/memory_attachment_storage.go`): |
| 39 | + |
| 40 | +- `sync.Map`-backed in-memory store |
| 41 | +- Used when `GCS_BUCKET_NAME` is empty/unset (local dev, testing) |
| 42 | + |
| 43 | +**DI selection** (in `container.go`): |
| 44 | + |
| 45 | +```go |
| 46 | +if os.Getenv("GCS_BUCKET_NAME") != "" { |
| 47 | + return NewGCSAttachmentStorage(bucket, tracer, logger) |
| 48 | +} |
| 49 | +return NewMemoryAttachmentStorage(tracer, logger) |
| 50 | +``` |
| 51 | + |
| 52 | +### 2. Environment Variables |
| 53 | + |
| 54 | +| Variable | Description | Default | |
| 55 | +| ----------------- | ------------------------------------------------------- | ---------------------------------- | |
| 56 | +| `GCS_BUCKET_NAME` | GCS bucket for attachments. Empty = use memory storage. | `httpsms-86c51.appspot.com` (prod) | |
| 57 | + |
| 58 | +The API base URL for constructing download links is derived from `EVENTS_QUEUE_ENDPOINT` by stripping the `/v1/events` suffix. |
| 59 | + |
| 60 | +### 3. Request & Validation Changes |
| 61 | + |
| 62 | +**Updated `MessageReceive` request** (`pkg/requests/`): |
| 63 | + |
| 64 | +```go |
| 65 | +type MessageReceive struct { |
| 66 | + From string `json:"from"` |
| 67 | + To string `json:"to"` |
| 68 | + Content string `json:"content"` |
| 69 | + Encrypted bool `json:"encrypted"` |
| 70 | + SIM entities.SIM `json:"sim"` |
| 71 | + Timestamp time.Time `json:"timestamp"` |
| 72 | + Attachments []MessageAttachment `json:"attachments"` // NEW |
| 73 | +} |
| 74 | + |
| 75 | +type MessageAttachment struct { |
| 76 | + Name string `json:"name"` |
| 77 | + ContentType string `json:"content_type"` |
| 78 | + Content string `json:"content"` // base64-encoded |
| 79 | +} |
| 80 | +``` |
| 81 | + |
| 82 | +**Updated `MessageReceiveParams`** (`pkg/services/`): |
| 83 | +The `ToMessageReceiveParams()` method must propagate attachments to the service layer: |
| 84 | + |
| 85 | +```go |
| 86 | +type MessageReceiveParams struct { |
| 87 | + // ... existing fields ... |
| 88 | + Attachments []requests.MessageAttachment // NEW — raw attachment data for upload |
| 89 | +} |
| 90 | +``` |
| 91 | + |
| 92 | +**Filename sanitization:** |
| 93 | +The `Name` field from the Android client must be sanitized to prevent path traversal attacks. Strip all path separators (`/`, `\`), directory traversal sequences (`..`), and non-printable characters. If the sanitized name is empty, use a fallback like `attachment-{index}`. |
| 94 | + |
| 95 | +**Content type allowlist:** |
| 96 | +Only allow known-safe MIME types from the extension mapping table (Section 5). Reject attachments with unrecognized content types with a 400 error. |
| 97 | + |
| 98 | +**Validation rules** (in `pkg/validators/`): |
| 99 | + |
| 100 | +- Attachment count must be ≤ 10 |
| 101 | +- Each decoded attachment must be ≤ 1.5 MB (1,572,864 bytes) |
| 102 | +- Content type must be in the allowlist |
| 103 | +- If any limit is exceeded → **reject entire request with 400 Bad Request** |
| 104 | +- Validation happens before any upload or storage |
| 105 | + |
| 106 | +### 4. Upload Flow (Synchronous in Receive) |
| 107 | + |
| 108 | +In `MessageService.ReceiveMessage()`: |
| 109 | + |
| 110 | +1. Validate attachment count, sizes, and content types |
| 111 | +2. Upload attachments **in parallel** using `errgroup`: |
| 112 | + a. Decode base64 content |
| 113 | + b. Sanitize `name` (strip path separators, `..`, non-printable chars; fallback to `attachment-{index}`) |
| 114 | + c. Map `content_type` → file extension (e.g., `image/jpeg` → `.jpg`, unknown → `.bin`) |
| 115 | + d. Upload to storage at path: `attachments/{userID}/{messageID}/{index}/{sanitizedName}.{ext}` |
| 116 | + e. Build download URL: `{apiBaseURL}/v1/attachments/{userID}/{messageID}/{index}/{sanitizedName}.{ext}` |
| 117 | +3. If any upload fails → best-effort delete of already-uploaded files, then return 500 |
| 118 | +4. Collect download URLs into `message.Attachments` (existing `pq.StringArray` field) |
| 119 | +5. Set `Attachments` on `MessagePhoneReceivedPayload` before dispatching event |
| 120 | +6. `storeReceivedMessage()` copies `payload.Attachments` → `message.Attachments` |
| 121 | +7. Store message in database |
| 122 | +8. Fire `message.phone.received` event (includes attachment URLs) |
| 123 | + |
| 124 | +### 5. Content Type → Extension Mapping |
| 125 | + |
| 126 | +A utility function maps MIME types to file extensions: |
| 127 | + |
| 128 | +| Content Type | Extension | |
| 129 | +| ----------------- | --------- | |
| 130 | +| `image/jpeg` | `.jpg` | |
| 131 | +| `image/png` | `.png` | |
| 132 | +| `image/gif` | `.gif` | |
| 133 | +| `image/webp` | `.webp` | |
| 134 | +| `image/bmp` | `.bmp` | |
| 135 | +| `video/mp4` | `.mp4` | |
| 136 | +| `video/3gpp` | `.3gp` | |
| 137 | +| `audio/mpeg` | `.mp3` | |
| 138 | +| `audio/ogg` | `.ogg` | |
| 139 | +| `audio/amr` | `.amr` | |
| 140 | +| `application/pdf` | `.pdf` | |
| 141 | +| `text/vcard` | `.vcf` | |
| 142 | +| `text/x-vcard` | `.vcf` | |
| 143 | +| _(default)_ | `.bin` | |
| 144 | + |
| 145 | +This covers common MMS content types. New mappings can be added as needed. |
| 146 | + |
| 147 | +### 6. Download Handler |
| 148 | + |
| 149 | +**New file: `pkg/handlers/attachment_handler.go`** |
| 150 | + |
| 151 | +**Route:** `GET /v1/attachments/:userID/:messageID/:attachmentIndex/:filename` |
| 152 | + |
| 153 | +- Registered with **both** `AuthenticatedMiddleware` (Firebase bearer) **and** `APIKeyMiddleware` — so both the end-user (via Firebase token) and webhook consumers (via API key) can download attachments |
| 154 | +- Authenticates that `:userID` matches the authenticated user's ID (from either auth method) |
| 155 | +- Returns 401 if mismatch |
| 156 | + |
| 157 | +**Download flow:** |
| 158 | + |
| 159 | +1. Parse URL params (userID, messageID, attachmentIndex, filename) |
| 160 | +2. Verify authenticated user ID matches `:userID` |
| 161 | +3. Construct storage path: `attachments/{userID}/{messageID}/{attachmentIndex}/{filename}` |
| 162 | +4. Fetch bytes from `AttachmentStorage.Download(ctx, path)` |
| 163 | +5. Derive `Content-Type` from filename extension |
| 164 | +6. Set security headers: `Content-Disposition: attachment`, `X-Content-Type-Options: nosniff` |
| 165 | +7. Respond with binary data + correct `Content-Type` header |
| 166 | +8. Return 404 if attachment not found in storage |
| 167 | + |
| 168 | +### 7. Webhook Event Changes |
| 169 | + |
| 170 | +**Updated `MessagePhoneReceivedPayload`** (`pkg/events/message_phone_received_event.go`): |
| 171 | + |
| 172 | +```go |
| 173 | +type MessagePhoneReceivedPayload struct { |
| 174 | + MessageID uuid.UUID `json:"message_id"` |
| 175 | + UserID entities.UserID `json:"user_id"` |
| 176 | + Owner string `json:"owner"` |
| 177 | + Encrypted bool `json:"encrypted"` |
| 178 | + Contact string `json:"contact"` |
| 179 | + Timestamp time.Time `json:"timestamp"` |
| 180 | + Content string `json:"content"` |
| 181 | + SIM entities.SIM `json:"sim"` |
| 182 | + Attachments []string `json:"attachments"` // NEW — download URLs |
| 183 | +} |
| 184 | +``` |
| 185 | + |
| 186 | +Webhook subscribers will receive the array of download URLs. They can `GET` each URL with their **API key** (via `x-api-key` header) or **bearer token** to download the attachment. |
| 187 | + |
| 188 | +### 8. Files Changed / Created |
| 189 | + |
| 190 | +**New files:** |
| 191 | + |
| 192 | +- `pkg/repositories/attachment_storage.go` — Interface definition |
| 193 | +- `pkg/repositories/gcs_attachment_storage.go` — GCS implementation |
| 194 | +- `pkg/repositories/memory_attachment_storage.go` — Memory implementation |
| 195 | +- `pkg/handlers/attachment_handler.go` — Download endpoint handler |
| 196 | +- `pkg/validators/attachment_handler_validator.go` — Download param validation |
| 197 | + |
| 198 | +**Modified files:** |
| 199 | + |
| 200 | +- `pkg/requests/message_receive.go` (or wherever `MessageReceive` is defined) — Add `Attachments` field |
| 201 | +- `pkg/validators/message_handler_validator.go` — Add attachment count/size validation |
| 202 | +- `pkg/services/message_service.go` — Add upload logic to `ReceiveMessage()` |
| 203 | +- `pkg/events/message_phone_received_event.go` — Add `Attachments` field to payload |
| 204 | +- `pkg/di/container.go` — Wire storage, new handler, pass storage to message service |
| 205 | +- `api/.env.docker` — Add `GCS_BUCKET_NAME` variable |
| 206 | +- `go.mod` / `go.sum` — Add `cloud.google.com/go/storage` dependency |
| 207 | + |
| 208 | +### 9. Validation Constraints |
| 209 | + |
| 210 | +| Constraint | Value | Behavior | |
| 211 | +| ------------------------------- | ------------------------ | ---------------------------------------------------- | |
| 212 | +| Max attachment count | 10 | 400 Bad Request | |
| 213 | +| Max attachment size (decoded) | 1.5 MB (1,572,864 bytes) | 400 Bad Request | |
| 214 | +| Content type not in allowlist | — | 400 Bad Request | |
| 215 | +| Missing/empty attachments array | — | Message stored without attachments (normal SMS flow) | |
| 216 | + |
| 217 | +### 10. Error Handling |
| 218 | + |
| 219 | +- Storage upload failure → Best-effort delete of already-uploaded attachments, then return 500; message is NOT stored |
| 220 | +- Storage download failure → Return 404 or 500 depending on error type |
| 221 | +- Invalid base64 content → Return 400 Bad Request |
| 222 | +- UserID mismatch on download → Return 401 Unauthorized |
| 223 | +- All errors wrapped with `stacktrace.Propagate()` per project convention |
0 commit comments