Skip to content

parseDataRecord() rejects complex SWE component types (Vector, DataArray, Matrix, DataChoice, Geometry) as DataRecord fields #101

@Sam-Bolling

Description

@Sam-Bolling

Task

Handle complex SWE Common component types (Vector, DataArray, Matrix, DataChoice, Geometry) as valid DataRecord fields and DataArray element types — parseDataRecord() and parseElementType() currently throw SweCommonParseError for these spec-valid types.

Finding Reference: Issue #101, discovered via OS4CSAPI/ogc-csapi-explorer#30 — encoding display in schema views
Severity: Moderate
Category: Code bug — spec-conformance gap
Ownership: Ours


Problem Statement

parseDataRecord() in data-record.ts only handles simple scalar components (Quantity, Count, Boolean, Text, Time, Category, Range variants) and nested DataRecords as field types. Any DataRecord field containing a complex aggregate type — Vector, DataArray, Matrix, DataChoice, or Geometry — throws SweCommonParseError with "unsupported component type", even though these are valid SWE Common 3.0 AnyComponent types per OGC 24-014.

The same gap exists in data-array.ts parseElementType(), which throws for Vector, Matrix, DataChoice, and Geometry as element types.

Evidence:

GET /controlStreams/6b6p9mhqg5do/schema
Content-Type: application/swe+json

Response contains a DataRecord with a Vector field:
{
  "type": "DataRecord",
  "label": "FCU_Command",
  "fields": [
    {
      "name": "locationVectorLLA",
      "type": "Vector",
      "label": "Location",
      "referenceFrame": "http://www.opengis.net/def/crs/EPSG/0/4979",
      "coordinates": [
        { "name": "lat", "type": "Quantity", "uom": { "code": "deg" } },
        { "name": "lon", "type": "Quantity", "uom": { "code": "deg" } },
        { "name": "alt", "type": "Quantity", "uom": { "code": "m" } }
      ]
    },
    { "name": "active", "type": "Boolean" },
    { "name": "numWaypoints", "type": "Count" }
  ]
}

parseDataRecord(schemaJson);
// ❌ SweCommonParseError:
//   DataRecord field "locationVectorLLA" has unsupported component type: "Vector"
//   path: fields[0].type

Impact: Any server returning DataRecords with Vector fields (extremely common for UAV/sensor payloads with positional data) causes parse failures. The entire schema parse fails, breaking schema display in downstream consumers.

Root Cause

data-record.ts parseField() (L88–146) and data-array.ts parseElementType() (L92–149) use hard-coded type whitelists. They cannot import parseSWEComponent() from parser.ts because that would create a circular import (parser.tsdata-record.tsparser.ts).

The parseField() in parser.ts (L148–195) already handles all 16 types correctly because it lives in the same file as parseSWEComponent(). The fix needs to bring this capability to data-record.ts and data-array.ts without circular imports.

Files to Create or Modify

File Action Est. Lines Purpose
src/ogc-api/csapi/formats/swecommon/data-record.ts Modify ~15 changed Add optional componentParser callback parameter
src/ogc-api/csapi/formats/swecommon/data-array.ts Modify ~15 changed Same callback pattern for parseElementType()
src/ogc-api/csapi/formats/swecommon/parser.ts Modify ~2 changed Pass parseSWEComponent as callback to parseDataRecord() and parseDataArray()
src/ogc-api/csapi/formats/swecommon/data-record.spec.ts Modify ~30–50 added Tests for complex types as DataRecord fields
src/ogc-api/csapi/formats/swecommon/data-array.spec.ts Modify ~20–30 added Tests for complex types as DataArray element types

Proposed Fix

Option A (Recommended): Callback injection — no circular imports, minimal coupling

Add an optional componentParser callback parameter that parser.ts passes in when calling parseDataRecord() and parseDataArray(). This breaks the circular dependency cleanly:

// data-record.ts
export type ComponentParser = (json: unknown) => AnyComponent;

function parseField(json: unknown, index: number, componentParser?: ComponentParser): DataField | TypedDataField {
  // ...existing validation...
  if (type === 'DataRecord') {
    return { name, component: parseDataRecord(json, componentParser) } as TypedDataField;
  }
  if (SIMPLE_COMPONENT_TYPES.has(type)) {
    return { name, component: parseSimpleComponent(json) } as TypedDataField;
  }
  // Complex types — delegate to injected parser
  if (componentParser) {
    return { name, component: componentParser(json) } as TypedDataField;
  }
  throw new SweCommonParseError(
    `DataRecord field "${name}" has unsupported component type: "${type}". ` +
    `Complex types require a componentParser callback.`,
    `fields[${index}].type`
  );
}
// parser.ts — the only change:
case 'DataRecord':
  return parseDataRecord(json, parseSWEComponent);  // ← pass callback
case 'DataArray':
  return parseDataArray(json, parseSWEComponent);    // ← pass callback

Pros: No circular imports, fully backward-compatible (componentParser is optional), existing standalone callers still work for simple schemas, ~30 line diff.

Option B: Move parseField to a shared module — larger refactor, adds a new file.
Option C: Inline everything into parser.ts — makes parser.ts even larger (already 1440 lines).

Recommended: Option A — smallest diff, backward-compatible, independently validated by the CSAPI Explorer app (commit 5ec3df7).

Scope — What NOT to Touch

  • ❌ Do NOT modify parser.ts parseField() — it already works correctly for all 16 types
  • ❌ Do NOT modify parser.ts parseSWEComponent() — it already dispatches all types correctly
  • ❌ Do NOT modify types.tsAnyComponent, DataField, TypedDataField are already correct
  • ❌ Do NOT modify components.ts — simple component parsing is unrelated
  • ❌ Do NOT modify _helpers.ts — base property parsing is unrelated
  • ❌ Do NOT move parseDataRecord out of data-record.ts into parser.ts (Option C)
  • ❌ Do NOT modify files outside the "Files to Create or Modify" table above
  • ❌ Do NOT refactor existing code unless required to complete this task

Acceptance Criteria

  • data-record.ts parseField() accepts an optional componentParser callback and delegates complex types to it when provided
  • data-record.ts parseDataRecord() accepts and forwards an optional componentParser parameter
  • data-array.ts parseElementType() accepts an optional componentParser callback and delegates complex types to it when provided
  • data-array.ts parseDataArray() accepts and forwards an optional componentParser parameter
  • parser.ts passes parseSWEComponent as the callback when calling parseDataRecord() and parseDataArray()
  • New tests: DataRecord with Vector field parses correctly through parseSWEComponent()
  • New tests: DataRecord with DataArray field parses correctly
  • New tests: DataArray with Vector element type parses correctly
  • Existing "unsupported type" tests still pass (they use 'UnknownType', not valid complex types)
  • Standalone parseDataRecord() calls (without callback) still throw for complex types — backward-compatible
  • All existing tests pass (npm test)
  • No lint errors, tsc --noEmit clean

Dependencies

Blocked by: Nothing
Blocks: Nothing


Operational Constraints

⚠️ MANDATORY: Before starting work on this issue, review docs/governance/AI_OPERATIONAL_CONSTRAINTS.md.

Key constraints:

  • Precedence: OGC specifications → AI Collaboration Agreement → This issue description → Existing code → Conversational context
  • No scope expansion: Fix the finding, nothing more
  • No server-specific branches: Fix must generalize to all complex SWE component types
  • Minimal diffs: Prefer the smallest change that satisfies the acceptance criteria
  • Verify against test suite: All 1,251 CSAPI tests + 724 format tests + 243 SensorML tests must pass

Findings Report

📄 Issue #101 Findings Report — parseDataRecord() Complex Types

Recommendation: FIX — genuine spec-conformance gap. OGC SWE Common 3.0 (24-014) requires DataRecord fields to accept any AbstractDataComponent. Option A is minimal, backward-compatible, and independently validated.


References

# Document What It Provides
1 Issue #101 Findings Report Full analysis, risk assessment, and recommendation
2 docs/governance/AI_OPERATIONAL_CONSTRAINTS.md Mandatory operational constraints
3 src/ogc-api/csapi/formats/swecommon/data-record.ts L88–146 Bug location — parseField() throws for complex types
4 src/ogc-api/csapi/formats/swecommon/data-array.ts L92–149 Same gap — parseElementType() throws for complex types
5 src/ogc-api/csapi/formats/swecommon/parser.ts L148–195 Working pattern — parseField() delegates all types via parseSWEComponent()
6 OGC SWE Common 3.0 (24-014) DataRecord, AnyComponent — spec requires all 16 types as valid fields
7 Explorer fix commit 5ec3df7 Independently validated Option A implementation
8 OS4CSAPI/ogc-csapi-explorer#30 Discovery source — encoding display in schema views

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions