Skip to content

Contact Rollups: remove legacy pre-2017 PD attendance extraction - #74614

Open
carl-codeorg wants to merge 1 commit into
stagingfrom
claude/contact-rollups-remove-legacy-pd-attendance
Open

Contact Rollups: remove legacy pre-2017 PD attendance extraction#74614
carl-codeorg wants to merge 1 commit into
stagingfrom
claude/contact-rollups-remove-legacy-pd-attendance

Conversation

@carl-codeorg

Copy link
Copy Markdown
Contributor

What

Deletes the nightly Contact Rollups extraction over the pre-2017 PD attendance model, plus its processing code.

Before Sept 2017, PD workshop attendance was modeled as sections rows with a non-null section_type joined through followers. Commit 7d3b6ab ("Remove all workshop.section references, switch completely to the new attendance model", 2017-09-06) moved attendance entirely to pd_attendances, and nothing has written a workshop section since — the matching rows have been frozen for eight years. The nightly job still re-scanned usersfollowerssections (no index on section_type; followers is one of the largest tables in the database) every single run to re-extract them.

Marketing confirmed that contacts whose only PD attendance predates the 2017 model are no longer relevant, so this is a delete, not a snapshot.

  • Remove extract_professional_learning_attendance_old_attendance_model and its call in collect_contacts.
  • Rename extract_professional_learning_attendance_new_attendance_modelextract_professional_learning_attendance, since there is only one model now.
  • Remove the dashboard.followers handling in ContactRollupsProcessed: the section_type → course mapping (SECTION_TYPE_INVERTED_MAP) in extract_professional_learning_attended, and the dashboard.followers key check in extract_roles' Teacher rule.

Expected Pardot effect (one-time)

  • Teachers whose attendance list included old-model courses get one update shortening db_Professional_Learning_Attended on the next nightly run.
  • Pardot multi-value fields sync as indexed keys (db_X_0, db_X_1, …) and the delta calculation only sends keys present in new data, so when a list shrinks, trailing indexed values are not cleared in Pardot. Those stale tail values are a known, accepted one-time artifact of dropping this data.
  • Teachers whose only teacher signal was an old-model followers row lose that source for the Teacher role; any surviving role change syncs the same way.

Performance

Removes one of the heaviest collect-phase queries (full scan of sections filtered on unindexed section_type, joined to followers and users) and the corresponding raw-row volume from every downstream step (aggregation, parsing, sync joins).

Testing

  • Updated contact_rollups_processed_test.rb: extract_professional_learning_attended cases now exercise pd_attendances only (including dedup/compact/sort); extract_roles no longer expects a followers-based Teacher role.
  • 32 tests pass across the three contact-rollups test files; the single error in contact_rollups_raw_test.rb (teacher-verification/...png missing from the local test asset pipeline) is a pre-existing local-environment issue, unchanged by this diff.
  • Lint (pre-commit hook) clean.

🤖 Generated with Claude Code

Before Sept 2017, PD workshop attendance was modeled as sections with a
non-null section_type plus followers rows; commit 7d3b6ab switched
entirely to the pd_attendances model and nothing has written a workshop
section since. The nightly contact rollups job still scanned
users x followers x sections (with no index on section_type) every run to
re-extract those frozen rows.

Marketing has confirmed that contacts whose only PD attendance predates
the 2017 model are no longer relevant, so this deletes the extraction and
its processing rather than snapshotting it:

- drop extract_professional_learning_attendance_old_attendance_model and
  rename the "new model" extraction to
  extract_professional_learning_attendance, since there is only one now
- drop the dashboard.followers handling (and SECTION_TYPE_INVERTED_MAP) in
  extract_professional_learning_attended and extract_roles

Expected Pardot effect: teachers whose only attendance came from the old
model get one update shortening db_Professional_Learning_Attended on the
next run. Because Pardot multi-value fields are synced as indexed keys
(db_X_0, db_X_1, ...) and the delta calculation only sends keys present in
new data, a removed trailing value is not cleared in Pardot; the
remaining stale tail values are accepted as a known one-time artifact of
dropping this data.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@carl-codeorg
carl-codeorg marked this pull request as ready for review August 13, 2026 18:10
@carl-codeorg
carl-codeorg requested a review from a team August 13, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants