Getting Oriented: What's Top of Mind for Your Team?

• In one sentence, what is the single highest-priority quality reporting outcome your team must deliver this year?
• Which reporting programs are highest priority for you right now? Options: CMS (e.g., Hospital QRP, HAC, HEDIS/MIPS), State/regulatory reporting, Commercial payer contracts, Accreditation reporting (e.g., The Joint Commission), All of the above, Other
• Who on your team feels the most pressure when reporting season approaches—and why?
• How do you currently measure whether your reporting activities are on track (targets, cadence, owners)? Options: Weekly dashboard, Monthly review, Ad hoc checks, Only at submission time, Other
• When reporting goes well, what specific downstream impact do you see (e.g., avoided penalties, improved star ratings, stronger payer contracts)?

Are You Quietly Losing Revenue — Or Is It Just Stress?

• How confident are you that current quality measure results accurately protect your expected reimbursement? Options: Very confident, Somewhat confident, Unsure, Not confident
• Tell us about the last time a quality score affected revenue or contract performance—what happened and who felt the impact?
• Which of the following financial risks have you experienced or are you most worried about? Options: Direct payment penalties, Lost shared savings, Negative contract renegotiation, Reduced incentive payments, Public-rating-driven patient loss, None yet, Other
• How closely do your finance and quality teams coordinate on reporting outcomes and forecasts? Options: Tightly integrated, Occasional alignment, Separate with limited communication, No routine coordination
• If a 1–3% reimbursement swing was on the table, what would change about how you prioritize measurement and validation?

Where Does Your Data Hide Its Secrets?

• How often do you discover that the EHR or data extracts don't contain the elements you need to calculate a measure? Options: Almost always, Often, Sometimes, Rarely, Never
• Which EHR(s) and ancillary systems host the records you must use for quality reporting? Options: Epic, Cerner/Oracle, MEDITECH, Allscripts, Athenahealth, Hybrid/multiple vendors, Other
• Where do you see the most consistent data gaps or ambiguities (structured fields, free-text notes, scanned documents, external PDFs, interfaces)? Options: Structured fields, Free-text / notes, Scans / PDFs, External systems / HIE, Interfaces / integrations, Other
• How accessible is sample clinical data for analysis (e.g., deidentified extracts, test FHIR feeds, CCDs)? Options: Full access to extracts/feeds, Limited test data only, Only production views with restrictions, No easy access yet, Other
• Give an example of a measure field that regularly causes mapping disagreements—what are the specific definitions and where do they diverge?

How Much of Abstraction Is a Bottleneck (And Where)?

• If your abstraction process could talk back, what would it say is the single biggest pain point it faces daily?
• Describe your current abstraction team: headcount, full-time vs. part-time, turnover cadence, and geographic distribution.
• Which of these best describes your abstraction workflow today? Options: 100% manual chart review, Manual review with basic digital tools, Hybrid: automated pre-fill + manual confirmation, Primarily automated with human exceptions, Other
• How long does it typically take from case identification to closed abstraction for a single chart? Options: <24 hours, 1–3 days, 4–10 days, 2–4 weeks, >1 month
• What error, discrepancy, or rework rates do you track for abstractions, and how do those typically trend during peak reporting?

When Will You Know the Numbers Are Right?

• What tolerance for variance against a manual benchmark would you accept before you’d consider an automated calculation 'validated'? Options: Exact match required, Within 1–2% variance, Within 3–5% variance, Within 6–10% variance, Depends on the measure
• How do you currently validate measures before submission (parallel runs, random audit, full reconciliation, external audit)? Options: Parallel automated vs manual, Sample-based audit, 100% manual reconciliation, Post-submission corrections only, Other
• Who signs off on acceptance criteria for measure outputs—what stakeholders or committees must be convinced? Options: Quality Director/COO, Clinical leads / CMOs, Compliance/Risk, Finance, IT/Analytics, All of the above, Other
• Describe a prior validation failure or near-miss: what root cause was identified, and what changes were put in place?
• How much time and resource do you budget to reconcile discrepancies and prepare submission-ready files? Options: Minimal (<5% of team time), Moderate (5–15%), Significant (15–30%), Major (>30%)

Who Needs to Own This Change (And Who Will Block It)?

• If I told you implementing an automated abstraction solution will require a week of cross-department work, who would push back first—and why?
• Which stakeholders must be involved for data access, clinical mapping, and validation (pick all that apply)? Options: Quality/Compliance, IT/Integration, Clinical leadership, Abstractors/ops, Finance, Legal/Privacy, Executive sponsor, Other
• What approvals or governance checkpoints will any vendor-driven change require (e.g., security review, contracting, CIO sign-off)? Options: Security review, Business associate agreement (BAA), Contract/rates approval, Clinical steering committee, No formal checkpoints, Other
• Who will be the day-to-day owner from your side for configurations, sample data access, and acceptance tests?
• How do change initiatives typically falter here—capacity, competing priorities, unclear ownership, or technical blockers? Options: Capacity, Competing initiatives, Unclear ownership, Technical/EHR limitations, Budget constraints, Other

If This Worked, What Would Next Year Feel Like?

• Imagine next year's reporting closes and you exceeded your targets—what concrete differences would you notice (staff workload, time to submission, score improvements)?
• Which measurable outcomes would make you say the project was a success (pick up to three)? Options: Reduced abstraction hours, Improved measure accuracy, Faster submission timelines, Higher public ratings, Reduced penalties, Lower validation rework, Other
• How would improved measurement free up capacity or create new opportunities for your team or clinicians?
• What would losing that improvement look like—what risks or negative consequences are you trying to avoid?
• If we delivered a pilot that showed <3% variance from manual benchmarks and cut abstraction time by half, how would you want to scale it? Options: Roll across measures, Pilot more sites first, Negotiate commercial terms, Delay until next year, Other

Practical Next Steps: Are We Ready to Move Toward a Pilot?

• Which of these readiness items do you already have in place for a pilot? Options: Deidentified sample data, EHR test environment or FHIR access, Executive sponsor, Abstractor participation, Clear acceptance criteria, None of the above
• How quickly could you provide a representative sample of charts and measure specs for initial mapping and validation? Options: Within 1 week, 1–2 weeks, 2–4 weeks, Longer than 1 month, Unsure
• What concerns would you want addressed before authorizing a pilot (data security, scope, cost, resource time)? Options: Data security/compliance, Scope clarity, Commercial terms, Staff time commitments, Technical feasibility, Other
• Who should we bring into an initial kickoff meeting from your side (names/roles), and who needs to be in the final sign-off conversation?
• What would be a reasonable decision timeline after we present a pilot plan and initial findings? Options: Immediately / same week, Within 2 weeks, 1 month, 2–3 months, Longer / unsure

• Current State & Success Criteria Alignment
• EHR Data Mapping Workshop (Field-by-Field)
• Automated Abstraction Simulation (Run on Customer Charts)
• Accuracy Validation & Discrepancy Resolution
• Decision Review & Next Steps to Scope

• Host: Produce a detailed validation report with metrics, discrepancy log, and remediation proposals for each measure.
• Host: Propose rule/NLP tweaks or additional mapping to reduce high-frequency exceptions identified in the run.
• Customer: Confirm whether the observed automated coverage meets their operational expectations or needs further tuning.
• Present Validation Metrics (per-measure)
• Provide a validated accuracy report that quantifies gaps versus manual abstraction.
• Classify each discrepancy with an owner and remediation action to improve automated accuracy.
• Mutually agree on per-measure acceptance thresholds and a deadline for remediation.
• Confirm need and scope for any additional training or specification clarifications.
• Introductions & Meeting Objectives
• Customer: Review and adjudicate remaining disputed cases and confirm final ground-truth outcomes.
• Host & Customer: Create a prioritized remediation backlog with owners, changes, and target completion dates.
• Customer: Confirm acceptance thresholds and sign-off criteria to enable transition to Solution Scope.
• Executive Summary of Findings
• Decision to proceed to Solution Scope or a documented list of gating items if not ready.
• Agree on initial deployment scope, validation targets, and timeline aligned to reporting cycles.
• Assign owners for scope deliverables and sign-off on acceptance criteria required for deployment.
• Document risks and mitigation actions that must be addressed in the Solution Scope phase.
• Customer: Provide formal approval to proceed or a written list of gating items that require resolution.
• Host: Draft a Solution Scope summary (measures, interfaces, validation plan, timeline, SOW skeleton) for customer review.
• Host: Update the remediation backlog with committed deadlines and owners based on validation outcomes.
• Customer & Host: Schedule handoff workshops for Deployment group once scope is approved.
• Produce one-sentence current state statement signed off by customer stakeholders.
• Agree and document quantified consequences tied to KPIs that make the problem urgent.
• Define one-sentence future state and 2–3 measurable success signals to prove during the experience.
• Confirm available sample charts, measures, and data extracts required for subsequent sessions.
• Customer: Provide a signed one-sentence current state and 3 consequence metrics (e.g., FTE hours, missed revenue).
• Customer: Deliver a defined set of sample charts (suggest 25–50 records covering edge cases) and manual abstraction benchmark data.
• Host: Prepare a mapping template and success-signal tracker to use across the experience meetings.
• Host: Confirm technical contacts and required access method (FHIR/API/SFTP) and schedule the EHR mapping workshop.
• Review Extract Sample & Field Inventory
• Complete field-level mapping for prioritized measures with mapped/unmapped status documented.
• Agree on mitigation strategy for each unmapped element (NLP, manual abstraction, proxy), with owner assigned.
• Confirm extraction method, cadence, and test payload to be provided by customer.
• Establish clear validation checks that gate the next automation simulation.
• Customer: Provide a secured test extract or sandbox access containing agreed sample records and field-level documentation.
• Host: Populate the shared mapping workbook with field mappings and mitigation notes and return for customer review.
• Customer & Host: Agree and document validation checks and success thresholds that the extract must satisfy.
• Customer: Identify an SME (clinical abstractor or informaticist) to join the next simulation session for adjudication.
• Pre-test Checklist & Baseline Reminder
• Demonstrate the platform producing extraction and automated abstraction outputs on customer's real charts.
• Identify and categorize exceptions and edge cases requiring human review or rule refinement.
• Obtain SME confirmation (or corrective feedback) on automated decisions for sampled cases.
• Document expected reduction in manual steps and any remaining manual touchpoints.
• Host: Deliver the raw extracted dataset, automated abstraction outputs, and exception logs from the simulation.
• Customer: Provide adjudications for discrepancies in the sampled cases and annotate rationale where manual judgment applied.
• Measure-by-Measure Mapping (interactive)
• Customer Current State (one sentence)
• Case-by-Case Discrepancy Review
• Review of Agreed Acceptance Criteria & Validation Results
• Live Run: Automated Extraction & Abstraction
• Scope for Next Phase (measure set, interfaces, abstraction coverage)
• Unstructured Data & Workarounds
• Root Cause Classification & Remediation Plan
• Consequence Quantification
• Review of Exceptions & Edge Cases
• Future State Definition (one sentence) + Success Signals
• Timeline Alignment to Reporting Cycles & Commitments
• Extraction Method & Frequency
• Tying Outputs to Measure Rules
• Agree on Validation Targets & Acceptance Criteria

• Configure EHR Interface for eCQM Extraction
• Map Measure Specifications to EHR Data Elements
• Deploy Automated Chart Abstraction Engine
• Implement Abstraction Exception Handling and Routing
• Identify Care Gaps and Generate Provider Alerts
• Calculate Measure Scores and Compliance Rates
• Generate Submission-Ready Reports and Files
• Submit Measures to CMS, Payers, and Registries
• Integrate HEDIS, MIPS, Hospital, and ACO Libraries
• Launch Measure Trend Analysis Dashboards
• Train Abstraction Team on Platform Workflows
• Configure Audit Trail and Change Tracking

Configure EHR Interface for eCQM Extraction

• Which EHR vendor(s) and version(s) are in scope for extraction?
• Which data access methods are available or preferred for integration? Options: FHIR (R4/R3), Direct API / Proprietary API, HL7 v2 feeds, C-CDA/CCD, Database query / ETL, SFTP / flat file export
• Do you have existing eCQM endpoints or export mappings configured in the EHR? Options: Yes - eCQM endpoints available, Partially - some endpoints available, No - need to configure
• Is there an internal IT/HIT resource assigned to support connectivity (name/role and weekly hours available)?
• Are sandbox/test instances and synthetic/sample data accessible for initial integration? Options: Yes - full sandbox available, Limited test data available, No sandbox/test environment
• Do you require VPN, IP allow-listing, or special security/compliance controls for the interface? Options: Yes - VPN/IP restrictions, Yes - specific org security controls, No special controls
• What is the expected monthly chart / encounter volume to be extracted? Options: < 1,000, 1,000–10,000, 10,000–50,000, > 50,000

Map Measure Specifications to EHR Data Elements

• Which measure libraries and specific measures should be mapped initially?
• Are there local/custom measures or payer-specific definitions that differ from standard specs? Options: Yes - multiple local/custom measures, Yes - a few variations, No - standard specs only
• Who will own final mapping decisions when EHR element ambiguity exists (customer SME, vendor, joint review)? Options: Customer SME, Customer SME with vendor validation, Vendor-led (with customer approval)
• Do clinical concepts live in structured fields, free-text notes, scanned documents, or external systems? Options: Structured fields, Free-text clinical notes, Scanned/PDF documents, External registries / systems
• Are value sets/code systems (e.g., LOINC, SNOMED, ICD) standardized across sites, or are local codes used? Options: Standardized (LOINC/SNOMED/ICD), Mixed standard and local codes, Primarily local/custom codes
• What acceptance criteria do you require for mapped elements (e.g., coverage %, manual override allowance)? Options: Coverage target (e.g., 90%+), Allow manual overrides with audit, Strict automated-only mapping, Other

Deploy Automated Chart Abstraction Engine

• What percent of charts do you expect to be auto-abstracted vs. manually abstracted initially? Options: <25%, 25-50%, 50-75%, >75%
• Which care settings are in scope for automated abstraction (e.g., inpatient, outpatient, ED, post‑acute)? Options: Inpatient, Outpatient/ambulatory, Emergency Department, Post-acute / SNF, All of the above
• Are scanned documents / PDFs and image-based records required for abstraction? Options: Yes - a lot, Some, No
• What throughput and SLA do you require for automated abstraction processing? Options: Near real-time, Daily batch, Weekly, Custom schedule
• Will the abstraction engine need to integrate with existing human abstraction queues/workflows? Options: Yes - tight integration, Partial integration, No - separate workflows
• What accuracy/validation target should the engine meet before we reduce manual review (e.g., F1 score, percent agreement)?

Implement Abstraction Exception Handling and Routing

• What exception types should trigger human review (e.g., low-confidence extractions, missing data, conflicting data) ? Options: Low-confidence extraction, Missing critical fields, Conflicting data points, Business-rule violations, Other
• How should exceptions be routed by priority or type (e.g., to specific abstractors, clinical leads, or specialty teams)? Options: Role-based routing, Team-based queues, Escalate to clinical lead, Round-robin assignment
• What SLA times do you require for exception resolution (e.g., 24/48/72 hours)? Options: Same day, 24 hours, 48 hours, 72 hours, Custom
• Which notification channels should be used for exceptions (EHR inbox, email, platform task, pager)? Options: EHR inbox, Email, Platform task/notifications, Slack/MS Teams, Other
• Do you require audit logging and reason capture for each exception and resolution? Options: Yes - full audit required, Partial audit (critical fields only), No
• Should certain exceptions be auto-escalated to leadership or compliance teams for review? Options: Yes - auto-escalate, No - manual escalation only, Conditional

Identify Care Gaps and Generate Provider Alerts

• Which care gap types should be detected and surfaced (e.g., missing immunizations, overdue screenings, medication reconciliation)?
• Do you prefer real-time alerts at encounter time or periodic batch alerts to providers/care teams? Options: Real-time (encounter), Daily batch, Weekly summary, Both real-time and batch
• Which alert channels are acceptable for provider notifications (EHR inbox, in-workflow CDS, email, secure messaging)? Options: EHR inbox/CPOE, In-workflow CDS, Email, Secure messaging, Mobile push
• Should alerts include suggested actions and documentation templates or only flags? Options: Include suggested actions and templates, Flags only, Configurable per measure
• Who will own closing gaps: primary care, specialists, care coordinators, or centralized quality team? Options: Primary care, Specialists, Care coordinators, Centralized quality team, Mixed model
• How should false-positive alerts be reported and used to refine detection logic? Options: Report via feedback UI, Periodic review meetings, Automated learning from feedback, Other

Calculate Measure Scores and Compliance Rates

• Which reporting periods and cadences must the calculation engine support (e.g., calendar year, rolling 12 months, quarterly)? Options: Calendar year, Rolling 12 months, Quarterly, Custom reporting periods
• What tolerance or acceptance thresholds do you require before considering scores production-ready (e.g., +/- X% vs manual benchmark)?
• Do you require denominator/exclusion adjudication rules to be customized for payer programs? Options: Yes - payer-specific customization, No - standard rules only, Some payers require tweaks
• Should the system support stratified reporting (by site, provider, patient cohort) and risk adjustment? Options: Site-level, Provider-level, Patient cohort, Risk adjustment required, All of the above
• How frequently should reconciliations and re-calculations run after data fixes (immediate, nightly, monthly)? Options: Immediate/near real-time, Nightly, Weekly, Monthly
• Are there specific rounding, suppression, or minimum-n size rules we must apply for public reporting? Options: Yes - provide spec, No special rules, Use standard CMS suppression rules

Generate Submission-Ready Reports and Files

• Which file formats and standards are required for submissions (e.g., QRDA I/III, CSV, XML, HL7)? Options: QRDA I, QRDA III, CSV/Flat file, XML, Other
• Do you require pre-submission validations and schema checks in-platform before export? Options: Yes - mandatory pre-checks, Optional checks, No pre-checks required
• Who will approve final submission-ready files (role/title) and how should approvals be captured?
• What delivery method is preferred for finalized files (secure SFTP, direct portal upload, vendor submission)? Options: Secure SFTP, Portal upload, Vendor submits on behalf, Email (not recommended)
• How many report templates and recipient lists need to be configured (e.g., internal leaders, payers, registries)?
• Do you need scheduled automated exports and distribution (daily/weekly/monthly)? Options: Yes - scheduled exports, No - manual only, Mixed

Submit Measures to CMS, Payers, and Registries

• Will your organization submit directly to CMS/payers/registries or require vendor-assisted submission? Options: Customer submits directly, Vendor submits on behalf, Hybrid (vendor prepares, customer submits)
• Do you have existing submitter credentials and access to each destination (CMS ID, payer portals)? Options: Yes - all credentials ready, Partial - some portals, No - credentials need provisioning
• Do you require dry-run test submissions and reconciliation reports prior to official submission windows? Options: Yes - required, Optional, No
• Who is responsible for attestations, sign-offs, and legal declarations for final submissions?
• Should submission processes include automated confirmation and error-handling workflows (retries, error alerts)? Options: Yes - automated retries/alerts, Basic notifications only, No automation required
• Are there payer- or registry-specific business rules or file transforms required post-export? Options: Yes - multiple transforms required, Some payers require transforms, No special transforms

Integrate HEDIS, MIPS, Hospital, and ACO Libraries

• Which libraries do you require in scope at launch (select all that apply)? Options: HEDIS, MIPS, Hospital inpatient measures, ACO measures, State-specific libraries
• Do you need historical measure versions supported for look-back or reconciliation? Options: Yes - historical versions required, No - current year only, Some historical support needed
• How often do you expect measure library updates to be applied (annual, quarterly, as-released)? Options: Annually, Quarterly, As-released, Custom cadence
• Are there local payer/program variations to standard library definitions that must be modeled? Options: Yes - many variations, A few variations, No
• Who should own change control and version approval for library updates (customer, vendor, joint governance)? Options: Customer-owned, Vendor-managed with notification, Joint governance
• Do you require mapping reports that show EHR element -> value set -> measure mapping for audit/regulatory review? Options: Yes - detailed mapping reports, Summary mapping only, No mapping reports required

Launch Measure Trend Analysis Dashboards

• Which KPIs and views are highest priority for dashboards (e.g., measure-level trend, provider performance, gap closure rate)?
• What time granularity is required (daily, weekly, monthly, rolling 12-month)? Options: Daily, Weekly, Monthly, Rolling 12-month, Custom
• Which user roles need access and what level of drill-down should each role have? Options: Executive (high-level), Quality lead (site-level), Abstractors (case-level), Clinicians (provider-level)
• Do you require embedded explanations of measure calculation logic and data lineage within dashboards? Options: Yes - show data lineage, Basic calculation notes, No
• Should dashboards support scheduled distribution and export formats (PDF, CSV)? Options: PDF, CSV/Excel, API export, All of the above

• Statement of Work (SOW)
• Master Services Agreement (MSA)
• Business Associate Agreement (BAA)
• Data Processing Agreement (DPA)
• Pricing & Payment Schedule
• Acceptance Criteria & Validation Plan
• EHR Interface & Data Access Matrix
• Timeline & Reporting Cycle Alignment
• Implementation Roles, RACI & Escalation
• Training & Knowledge Transfer Plan
• Change Order & Scope Management
• Compliance, Audit Rights & Security Controls
• Termination, Data Return & Offboarding Plan
• Final Sign-off & Go‑Live Authorization

• Success Outcomes Review
• Validation & Reconciliation Workshop
• Lessons Learned & Continuous Improvement
• Enhancement Backlog Prioritization & Roadmap
• Ongoing Governance, Monitoring & Escalation Cadence

• Assign product/engineering owners and confirm resource allocation for committed items.
• Ensure learnings are entered into the shared log and made accessible for future engagements.
• Publish a formal Lessons Learned document and link it to the shared issues/enhancements log.
• Create SOP update tickets and assign owners for each process change.
• Schedule targeted training sessions addressing identified gaps.
• Capture metrics to monitor the effectiveness of implemented process changes.
• Review Shared Issues & Enhancements Log
• Produce a prioritized enhancement backlog with impact/effort scores.
• Agree a near‑term roadmap with milestones tied to reporting cycles and resource owners.
• Commit resources and define next checkpoints for roadmap progress review.
• Create prioritized epics in the project tracker with acceptance criteria and target milestones.
• One‑sentence Current State Recap
• Publish the roadmap and circulate to executive sponsors and operational teams.
• Schedule a quarterly roadmap review aligned to reporting cycles.
• Define Governance Roles & RACI
• Establish a clear governance model with owners, SLAs, and triage rules for the shared log.
• Agree a repeatable KPI review cadence and dashboard to detect and prevent measure drift.
• Set escalation thresholds and a contact path to resolve high‑impact issues rapidly.
• Configure and share the KPI dashboard and train owners on its use.
• Provision access to the shared issues/enhancements log and enforce triage SLAs.
• Publish the governance RACI and 90‑day meeting schedule to stakeholders.
• Define automated alerts for threshold breaches and assign on‑call contacts.
• Confirm whether the delivered outcomes meet the agreed success signals for acceptance.
• Quantify the business impact of any deviations and agree remediation or mitigation actions.
• Assign owners, timelines, and revalidation criteria for all identified gaps.
• Publish a consolidated outcomes vs success signals report and distribute to stakeholders.
• Create remediation tickets with owners and deadlines for all high‑priority gaps.
• Schedule follow-up validation meeting and define re‑run data windows.
• Update the shared issues/enhancements log with accepted decisions and statuses.
• Pre‑work & Data Availability Check
• Reconcile and document root causes for measure discrepancies with traceable evidence.
• Produce a prioritized list of fixes with test cases that, when implemented, will move measures to acceptance criteria.
• Agree objective re‑run and sign‑off criteria for each corrected measure.
• Record and log each discrepancy with source evidence (chart IDs, screenshots, logs).
• Create implementation tickets for each agreed fix with linked test cases.
• Schedule the re‑run window and validation sampling approach.
• Update measure specification mapping documentation where interpretation fixes are needed.
• Timeline & Milestone Review
• Document a prioritized set of lessons learned with supporting evidence.
• Define and assign process improvements, SOP updates, and training actions to owners with timelines.
• Monthly KPI Review Format
• Value & Impact Scoring
• What Worked / What Didn’t (Structured Retrospective)
• Walkthrough: Sample Measures & Charts
• Recap of Agreed Success Signals & Acceptance Criteria
• Process Mapping — Abstraction & Escalation
• Outcome Data Presentation
• Shared Log Access, Triage Rules & SLAs
• Root Cause Mapping
• Feasibility & Effort Assessment
• Escalation Paths & Decision Thresholds
• Training & Role Adjustments
• Backlog Prioritization & Roadmap Draft
• Define Fixes and Test Cases
• Consequence Analysis
• 90‑Day Monitoring Schedule
• Reconciliation Sign‑off Criteria
• Agree Actionable Process Changes
• Gap Triage & Immediate Decisions
• Decision & Resource Commitments