Clarifying the outcome for a short hands-on trial

• To get started, which single outcome would make this 30-day technical evaluation a clear success for your team? Options: Higher developer fix rate on specific findings, Clear reduction in exploitable dependency exposure, Lower false-positive rate versus current scanner, Faster time-to-fix for critical findings, Other
• Tell me about the three repositories you would pick for the trial and why those repositories matter to your risk profile
• Walk me through the stakeholders who must approve repo and CI access during the evaluation, including engineering owners, AppSec, and legal Options: Engineering repo owners, AppSec lead, Platform/CI owners, Compliance/legal, Product owner, Other
• Estimate the average number of active pull requests per week across the chosen repos Options: 0-10, 11-50, 51-200, 201-500, 500+
• Which measurable acceptance criteria should we commit to at the start, for example target false-positive ratio, fix rate, or CI performance impact Options: Specific fix rate, False-positive percentage, Time-to-fix, CI pipeline time overhead, Number of specific dependency findings
• If the pilot meets those targets, who on your side would be authorized to approve a broader rollout

Where your current security checks fall short

• What single visibility gap in your current scanning or inventory would make you stop and demand a new approach Options: Unknown transitive dependencies, Lack of reachability/exploitability context, No CI integration, Untracked container or IaC scanning, Other
• Describe a recent audit finding or incident that revealed gaps in dependency visibility or code-level vulnerabilities
• On average, how many repositories does your current scanning tooling actually cover compared with the total number of active repositories Options: All repos, Most high-risk repos, Only a handful of top repos, Less than 10%
• Who currently triages pull request findings, and how long from detection to a remediation decision does that workflow typically take Options: Dedicated AppSec triage team, Shared engineering/security triage, Individual repo owners, Ad hoc
• Give an example of a finding developers dismissed as noise, and tell us what downstream cost or risk resulted
• What single failure mode in your current approach would force you to select a different solution immediately Options: Inability to cover critical repos, Unacceptable false-positive rate, Missed exploitable vulnerabilities, CI performance degradation, Regulatory audit failure

Why developers may ignore security feedback

• Walk me through the last time engineers ignored a security finding, including what made them ignore it and what finally changed their behavior
• How much additional CI runtime per pull request would your teams tolerate before build velocity becomes the deciding factor Options: <30 seconds, 30-90 seconds, 1-3 minutes, >3 minutes
• Which false-positive threshold in prior tools led to either suppressed alerts or restored developer trust Options: <1%, 1-5%, 5-10%, >10%
• Describe the minimum context a developer needs in a finding to fix it without escalating to AppSec Options: Stack trace and file/line, Exploitability evidence, Suggested code change, Dependency remediation steps, All of the above
• If a new scanner reduced false positives but required two weeks of tuning effort from your engineers, would you prioritize launching it or keeping the status quo Options: Prioritize launch and invest tuning time, Maintain status quo to avoid upfront effort, Need executive direction

What other options are on the table

• Name the primary alternative you are most likely to choose instead of bringing an external vendor Options: Renew incumbent contract, Open-source tooling, Build internal solution, No change, Other
• List the incumbent vendors, categories of tools, or internal projects your team evaluated in the last six months Options: Current SAST, Dependency scanner, Container image scanner, Internal scripts, Other
• Under what conditions would you decide to stay with your existing scanning approach rather than switch Options: Cost savings, Sufficient coverage, Low risk tolerance for change, Satisfied stakeholders, Other
• Has anyone proposed an internal build as the preferred path, and if so who would own that internal project Options: Yes, engineering platform, Yes, AppSec team, No internal proposal, Other
• Who would still need to sign off even if the pilot shows a 50% improvement in specific fix rate Options: CISO, VP Engineering, Procurement, Legal, Budget owner, Other

Practical gates, integrations, and access requirements

• Identify the one integration dependency that, if missing, will stop a hands-on evaluation from starting Options: Repository admin access, CI webhook capability, Service account or API keys, Network egress to vendor services, Other
• List the CI/CD systems, repository hosts, and artifact registries that must be integrated and indicate whether APIs and service accounts are available Options: Git provider with API, Self-hosted Git with admin, CI system with webhook support, Artifact registry with API, None of the above
• Confirm whether a dedicated engineering contact will be available for configuration, tuning, and triage during the 30-day trial Options: Yes, full-time contact, Yes, part-time contact, No named contact yet, Will identify before start
• Estimate the headcount and skillset available on your side for access setup, webhook configuration, and daily triage during the pilot Options: 1-2 engineers, experienced, 3-5 engineers, mixed experience, Dedicated platform team, Limited or no engineering bandwidth
• Point to the compliance, legal, or procurement approvals that typically take the longest and tell us whether any could block repository access for more than two weeks Options: Legal review, Security/data access review, Procurement contract negotiation, Compliance approval, None expected

Acceptance metrics that make the decision easy

• Select the single metric that would prove the tool works for your engineers and explain why Options: Developer fix rate, False-positive reduction, Time-to-fix improvement, Reduction in exploitable findings, Developer engagement on PRs
• Rank these evaluation metrics by importance for your team: developer fix rate, false-positive rate, time-to-fix, CI performance impact, number of high-risk dependencies Options: 1st, 2nd, 3rd, 4th, 5th
• How soon after integration do you expect to see meaningful developer engagement, for example comments, patch commits, or closed PRs Options: Within 1 week, 1-2 weeks, 2-4 weeks, After pilot end
• State the maximum false-positive percentage relative to your existing scanner that would be acceptable for engineers to trust results Options: <1%, 1-3%, 3-5%, 5-10%
• Assuming the pilot meets the agreed metrics, what specific action would you commit to taking within one week to expand the tool Options: Approve purchase order, Schedule org-wide pilot, Allocate rollout resources, Request contract review, Other

Timing, decision makers, and budget

• Identify who will call the final go or no-go and describe what would push them to move faster than the current timeline
• In your procurement process, what are the typical steps and how long does each step take Options: Initial legal review, Security/data review, Procurement approval, Budget sign-off, Contract negotiation
• Provide the budget range you expect for replacing your current application security testing toolset Options: <$50k, $50k-$150k, $150k-$500k, >$500k
• When do you expect leadership to make a decision if the pilot demonstrates the agreed outcomes Options: Immediately, Within 2 weeks, Within 1 month, Longer than 1 month
• Name the unresolved contract or budget item that would stop a purchase even if the pilot is successful

Handoff, operational ownership, and next steps

• Assuming the pilot validates the targets, who on your side will take ownership of rollout and what resources will they need
• Map the communication and escalation paths between engineering, AppSec, and legal during rollout Options: Weekly syncs, Dedicated Slack channel, Ticket-driven escalation, Executive check-ins, Other
• Outline how ongoing operational success will be measured after deployment and which team will own each metric
• Confirm whether you have a preferred cadence for post-deployment reviews and specify that cadence Options: Weekly, Biweekly, Monthly, Quarterly, Ad hoc
• State the rollback or mitigation plan leadership expects if the pilot does not show the agreed improvement Options: Tune and re-run pilot, Limit deployment to fewer repos, Cancel rollout and stay with incumbent, Engage engineering for custom fixes

• gaps
• current_state
• stakeholders
• decision_readiness
• desired_state
• success_criteria
• desired_state
• success_criteria
• stakeholders
• gaps
• current_state
• decision_readiness
• stakeholders
• current_state
• desired_state
• success_criteria
• gaps
• decision_readiness
• decision_readiness
• decision_readiness
• decision_readiness
• decision_readiness

• Repository Integration
• CI/CD Pipeline Scanning
• IDE Real‑Time Scanning
• Open‑Source Dependency Scanning
• SBOM Generation and Export
• Container Image Scanning
• Infrastructure-as-Code Security Scanning
• Reachability Prioritization Engine
• Initial Tuning and Noise Suppression
• Tailored Fix Guidance
• Pull Request Security Checks
• Security Policy Configuration

Repository Integration

• Which repositories should be included in the evaluation (list repository names, repository selectors, or repository hosting URLs)?
• How many active repositories (with commits in the last 30 days) will you integrate in the initial rollout? Options: 1-3, 4-10, 11-25, 26-100, More than 100
• Describe branch protection rules and pull request workflows that will affect scanning (for example, protected main branch, required reviewers, merge strategies).
• Who is authorized to grant repository access tokens and approve connector configuration (name or role)?
• What evidence will validate successful repository integration (for example, list of mapped repositories with recent commit timestamps, successful test scan on a current commit, connector health logs)? Options: Repository list with timestamps, Successful test scan on a recent commit, Connector token health check logs

CI/CD Pipeline Scanning

• Which CI/CD systems host the build pipelines to be scanned (provide system names or internal pipeline identifiers)?
• List the pipeline stages that should trigger scanning (for example, pull request build, merge pipeline, nightly build). Options: Pull request build, Post-merge pipeline, Nightly build, Release pipeline, Other
• How long is an acceptable additional build time per pipeline step for scanning in your CI environment (in seconds)? Options: <10s, 10-30s, 31-60s, 61-180s, No limit provided
• What acceptance criteria will confirm pipeline scanning is operational (for example, successful scan on a merged commit within 30 minutes, SARIF export produced, no build failures caused by the scanner)? Options: Successful scan within time window, SARIF export present in artifact store, No build failures from scanner
• Do you require scan artifacts or SARIF exports to be pushed to a specific storage (for example, S3 bucket, internal artifact registry)? Options: Yes, S3 or equivalent, Yes, internal artifact registry, No persistent storage required

IDE Real‑Time Scanning

• List the IDEs and exact versions used by your developers (for example, VS Code 1.60, IntelliJ 2021.2).
• Specify the authentication mechanism your team prefers for IDE integrations (for example, OAuth app, personal access token, enterprise single sign-on). Options: OAuth app, Personal access token, Enterprise single sign-on, No preference
• Describe how inline fix suggestions should appear in the IDE (for example, code actions, quick fixes, inline decorations with suggested diff).
• Are there offline development scenarios where IDE scanning must function without internet access (for example, air-gapped workstations or restricted networks)? Options: Yes, air-gapped, Partially offline, No, always online
• Do you want anonymous usage telemetry from IDE plugins (for example, feature usage, number of ignored findings) sent to your telemetry endpoint? Options: Yes, No, Only aggregate metrics

Open‑Source Dependency Scanning

• Identify the package managers and ecosystems across your repositories (for example, npm, Maven, PyPI, Go modules). Options: npm/Node, Maven/Java, PyPI/Python, Go modules, Other
• Provide the count of unique dependency manifests across the initial repositories (for example, number of package.json, pom.xml, requirements.txt files). Options: 1-10, 11-50, 51-200, 201-1000, More than 1000
• Specify the desired scanning frequency for dependencies (for example, on every pull request, nightly, weekly). Options: On every pull request, On merge, Nightly, Weekly, On demand
• Who approves dependency remediation pull requests in your workflow (role or team name)?
• Provide examples of acceptable dependency remediation pull requests (for example, target branch naming, exact version bump, included test results).

SBOM Generation and Export

• Select the SBOM (software bill of materials) formats you require (for example, SPDX, CycloneDX). Options: SPDX, CycloneDX, Both, Other
• State the method for SBOM export and storage (for example, push to S3 bucket, store in artifact registry, attach to release). Options: Push to S3, Store in artifact registry, Attach to release notes, Other
• Identify the teams and roles that must receive SBOM exports and the access controls required on the destination (for example, security team read-only, audit team write).
• Are SBOMs required for container images, application artifacts, or both in your compliance scope? Options: Container images only, Application artifacts only, Both, None required
• Should SBOM contents be correlated with code scanning results to present combined findings (for example, map vulnerable dependency to repo path and PR)? Options: Yes, No, Optional

Container Image Scanning

• Name the container registries and registry URLs you use (for example, internal registry domain, external registry).
• Indicate whether scanning should occur at build time, as a registry scan, or at deployment time. Options: Build time, Registry scan, Deployment time, Multiple
• Name the owners of image build pipelines and the contacts who can provide registry credentials.
• State the vulnerability score threshold that should block promotion to production (for example, CVSS 9.0 or all critical severity). Options: Block on critical only, Block on CVSS >= 9.0, Block on high and critical, No blocking required
• Should image SBOMs be required as part of image scanning output? Options: Yes, No, Optional

Infrastructure-as-Code Security Scanning

• Confirm the directories and file patterns that contain your infrastructure-as-code (for example, Terraform *.tf, CloudFormation templates).
• Name the cloud providers and account or project identifiers to include in scanning (for example, AWS account IDs, GCP project IDs).
• Describe how you want secrets referenced in templates handled during scanning (for example, masked, replaced with test values, scanned in a safe mode). Options: Masked, Replaced with test values, Omitted from scan, Scan with safe-mode
• Which IaC frameworks and versions must be supported (for example, Terraform 0.12+, CloudFormation YAML, Pulumi TypeScript)? Options: Terraform, CloudFormation, Pulumi, Other
• Do you require drift detection alerts that map infrastructure drift back to the originating commit or pull request? Options: Yes, No, Consider later

Reachability Prioritization Engine

• Which repositories or services are your highest-risk targets for reachability analysis (for example, public-facing services, authentication modules)?
• Specify the call graph depth or code-path analysis sensitivity you prefer for reachability scoring (for example, shallow, default, deep). Options: Shallow, Default, Deep
• What integration points must the prioritization engine use (for example, test coverage reports, runtime telemetry, API gateway logs)? Options: Test coverage, Runtime telemetry, API gateway logs, None
• Who will validate prioritized findings and accept the risk classification (role or team)?
• Are you requiring reachability results to be surfaced in pull requests, ticketing systems, or both? Options: Pull requests, Ticketing system, Both, Neither

Initial Tuning and Noise Suppression

• Which repositories or services should receive initial noise suppression and tuning during the evaluation period?
• Provide your current baseline false-positive rate or an example metric for comparison (for example, average findings per pull request).
• What temporary suppression rules are acceptable during the trial (for example, suppress low-confidence findings for 14 days)? Options: Suppress low-confidence, Suppress specific rule IDs, No suppression, Other
• How will you verify the false-positive rate meets your threshold during tuning (for example, developer feedback sample, automated comparison against baseline)? Options: Developer feedback sample, Automated comparison, Manual review
• What defines an acceptable false-positive rate for adoption (specify a numeric threshold or developer feedback target)? Options: <5% per PR, <10% per PR, Developer triage rate acceptable, Other

Tailored Fix Guidance

• Which languages and frameworks should the fix guidance target with code examples (for example, Java with Spring Boot, JavaScript with Express)?
• Specify whether fix guidance should include suggested pull request diffs, remediation snippets, or only advisory steps. Options: PR diffs, Remediation snippets, Advisory steps only, All of the above
• Who will approve or publish the tailored guidance templates for use by developers?
• Provide any internal coding standards or secure-coding guidelines the fix guidance must adhere to (for example, internal lint rules, approved libraries).
• Do you require fix guidance to include testing instructions or regression test examples? Options: Yes, include tests, Optional, No

Pull Request Security Checks

• Which pull request events should trigger security checks (for example, open, update, merge)? Options: Open, Update, Merge, All
• Specify required blocking conditions for pull requests (for example, critical vulnerability found, unresolved high severity). Options: Block on critical, Block on high and critical, Inform only
• What status checks or CI flags must be set on a PR to indicate security scan completion?
• Who should be notified on PR security failures (for example, author, repo-owner, security team)? Options: Author, Repo owners, Security team, All
• Are automatic fix pull requests acceptable for your workflow, or do you require manual remediation PRs? Options: Automatic fixes OK, Manual remediation required, Manual review of automatic PRs

Security Policy Configuration

• Confirm the policy types you need configured (for example, blocking policies by severity, allowed dependency lists, exception workflows). Options: Blocking by severity, Allowed list, Exception workflows, Other
• State the approval workflow for security policy exceptions (for example, approver role, time-limited exception, audit logging requirements).
• Which compliance or audit standards must policy reports support (for example, SOC 2 evidence, internal audit CSV exports)?
• Who will be the policy owner responsible for ongoing policy changes and reviews?
• Do you require scheduled policy reviews or automated alerts when policies are violated? Options: Scheduled reviews, Automated alerts, Both, Neither

• Subscription Agreement
• Order Form
• Data Processing Agreement (DPA)
• Repository & CI Access Authorization
• Acceptance Criteria & Go/No-Go Agreement
• Service Level Agreement (SLA)
• Change Order Agreement
• Termination & Data Return Addendum

• Go-live Health Check (weeks 1-4)
• First Measurement Review (weeks 4-10)
• Acceptance Gate Review (around day 90)
• Ongoing Operational Review (quarterly)

• Reconfirm success criteria and owners
• Apply documented reachability tuning and suppression rules aimed at reducing the false positive rate by the agreed amount.
• Deliver an updated sample findings pack showing before-and-after results for the tuned rules.
• Schedule the next metrics snapshot and data export for acceptance-gate analysis.
• Restate numeric acceptance criteria
• Document a pass or fail for each numeric acceptance criterion recorded in Solution Scope and record the formal acceptance decision.
• If any criteria failed, capture a remediation plan with clear success metrics and deadlines to close the gap.
• Publish the acceptance decision and the evidence packet used to reach it.
• Create remediation tickets for any failed criteria with target completion dates and verification steps.
• Document the incumbent decommission or retention plan including archive confirmation and contract disposition.
• KPI dashboard review
• Confirm that false positive rate per pull request and vulnerability fix rate within 14 days remain at or improving toward Solution Scope targets.
• Prioritize the enhancement backlog and agree delivery timelines for the top items.
• Update tuning rules and configuration changes noted in the meeting and schedule their deployment window.
• Publish the prioritized enhancement backlog with target delivery quarters for the top items.
• Prepare the next quarterly KPI snapshot and pre-read two weeks before the next review.
• Confirm all integrations required for the evaluation are active and producing telemetry.
• Identify the top 3 operational blockers and record remediation actions with target completion dates.
• Publish a deployment runbook including repository selectors, CI hook endpoints, and access inventory.
• Create remediation tickets for each blocker with target resolution dates and verification steps.
• Present first 30-60 day outcome data
• Determine whether the false positive rate per pull request and developer engagement rate with fix suggestions are trending toward the Solution Scope targets.
• Agree a short list of corrective tuning actions and a re-measurement date before the acceptance gate.
• Deployment verification
• Open issues and incident burn-down
• Present 90-day outcome data against each criterion
• Diagnose gaps and sample triage
• Early adoption signals and usage patterns
• Enhancement request backlog and prioritization
• Incumbent decommission status check
• Tuning and suppression impact review
• Agree corrective actions and timeline to acceptance gate
• Operational incidents and blockers
• Tuning and config change review
• Acceptance decision and documented outcome
• Agree immediate remediation actions
• Agree remediation plan for unmet criteria
• Agree next operational adjustments and cadence