Weld Monitoring Pilot Acceptance Criteria Checklist

Acceptance criteria turn a weld monitoring pilot into a production decision. They state the scope, evidence, independent reference and conditions for moving beyond one cell.

This checklist complements the method in Test False Alerts in a Weld Monitoring Pilot. That article explains how to build and read the reference set. Here the focus is the document that must be approved before collection starts: the rules for deciding whether the pilot passes, needs repair or remains inconclusive.

Key takeaways

• Freeze the cell, joint families, event unit and intended response.
• Define valid, invalid and missing records before calculating performance.
• Set separate criteria for coverage, false alerts and missed detections.
• Use an independent reference process with controlled disagreement handling.
• Test retrieval, workflow timing and exception ownership as well as analysis.
• Approve scale-up only for the conditions the pilot actually represented.

Define the decision and the scope boundary

Begin with one sentence that names the decision: whether the workflow is ready to request operator review on a specified cell and joint family while preserving retrievable evidence. Avoid “prove the AI works”, which does not identify the action or population.

Record the cell, process, material and joint families, recipe revisions and production states included. Define the judged unit: weld, pass, segment or layer. List exclusions; a pass for one population is not evidence for another fixture or material range.

The welding monitoring system overview provides the approved Therness system context. The scope and release decision remain specific to the customer’s controlled production and inspection plan.

[Assumption / ipotesi] The chosen pilot population represents the first intended production use. Production and quality owners must confirm that representation before data collection begins.

Freeze the label and reference method

Define the event that should trigger review in operational terms. Name the observable condition, the unit of analysis and the required action. Then specify how an independent reference reviewer will classify the same production event without being influenced by the monitoring score.

The reference may use visual testing or another method required by the applicable plan. ISO 17635, Non-destructive testing of welds — General rules for metallic materials, provides inspection-planning context. ISO 5817, Welding — Fusion-welded joints in steel, nickel, titanium and their alloys — Quality levels for imperfections, may provide contractual context when invoked. Check editions, clauses and acceptance route in controlled documents.

An uncertain label should remain unresolved until the named review occurs. Forcing it into pass or fail weakens the decision.

Set data-validity and coverage criteria first

Before judging alerts, define a usable record: correct production identifier, intended weld region, required configuration and recipe references, valid sensor data and retrieval from the agreed system.

Classify missing capture, identity mismatch, obstruction, invalid synchronization and unavailable analysis separately. These are evidence-chain failures, not automatically false alerts or missed detections.

Use an independent production count as the coverage denominator. The guide to every-weld thermal monitoring coverage explains how to reconcile production events with usable records. Define acceptable completeness and the maximum unresolved gap before the pilot. The quality owner supplies the thresholds; this article does not invent them.

Define false-alert and missed-detection limits separately

Pair every valid monitoring result with its reference label and place it in the confusion matrix. Keep the false-alert result separate from the missed-detection result because they represent different operational risks. False alerts consume review capacity and may reduce trust. Missed detections mean the monitoring workflow did not request review for a reference-positive event.

For each measure, write the allowed limit, applicable population and repeatability evidence. Include the minimum valid labelled events needed for a decision. A percentage without event counts can hide an unbalanced sample.

Do not change a threshold while scoring the acceptance dataset. If investigation shows that a revision is necessary, preserve the original result, approve a new threshold version and run the affected portion again. The acceptance record should make that repair visible.

Test workflow responses and timing

An accurate signal can still fail operationally if it reaches the wrong person, arrives after the part has moved or cannot be connected to the required action. For every monitoring status, define the owner, response, allowed processing state and evidence of closure. Include at least valid/no-review, valid/review, invalid capture, identity mismatch and system unavailable.

Verify on the cell that the event is captured, the intended instruction arrives, the part state is controlled and the record remains retrievable. Measure timing against the pilot plan, not a generic claim.

The HeatCore AI product page describes the Therness analysis layer. Product capability does not decide the customer’s response policy; that policy must be approved with the cell integration and quality workflow.

[Assumption / ipotesi] The production control system can associate a review or hold action with the same identifier used by the monitoring record. Verify the interface and fallback procedure before live pilot events are collected.

Include stress cases and exception ownership

Select representative edge cases: operating-window boundaries, joint or fixture states, planned interruptions and invalid sensor data. Test acceptance rules without creating defects or bypassing safety controls.

Assign an owner and action to every exception category. A missing identifier may belong to the interface, an obstructed view to fixture work and an uncertain label to quality review. A miscellaneous bucket hides which process needs repair.

Keep out-of-scope events out of the pass-rate denominator, but report their count and reason. Silent exclusion produces a result that cannot be audited.

Use a signed acceptance record

The final record should contain scope, dataset and exclusions, validity and coverage results, confusion matrix, threshold and configuration versions, exceptions, retrieval test and decision. Outcomes may be pass for the stated scope, conditional pass, extend for a defined gap, or do not scale.

The decision owner should state which conditions may be transferred to the next cell and which must be revalidated. The weld evidence pack workflow shows how the underlying records can be organized for that transfer. Final weld acceptance remains under the approved inspection and quality plan; monitoring supplies in-process evidence rather than certification.

Pilot acceptance checklist

Before collection begins, confirm that the controlled pilot document answers every item below:

1. The decision, cell, population and event unit are named.
2. Included and excluded joint, material and recipe conditions are listed.
3. The monitoring label and required response are operationally defined.
4. The independent reference method and uncertain-label workflow are approved.
5. Valid, invalid, missing and mismatched records are distinguished.
6. Coverage, false-alert and missed-detection criteria have separate thresholds.
7. Minimum evidence counts and review rules are fixed before scoring.
8. Threshold and configuration changes create a new traceable version.
9. Every exception category has an owner and closure action.
10. Retrieval and cell-response tests have explicit pass evidence.
11. The scale decision names its limits and revalidation requirements.

Sources

• Therness false-alert testing guide — reference-set and confusion-matrix method.
• Therness every-weld coverage guide — production-population and capture-validity workflow.
• Therness welding monitoring system — system and integration context.
• Therness HeatCore AI — monitoring analysis layer context.
• ISO 17635, Non-destructive testing of welds — General rules for metallic materials — inspection-planning context; verify the applicable edition and clauses.
• ISO 5817, Welding — Fusion-welded joints in steel, nickel, titanium and their alloys — Quality levels for imperfections — contractual quality-level context when invoked.

Therness solution: Welding monitoring system — Every weld inspected in the cycle, with defect flags and the ISO 3834 / ISO 17635 evidence kept per weld.

Written by Luca Santoro , founder