A weld monitoring pilot is not complete when the dashboard starts producing alerts. It is complete when the quality team can show, with labelled production evidence, what the system detects, what it misses, and how often it asks people to review an acceptable weld. That distinction turns a sensor demonstration into a controlled qualification exercise.
False alerts and missed detections affect different parts of the operation. A false alert consumes review time and can erode operator trust. A missed detection may allow a relevant condition to continue without an in-process signal. Neither rate should be assumed from a laboratory result or a supplier statement. Both need to be measured on the target cell, joint family, material and operating window.
Key takeaways
- Define the event to detect before collecting pilot data.
- Keep the monitoring decision separate from the reference inspection result.
- Label every pilot weld with an agreed review process.
- Report the confusion matrix, not only an overall accuracy figure.
- Investigate disagreements by joint family and operating condition.
- Freeze acceptance criteria before deciding whether to scale.
Start with an operational definition
The team first needs a precise definition of a positive event. “Abnormal weld” is too broad. A useful definition identifies the signal, the unit of analysis and the required response. The unit might be a weld, a pass, a layer or a defined segment. The response might be “review the record”, “hold the part” or “check the process setting”. These are workflow decisions, not automatic statements about final weld acceptance.
Write the definition in the pilot protocol and keep it stable during the evaluation window. If the team changes the label after seeing the results, it becomes impossible to tell whether the detector improved or the test moved. When a new condition appears, log it as a new label candidate for a later revision.
ISO 17635, Non-destructive testing of welds — General rules for metallic materials, belongs in the source set because the inspection plan determines how the final weld is examined. ISO 5817, Welding — Fusion-welded joints in steel, nickel, titanium and their alloys — Quality levels for imperfections, may also be relevant when the contract invokes it. This article does not assign either standard a monitoring requirement or reproduce acceptance limits.
Build an independent reference set
Use production-representative welds from one cell and preserve the identifier that links the monitoring record to the part, joint and reference review. The person reviewing the reference result should not be influenced by the monitoring score. Likewise, the monitoring threshold should not be tuned while each reference result is being revealed.
A practical record contains:
- weld or segment identifier;
- recipe, joint family and material family;
- monitoring decision and timestamp;
- reference inspection disposition;
- reviewer role and review date;
- reason code for any disagreement;
- notes on missing or invalid data.
The weld monitoring system overview explains how thermal, visual and process information can be brought into one workflow. The HeatCore AI product page describes the Therness analysis layer. Those pages document product scope; they do not replace the cell-specific reference set.
Before data collection, verify with production, MES and quality owners that the cell can expose a stable identifier shared by the monitoring record and the inspection record. If it cannot, the pilot needs a temporary but controlled mapping table.
Read the confusion matrix correctly
Once monitoring decisions and reference labels are paired, place every item in one of four groups:
| Monitoring signal | Reference result | Interpretation |
|---|---|---|
| review | relevant condition present | detected event |
| review | relevant condition absent | false alert |
| no review | relevant condition present | missed detection |
| no review | relevant condition absent | correctly cleared item |
Do not collapse this table into one headline percentage. A dataset dominated by acceptable welds can produce an attractive overall figure while still missing the smaller class that matters. Report false-alert and missed-detection behaviour separately, together with the number of labelled items in each cell of the table.
Segment the results by the factors that could change the signal: joint family, recipe, material family, fixture state and sensor validity. The purpose is not to find a flattering slice. It is to identify where one threshold is credible and where a separate rule, more data or a different sensing arrangement is needed.
Investigate disagreements before changing thresholds
Every disagreement deserves a reason code. Useful categories include reference uncertainty, identifier mismatch, sensor obstruction, recipe outside the trained scope, threshold sensitivity and an event definition that is too vague. A threshold change should follow this review, not precede it.
The real-time monitoring guide provides context on in-process signals, while the ISO 17635 reference page describes the inspection-planning context. Keep those roles explicit: monitoring is evidence about process behaviour, and the applicable inspection plan governs final disposition.
If the reference label itself is uncertain, do not force the item into a pass/fail bucket. Mark it unresolved and define the additional review needed. Forced labels make the dataset look complete but weaken the decision.
Freeze the scale decision in advance
Before the pilot begins, the quality owner should approve the maximum tolerable false-alert burden, the maximum tolerable missed-detection risk for the scoped event, the minimum data completeness and the rule for unresolved labels. These are business and quality decisions. The monitoring supplier can provide evidence, but should not silently choose the customer’s risk threshold. The weld monitoring pilot acceptance criteria checklist turns these decisions into a document approved before collection starts.
At the end of the pilot, produce a signed decision record with the scope that passed, the conditions excluded, the remaining failure modes, the threshold version and the monitoring plan for drift. If the evidence is insufficient, extend the pilot with a named data gap. Do not convert absence of evidence into a claim of performance.
Treat a single cell as representative for the first decision only: verify recipe range, tooling, operator practices and environmental conditions before carrying the result to another cell.
Sources
- Therness welding monitoring system overview — workflow and sensing context.
- Therness HeatCore AI — documented product scope.
- ISO 17635, Non-destructive testing of welds — General rules for metallic materials — standard identified for inspection-planning context; verify the applicable edition and clauses.
- ISO 5817, Welding — Fusion-welded joints in steel, nickel, titanium and their alloys — Quality levels for imperfections — standard identified for contractual quality-level context; verify applicability.
Plan a weld monitoring pilot around evidence
Therness starts with one cell, an agreed reference set and acceptance criteria fixed before scale-up.
Discuss a pilotTherness solution: Welding monitoring system — Every weld inspected in the cycle, with defect flags and the ISO 3834 / ISO 17635 evidence kept per weld.
