FailureSensorIQ
We show this table for reference; we do not rank on it.
IBM Research's FailureSensorIQ asks which sensors reveal which equipment failures across 10 kinds of industrial asset. It is the first published part of AssetOpsBench.
Evaluation results published by IBM Research. How Kaggle Benchmarks reruns tests.
Accuracy (%) on FailureSensorIQ v1 — Latest evaluation September 30, 2026
We mirror the published accuracy (%) view for FailureSensorIQ v1. GPT-6 Astra has the highest reported value at 69.6%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. We do not use these results to rank models overall.
GPT-6 Astra
Consistency 64.0%; F-Score 69.5%; Selection Accuracy 63.1%; Elimination Accuracy 71.2%; Evaluated 2026-09-16
Gemini 3 Pro Preview
Consistency 63.0%; F-Score 69.1%; Selection Accuracy 66.0%; Elimination Accuracy 70.8%; Evaluated 2025-11-22
Gemini 3.8 Flash
Consistency 65.7%; F-Score 69.7%; Selection Accuracy 65.1%; Elimination Accuracy 70.4%; Evaluated 2026-09-09
25 source rowsKnowledgeCurrentDisplay onlyUpdated Latest evaluation September 30, 2026
Accuracy (%) table (25 source rows)
| Rank | Model / configuration | Score | Parameters (B) | Open / closed |
|---|---|---|---|---|
| 1 | GPT-6 AstraCreator not reportedConsistency 64.0%; F-Score 69.5%; Selection Accuracy 63.1%; Elimination Accuracy 71.2%; Evaluated 2026-09-16 | 69.6% | Not reported | Unknown |
| 2 | Gemini 3 Pro PreviewCreator not reportedConsistency 63.0%; F-Score 69.1%; Selection Accuracy 66.0%; Elimination Accuracy 70.8%; Evaluated 2025-11-22 | 69.1% | Not reported | Unknown |
| 3 | Gemini 3.8 FlashCreator not reportedConsistency 65.7%; F-Score 69.7%; Selection Accuracy 65.1%; Elimination Accuracy 70.4%; Evaluated 2026-09-09 | 69.0% | Not reported | Unknown |
| 4 | o3Creator not reportedConsistency 59.1%; F-Score 67.4%; Selection Accuracy 63.7%; Elimination Accuracy 69.4%; Evaluated 2025-11-22 | 67.6% | Not reported | Unknown |
| 5 | Claude Sonnet 5.5Creator not reportedConsistency 60.9%; F-Score 68.9%; Selection Accuracy 63.7%; Elimination Accuracy 69.5%; Evaluated 2026-09-30 | 67.4% | Not reported | Unknown |
| 6 | GPT-5Creator not reportedConsistency 59.2%; F-Score 67.3%; Selection Accuracy 63.3%; Elimination Accuracy 69.4%; Evaluated 2025-11-22 | 67.2% | Not reported | Unknown |
| 7 | Gemini 2.5 ProCreator not reportedConsistency 57.5%; F-Score 67.0%; Selection Accuracy 62.6%; Elimination Accuracy 68.8%; Evaluated 2025-11-22 | 67.0% | Not reported | Unknown |
| 8 | Claude Opus 4.5Creator not reportedConsistency 59.9%; F-Score 67.1%; Selection Accuracy 63.6%; Elimination Accuracy 68.9%; Evaluated 2025-11-25 | 66.7% | Not reported | Unknown |
| 9 | Gemini 2.5 FlashCreator not reportedConsistency 56.1%; F-Score 65.8%; Selection Accuracy 60.8%; Elimination Accuracy 68.3%; Evaluated 2025-11-22 | 65.5% | Not reported | Unknown |
| 10 | GPT-5 miniCreator not reportedConsistency 56.8%; F-Score 65.5%; Selection Accuracy 59.0%; Elimination Accuracy 68.2%; Evaluated 2025-11-22 | 65.3% | Not reported | Unknown |
| 11 | o4 miniCreator not reportedConsistency 56.7%; F-Score 65.0%; Selection Accuracy 60.9%; Elimination Accuracy 67.4%; Evaluated 2025-11-22 | 64.8% | Not reported | Unknown |
| 12 | Grok 4.1 Fast ReasoningCreator not reportedConsistency 57.9%; F-Score 64.9%; Selection Accuracy 62.2%; Elimination Accuracy 66.4%; Evaluated 2025-11-22 | 64.6% | Not reported | Unknown |
| 13 | Claude Sonnet 4.5Creator not reportedConsistency 56.0%; F-Score 64.4%; Selection Accuracy 61.9%; Elimination Accuracy 65.3%; Evaluated 2025-11-22 | 64.2% | Not reported | Unknown |
| 14 | grok-4-fast-reasoningCreator not reportedConsistency 58.0%; F-Score 64.5%; Selection Accuracy 60.6%; Elimination Accuracy 66.6%; Evaluated 2025-11-22 | 64.1% | Not reported | Unknown |
| 15 | GPT-5.1Creator not reportedConsistency 51.6%; F-Score 63.9%; Selection Accuracy 57.3%; Elimination Accuracy 65.8%; Evaluated 2025-11-22 | 62.9% | Not reported | Unknown |
| 16 | Claude Haiku 4.5Creator not reportedConsistency 50.0%; F-Score 62.1%; Selection Accuracy 56.6%; Elimination Accuracy 64.0%; Evaluated 2025-11-25 | 61.8% | Not reported | Unknown |
| 17 | Qwen 3 235B A22B Instruct 2506Creator not reportedConsistency 49.7%; F-Score 62.5%; Selection Accuracy 59.2%; Elimination Accuracy 61.8%; Evaluated 2025-11-22 | 60.8% | Not reported | Unknown |
| 18 | Claude 3.5 SonnetCreator not reportedConsistency 46.7%; F-Score 58.5%; Selection Accuracy 53.0%; Elimination Accuracy 60.1%; Evaluated 2025-11-25 | 58.6% | Not reported | Unknown |
| 19 | Claude 3.5 HaikuCreator not reportedConsistency 45.5%; F-Score 58.6%; Selection Accuracy 54.1%; Elimination Accuracy 61.2%; Evaluated 2025-11-25 | 57.6% | Not reported | Unknown |
| 20 | Gemini 2.0 FlashCreator not reportedConsistency 44.8%; F-Score 61.2%; Selection Accuracy 52.8%; Elimination Accuracy 57.2%; Evaluated 2025-11-22 | 55.8% | Not reported | Unknown |
| 21 | gpt-oss-120bCreator not reportedConsistency 35.0%; F-Score 50.0%; Selection Accuracy 46.7%; Elimination Accuracy 51.6%; Evaluated 2025-11-28 | 50.5% | Not reported | Unknown |
| 22 | Mistral Small 3.1Creator not reportedConsistency 36.0%; F-Score 54.6%; Selection Accuracy 47.6%; Elimination Accuracy 48.3%; Evaluated 2025-11-22 | 47.9% | Not reported | Unknown |
| 23 | gpt-oss-20bCreator not reportedConsistency 26.1%; F-Score 45.9%; Selection Accuracy 42.1%; Elimination Accuracy 44.7%; Evaluated 2025-11-28 | 43.5% | Not reported | Unknown |
| 24 | deepseek-v3.1Creator not reportedConsistency 20.8%; F-Score 49.4%; Selection Accuracy 36.8%; Elimination Accuracy 37.5%; Evaluated 2025-11-22 | 36.6% | Not reported | Unknown |
| 25 | DeepSeek-R1Creator not reportedConsistency 11.2%; F-Score 38.4%; Selection Accuracy 38.3%; Elimination Accuracy 24.3%; Evaluated 2025-11-22 | 29.1% | Not reported | Unknown |
How to read this leaderboard
Use it to compare models on industrial maintenance and reliability knowledge. Read Consistency and the perturbation columns beside the score: a model that changes its answer when the options are shuffled is less dependable than its accuracy suggests.
Operator receipt: 25 sourced rows are currently displayable on this page; the highest published score among these 25 source rows is GPT-6 Astra at 69.6%, which does not establish a market leader.
Honest limit: Rows were evaluated between November 2025 and September 2026, so newer models are missing. It is multiple-choice knowledge, not agent work on live equipment data; IBM says agentic scenarios will follow.
How to read FailureSensorIQ
The questions are multiple choice and built from ISO documents. One type asks which sensor to monitor to catch a given failure early; the other asks which failure an abnormal sensor reading points to. The score is accuracy on the original questions. Kaggle also publishes Consistency, an F-score, accuracy on selection and elimination questions, and accuracy after the answer options are reordered or reworded.
Rows were evaluated between November 2025 and September 2026, so newer models are missing. It is multiple-choice knowledge, not agent work on live equipment data; IBM says agentic scenarios will follow.
Snapshot
These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. The table retains each source configuration and its reported value.
25 source rows are shown for FailureSensorIQ. The benchmark falls in the Knowledge category. We keep external benchmark mirrors separate from the weighted global scoring system, so these results remain source-specific evidence. FailureSensorIQ is currently displayed for reference but excluded from the scoring formula, so it does not directly affect overall rankings.
About FailureSensorIQ
Year
2025
Tasks
Multiple-choice questions on sensors and failure modes for 10 industrial assets
Format
Multiple-choice accuracy
Difficulty
Industrial maintenance and reliability knowledge
The questions are multiple choice and built from ISO documents. One type asks which sensor to monitor to catch a given failure early; the other asks which failure an abnormal sensor reading points to. The score is accuracy on the original questions. Kaggle also publishes Consistency, an F-score, accuracy on selection and elimination questions, and accuracy after the answer options are reordered or reworded. These results are display-only and do not enter overall or category rankings.
Freshness and provenance
Version
FailureSensorIQ 2025
Refresh cadence
Quarterly
Staleness state
Current
Question availability
Public benchmark set
BenchLM uses freshness metadata to decide whether a benchmark should still be treated as a strong differentiator, a benchmark to watch, or a display-only reference. For the full scoring policy, see the BenchLM methodology page.
Questions
What does FailureSensorIQ measure?
IBM Research's FailureSensorIQ asks which sensors reveal which equipment failures across 10 kinds of industrial asset. It is the first published part of AssetOpsBench.
Which source row has the highest reported FailureSensorIQ value?
GPT-6 Astra has the highest reported FailureSensorIQ value at 69.6%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference. BenchLM shows this benchmark for display only and does not use it in overall rankings.
How many models are evaluated on FailureSensorIQ?
The Latest evaluation September 30, 2026 snapshot contains 25 AI models.
Know when it’s worth switching models
The model to choose, the cheaper alternative, and the release we would wait on.
Read a sample issueJoin 7,000+ subscribers.
One email each week. Unsubscribe anytime.