# FailureSensorIQ

> IBM Research's FailureSensorIQ asks which sensors reveal which equipment failures across 10 kinds of industrial asset. It is the first published part of AssetOpsBench.

Canonical page: https://benchlm.ai/benchmarks/failuresensoriq

- Category: [Knowledge](/knowledge)
- Last updated: Latest evaluation September 30, 2026

Evaluation results published by [IBM Research](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench). undefined

The questions are multiple choice and built from ISO documents. One type asks which sensor to monitor to catch a given failure early; the other asks which failure an abnormal sensor reading points to. The score is accuracy on the original questions. Kaggle also publishes Consistency, an F-score, accuracy on selection and elimination questions, and accuracy after the answer options are reordered or reworded.

Rows were evaluated between November 2025 and September 2026, so newer models are missing. It is multiple-choice knowledge, not agent work on live equipment data; IBM says agentic scenarios will follow.

These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference.

- [Kaggle leaderboard and Apache 2.0 declaration](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench)
- [AssetOpsBench code](https://github.com/IBM/AssetOpsBench)
- [Published leaderboard data](https://www.kaggle.com/api/v1/benchmarks/ibm-research/asset-ops-bench/leaderboard)

## About FailureSensorIQ

- Year: 2025
- Tasks: Multiple-choice questions on sensors and failure modes for 10 industrial assets
- Format: Multiple-choice accuracy
- Difficulty: Industrial maintenance and reliability knowledge

The questions are multiple choice and built from ISO documents. One type asks which sensor to monitor to catch a given failure early; the other asks which failure an abnormal sensor reading points to. The score is accuracy on the original questions. Kaggle also publishes Consistency, an F-score, accuracy on selection and elimination questions, and accuracy after the answer options are reordered or reworded. These results are display-only and do not enter overall or category rankings.

FailureSensorIQ is currently displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.

## Leaderboard (25 source rows)

| Rank | Model | Configuration | Creator | Score |
|------|-------|---------------|---------|-------|
| 1 | [GPT-6 Astra](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 64.0%; F-Score 69.5%; Selection Accuracy 63.1%; Elimination Accuracy 71.2%; Evaluated 2026-09-16 | — | 69.6% |
| 2 | [Gemini 3 Pro Preview](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 63.0%; F-Score 69.1%; Selection Accuracy 66.0%; Elimination Accuracy 70.8%; Evaluated 2025-11-22 | — | 69.1% |
| 3 | [Gemini 3.8 Flash](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 65.7%; F-Score 69.7%; Selection Accuracy 65.1%; Elimination Accuracy 70.4%; Evaluated 2026-09-09 | — | 69.0% |
| 4 | [o3](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 59.1%; F-Score 67.4%; Selection Accuracy 63.7%; Elimination Accuracy 69.4%; Evaluated 2025-11-22 | — | 67.6% |
| 5 | [Claude Sonnet 5.5](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 60.9%; F-Score 68.9%; Selection Accuracy 63.7%; Elimination Accuracy 69.5%; Evaluated 2026-09-30 | — | 67.4% |
| 6 | [GPT-5](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 59.2%; F-Score 67.3%; Selection Accuracy 63.3%; Elimination Accuracy 69.4%; Evaluated 2025-11-22 | — | 67.2% |
| 7 | [Gemini 2.5 Pro](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 57.5%; F-Score 67.0%; Selection Accuracy 62.6%; Elimination Accuracy 68.8%; Evaluated 2025-11-22 | — | 67.0% |
| 8 | [Claude Opus 4.5](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 59.9%; F-Score 67.1%; Selection Accuracy 63.6%; Elimination Accuracy 68.9%; Evaluated 2025-11-25 | — | 66.7% |
| 9 | [Gemini 2.5 Flash](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 56.1%; F-Score 65.8%; Selection Accuracy 60.8%; Elimination Accuracy 68.3%; Evaluated 2025-11-22 | — | 65.5% |
| 10 | [GPT-5 mini](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 56.8%; F-Score 65.5%; Selection Accuracy 59.0%; Elimination Accuracy 68.2%; Evaluated 2025-11-22 | — | 65.3% |
| 11 | [o4 mini](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 56.7%; F-Score 65.0%; Selection Accuracy 60.9%; Elimination Accuracy 67.4%; Evaluated 2025-11-22 | — | 64.8% |
| 12 | [Grok 4.1 Fast Reasoning](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 57.9%; F-Score 64.9%; Selection Accuracy 62.2%; Elimination Accuracy 66.4%; Evaluated 2025-11-22 | — | 64.6% |
| 13 | [Claude Sonnet 4.5](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 56.0%; F-Score 64.4%; Selection Accuracy 61.9%; Elimination Accuracy 65.3%; Evaluated 2025-11-22 | — | 64.2% |
| 14 | [grok-4-fast-reasoning](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 58.0%; F-Score 64.5%; Selection Accuracy 60.6%; Elimination Accuracy 66.6%; Evaluated 2025-11-22 | — | 64.1% |
| 15 | [GPT-5.1](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 51.6%; F-Score 63.9%; Selection Accuracy 57.3%; Elimination Accuracy 65.8%; Evaluated 2025-11-22 | — | 62.9% |
| 16 | [Claude Haiku 4.5](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 50.0%; F-Score 62.1%; Selection Accuracy 56.6%; Elimination Accuracy 64.0%; Evaluated 2025-11-25 | — | 61.8% |
| 17 | [Qwen 3 235B A22B Instruct 2506](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 49.7%; F-Score 62.5%; Selection Accuracy 59.2%; Elimination Accuracy 61.8%; Evaluated 2025-11-22 | — | 60.8% |
| 18 | [Claude 3.5 Sonnet](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 46.7%; F-Score 58.5%; Selection Accuracy 53.0%; Elimination Accuracy 60.1%; Evaluated 2025-11-25 | — | 58.6% |
| 19 | [Claude 3.5 Haiku](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 45.5%; F-Score 58.6%; Selection Accuracy 54.1%; Elimination Accuracy 61.2%; Evaluated 2025-11-25 | — | 57.6% |
| 20 | [Gemini 2.0 Flash](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 44.8%; F-Score 61.2%; Selection Accuracy 52.8%; Elimination Accuracy 57.2%; Evaluated 2025-11-22 | — | 55.8% |
| 21 | [gpt-oss-120b](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 35.0%; F-Score 50.0%; Selection Accuracy 46.7%; Elimination Accuracy 51.6%; Evaluated 2025-11-28 | — | 50.5% |
| 22 | [Mistral Small 3.1](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 36.0%; F-Score 54.6%; Selection Accuracy 47.6%; Elimination Accuracy 48.3%; Evaluated 2025-11-22 | — | 47.9% |
| 23 | [gpt-oss-20b](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 26.1%; F-Score 45.9%; Selection Accuracy 42.1%; Elimination Accuracy 44.7%; Evaluated 2025-11-28 | — | 43.5% |
| 24 | [deepseek-v3.1](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 20.8%; F-Score 49.4%; Selection Accuracy 36.8%; Elimination Accuracy 37.5%; Evaluated 2025-11-22 | — | 36.6% |
| 25 | [DeepSeek-R1](https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench) | Consistency 11.2%; F-Score 38.4%; Selection Accuracy 38.3%; Elimination Accuracy 24.3%; Evaluated 2025-11-22 | — | 29.1% |

## FAQ

### What does FailureSensorIQ measure?

IBM Research's FailureSensorIQ asks which sensors reveal which equipment failures across 10 kinds of industrial asset. It is the first published part of AssetOpsBench.

### Which source row has the highest reported FailureSensorIQ value?

GPT-6 Astra has the highest reported FailureSensorIQ value at 69.6%. These are published source configurations. Compare the disclosed settings before treating a score difference as a model difference.

### How many models are evaluated on FailureSensorIQ?

The Latest evaluation September 30, 2026 contains 25 AI models.

### Does FailureSensorIQ affect BenchLM's overall score?

Not directly. FailureSensorIQ is still displayed on BenchLM for reference, but it is excluded from the weighted scoring formula.