SCRUBD Dataset

Overview

SCRUBD is a comprehensive dataset of Ethereum smart contracts labeled for two critical vulnerabilities:

• Reentrancy (RE): 985 labeled functions (245 vulnerable, 501 non-vulnerable)
• Unhandled Exceptions (UX): 566 labeled functions (275 vulnerable, 291 non-vulnerable)

Key Features

• ✅ Real-world contracts from Ethereum mainnet (via Google BigQuery)
• ✅ Rigorous labeling through crowdsourcing + expert validation
• ✅ Synthetic test cases covering RE edge cases
• ✅ Tool evaluation comparing 6 analysis tools (Slither, Mythril, etc.)
• ✅ Detailed documentation with vulnerability patterns

Research Findings

Tool Performance Comparison (F1-Scores):

Figure: F1-Scores of vulnerability detection tools on SCRUBD datasets

Key Results:

• Slither performs best on real-world contracts (SCRUBD/cD) for both RE (0.61) and UX (0.83)
• Sailfish excels on synthetic edge cases (SCRUBD/sD) for RE (0.75)
• Sailfish doesn't support UX detection (N/A)

Applications

• 🔬 Tool Evaluation: Benchmark RE/UX detection tools
• 🤖 ML Training: Train vulnerability prediction models
• 📊 Empirical Studies: Analyze vulnerability patterns