Demystifying Exploitable Bugs in Smart Contracts
May 12, 2024 ยท View on GitHub
This project aims to provide a valuable resource for Web3 developers and security analysts by facilitating their understanding of exploitable bugs in smart contracts. We conduct a thorough analysis of exploitable bugs extracted from code4rena and classify each bug according to its nature.
Our initial research suggests that a notable proportion of exploitable bugs in smart contracts are functional bugs, which cannot be detected using simple and general oracles like reentrancy. We aim to raise awareness about the significance of such bugs and encourage practitioners to develop more sophisticated and nuanced automatic semantical oracles to detect them.
๐ฐ ๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐๐๐ ๐๐ ๐๐ก๐๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐ ๐๐๐๐๐ ๐๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐๐๐๐ ๐๐๐ ๐๐๐๐๐๐๐๐ข ๐๐ ๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐, ๐ ๐๐๐๐ ๐๐๐๐๐๐ ๐๐ ๐๐๐๐๐๐๐๐ ๐๐๐๐๐ ๐๐๐๐๐๐ ๐๐๐ ๐๐๐๐๐๐๐ ๐๐๐๐๐๐๐.
Please be aware that this repository is currently undergoing active development, and the data may change over time due to ongoing code4rena contests.
Dataset Description
Folder Structure
The dataset is organized into four folders:
- papers/: contains our ICSE23 paper summarizing our preliminary results, as well as the supplementary material for the paper.
- results/: contains the bug classification in bugs.csv and the description for each contest in contests.csv.
- contracts/: contains all the smart contracts that we examined, using the version at the time of the contest.
- reports/: contains all the reports provided by code4rena.
Bug Labels
We classify the surveyed bugs into three main categories based on their nature:
- Out-of-scope bugs (denoted by O)
- Bugs with simple and general testing oracles (denoted by L)
- Bugs that require high-level semantical oracles (denoted by S)
As classifying functional bugs can be ambiguous, we welcome suggestions to improve our classification standards. You can find more detailed label information in our documentation, and we encourage you to refer to our current classification guidelines for more information.
Recommended Security Analysis Tools
Our goal is to create a comprehensive list of vulnerability detection techniques that will be a valuable resource for Web3 developers and security analysts. We will focus on two main categories:
- Vulnerability detection techniques that prioritize the development of semantical oracles for smart contracts.
- Publicly available security analysis tools that can be used for auditing
We warmly welcome any additional suggestions or contributions from the community to help expand and improve the list.
Vulnerability Detection with Automatic Semantical Oracles
We believe that future web3 security efforts will prioritize identifying functional bugs and developing corresponding oracles. To this end, we intend to compile a list of techniques that provide guidance in the creation of automatic semantic oracles. These techniques will be sourced from various materials, such as peer-reviewed research papers, pre-prints, industry tools, and online resources.
Publicly Available Security Analysis Techniques
This section will include open-source techniques that are publicly available and currently in active development. These techniques can be used either directly by Web3 developers and security analysts or as building blocks for other tools. We give priority to source-code level techniques, which are better suited for Web3 development and auditing contexts.
| Technique | Developer(s) | Description | Security-related Keywords |
|---|---|---|---|
| Slither | Trail of Bits | Static Analysis Framework | Vulnerability Detectors, SlithIR |
| Aderyn | Cyfrin | Static Analysis Framework | Static Analyzer, Custom Detectors, Markdown Reports |
| Foundry | Paradigm | Development Toolchain | Fuzzing, Stateful Fuzzing (Invariant Testing), Differential Testing |
| Echidna | Trail of Bits | Fuzzer | Fuzzing , Stateful Fuzzing (Invariant Testing), CI/CD |
| Optik | Trail of Bits | Hybrid Fuzzer (Symbolic Execution + Fuzzing) | Fuzzing, Stateful Fuzzing, Symbolic Execution |
| Woke | Ackee Blockchain | Development Toolchain | Cross-chain Testing, Invariant Testing, Vulnerability Detectors, IR |
| 4naly3er | Picodes | Static Scanner | Code4rena Pre-content Testing |
| Manticore | Trail of Bits | Symbolic Execution Tool | Symbolic Execution, Property Testing |
| Halmos | a16z | Symbolic Bounded Model Checker | Symbolic Execution, Bound Checker |
| Solidity SMTChecker | Ethereum Foundation | Formal Verification by Symbolic Execution | Solidity, Formal Verification, Symbolic Execution |
| Mythril | Consensys | Symbolic Execution Tool | Symbolic Execution, On-Chain Analysis, Vulnerability Detectors, Taint Analysis |
| Pyrometer [WIP] | Nascent | Symbolic Execution Tool | Symbolic Execution, Abstract Interpretation |
| greed | UCSB Seclab | Static/Symbolic Analysis Framework | Symbolic Execution, Bound Checker, Static Analyses, Property Testing |
| ethpwn | ethpwn | Dynamic analysis/Debugging | EVM simulations, EVM debugging |
In addition, we curate a catalogue of security utilities applicable to smart contract programming languages beyond Solidity.
| Technique | Language | Description | Security-related Keywords |
|---|---|---|---|
| Move Prover | Move | Formal Specification and Verification | Formal Verification |
Valuable Resources for Web3 Security
This section comprises a compilation of resources that pertain to web3 security.
| Resource | Keywords |
|---|---|
| Academic Smart Contract Papers | Academic Paper List |
| DeFi Hacks Reproduce - Foundry | Attack Replication |
| Smart Contract Security Verification Standard | Security Checklist |
| Awesome MythX Smart Contract Security Tools | Security Analysis Service |
| Common Security Properties of Smart Contracts | Security Compliance Suite |
| Immunefi PoC Templates | PoC Templates |
| Awesome MEV Resources | MEV Resources |
| Front-Running Attack Benchmark Construction and Vulnerability Detection Technique Evaluation | Front-Running Dataset |
| Ultimate DeFi & Blockchain Research Base | Blockchain Security All-in-One |
| Common Fork Bugs | Exploit Dataset |
Contributing
We welcome all types of contributions to our project, including but not limited to:
- Suggesting new reference techniques for smart contract security analysis.
- Adding newly disclosed code4rena contest bugs.
- Suggesting improvements to the classification standard
- Correcting mislabeled bugs
- Filling in any missing defillama entities in the
results/contests.csv
Further details can be found in our contribution guidelines.
Cite
If you are using our dataset for an academic publication, we would really appreciate a citation to the following work:
@inproceedings{DBLP:conf/icse/ZhangZXL23,
author = {Zhuo Zhang and
Brian Zhang and
Wen Xu and
Zhiqiang Lin},
title = {Demystifying Exploitable Bugs in Smart Contracts},
booktitle = {{ICSE}},
pages = {615--627},
publisher = {{IEEE}},
year = {2023}
}
Clarification
Please refer to our classification documentation.
Acknowledgments
We would like to extend our sincere thanks to code4rena for making this valuable information publicly available.