The Art of Debugging Open Book

This evolving book will teach you how to:

1. Debug normal issues really fast
2. Make complicated issues possible to debug

footnote: adapted from Perl's slogan: "Easy things should be easy and hard things should be possible".

This is a work-in-progress collection of methodologies and copy-n-paste recipes for successful debugging of simple and complicated software problems. Some sections are quite complete, while other will be finished at a later stage, and yet other haven't been started.

Besides debugging methodologies the second focus of this book is on sharing the best tools I have discovered to get the debugging job done successfully, while ideally "losing the least amount of hair" in the process.

Intro

I have been developing software since 1995 and a lot of this work involved debugging. Over the years I developed various efficient methodologies for discovering the source of the problem, which is the most difficult stage before it can be solved. Since after the problem is understood, typically its resolution is at easy reach.

Every so often someone I would be debugging some difficult problem with would suggest to share my approaches with the world. I always said that it'd be too difficult to generalize, but recently the planted seed seems to have sprouted and so in the following chapters I will try to share some of the insights to ease this very difficult at times process.

Writing about debugging in the void is very difficult and since I haven't been saving use cases, it will take some time to build this up, so expect these pages to be a Work In Progress (WIP) for many moons. But hopefully some ideas could be relayed to you sooner than later, and they would help to ease your burden of debugging in your professional and play projects.

Table of Contents

1. Fast Debugging Methodology

2. Unix Tools for Debugging - bash, strace, make, prompt, nohup, etc.

3. Debugging Compiled Programs - gdb, ldd, nm, LD_LIBRARY_PATH, LD_PRELOAD, etc.

4. Debugging Python Programs - py-spy, paths, auto-print, etc.

5. Debugging PyTorch Programs - cpu and gpu memory, performance, models, tensors, etc.

6. Debugging Machine Learning Projects (external)

Updates

I announce any significant updates on my twitter channel https://twitter.com/StasBekman.

Ebook versions of the book

You can download various ebook formats of this book:

• PDF
• EPUB

I will try to rebuild these once in a few weeks or so, but if you want the latest ebook versions, the instructions for building are here.

SKILL.md for AI agents

I maintain a SKILL.md file that you can use to teach your AI agent to debug your projects better.

See also the companion skills: Machine Learning Engineering and Stas' Python Cookbook.

Related Open Books

• Machine Learning Engineering Open Book — training and operating large-scale ML models.
• Stas' Python Cookbook — everyday Python and standard-library recipes.

Contributing

If you found a bug, typo or would like to propose an improvement please don't hesitate to open an Issue or contribute a PR.

License

The content of this site is distributed under Attribution-ShareAlike 4.0 International.

My repositories map

✔ Books: Machine Learning Engineering | The Art of Debugging | Stas' Python Cookbook

✔ Applications: ipyexperiments

✔ Tools and Cheatsheets: bash | conda | git | jupyter-notebook | make | python | tensorboard | unix

✔ Other Machine Learning: ML ways | Porting