Readme.org

#+OPTIONS: ^:nil #+TITLE:compile-db-gen Generate the [[http://clang.llvm.org/docs/JSONCompilationDatabase.html][JSON compilation database]] (compile_commands.json) from tracing the make command (or other compile tools).

** Why we need the compile-db-gen The compile-db-gen can work in complicate compile environment to generate the =compile_commands.json=.

Or, the compile-db-gen can save us from hardly getting the =compile_commands.json= scenario.

Here are some examples that getting the =compile_commands.json= easyly:

1. A project orgnized with CMake, just run cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON to get the =compile_commands.json=.

2. A project orgnized with automake, run make with [[https://github.com/rizsotto/Bear][Build EAR]] will get the =compile_commands.json= correctly.

3. A project orgnized with Makefile and using the CC/CXX variables, run [[https://clang-analyzer.llvm.org/scan-build.html][scan-build]] can get the =comile_commands.json=.

... Beside these, it maybe hard to get the =compile_commands.json= for some complicated scenario.

Here is an example that can not get the =compile_commands.json= with previous tools:

The compile steps will invoke some elf-64 tools to build some dirs, and will invoke some elf-32 tools to build other dirs, like: #+begin_src quote make -> elf-32 tool -> gcc(32bits) ... -> elf-64 tool -> gcc(64bits) ... #+end_src

Then the compile-db-gen.py can do well on these dynamic compile scenario to get the =compile_commands.json=.

** Usage Just run the compile-db-gen.py with make and its arguments, eg: #+begin_src sh ./compile-db-gen.py make -j1 #+end_src Then it will generate the =compile_commands.json= in current directory.

And follow commands will give more details: #+BEGIN_SRC sh ./compile-db-gen.py trace make ./compile-db-gen.py parse compile_commands.raw compile_commands.json #+END_SRC

For more options, please run #+BEGIN_SRC sh ./compile-db-gen.py -h #+END_SRC

** How dose it work The [[http://wikipedia.org/wiki/Strace][strace]] tool can trace system calls (execve, chdir) and write their arguments to log file.

Then the compile-db-gen.py follow the log file to construct the process calling tree, and output the compile commands.

** Requirments The strace-4.8 or high is required; python3 is required.

The compile-db-gen.py will track the child-process terminal message which contain parent pid from the strace log.

The follow log give hists that child(pid=937) is ended, and the pid for its parent is pid=936. #+BEGIN_QUOTE [pid 936] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=937, si_status=0, si_utime=0, si_stime=0} --- #+END_QUOTE but the strace-4.5 seems doesn't give the child pid. #+BEGIN_QUOTE [pid 936] --- si_code=CLD_EXITED SIGCHLD 0 --- #+END_QUOTE I didn't try the compile-db-gen.py with the strace-4.6, strace-4.7, if you had run the compile-db-gen.py with starce-4.6/4.7, please let me know it's success or failed.

** Similar tools [[http://clang-analyzer.llvm.org/scan-build.html][scan-build]] It replaces the CC, CXX environment string, then get the compile database, but lots of projects didn't use the CC, CXX environment settings, then scan-build won't work correctly.

[[https://github.com/rizsotto/Bear][Build EAR]] (Bear) It inject the libbear.so via modify the environment variables =LD_PRELOAD= or =DYLD_INSERT_LIBRARIES=, then filter the compile command. But the Bear doesn't work for a complicated sencen, for example, a makefile called both x86 and x64 exetable binaries, but the libear.so is ELF-64, then the LD_PRELOAD will fail on tracing the x86 binaries.

[[https://github.com/ffevotte/clang-tags/blob/master/clang-tags][clang-tags trace]] The clang-tags trace just simply parse the strace log line by line, will give wrong result for it didn't walk through the building tree.