Mastering Vim for coding large projects

In the previous post about Making Source-Insight-like Vim, we already knew how to make Vim become Source Insight by using some useful plugins. In this post, we will study the way to make it become more efficent when working with large projects.

We knew about tags which is generated by ctags application. It works well in most cases, but there are still weak points. Now we will find them out and learn how to work perfectly.

cstags and cscope

Let’s take a look back firstly. ctags enables two features: allowing you to jump from function calls to their definitions. The first means that when you are over a call to a method, hitting CTRL-] will jump to the place where that method is defined or implemented. The second feature means that when you type foo. or foo->, and if foo is a structure, then a pop-up menu with field completion will be shown.

Now I will introduce to you a new a developer’s tool for browsing source code, called cscope. cscope also has the first feature as same as ctags- using set cscopetag - but not the last. However cscope additionally adds the ability to jump to any of the places where a function is called as well.

So as far as jumping around a code base is concerned, ctags will only ever lead you towards the place where the function is implemented, whereas cscope can show you where a function is called too.

Why would you choose one over the other? Well, I use both. ctags is easier to set up, faster to run and if you only care about jumping one way it will show you less lines.

cscope is great for bigger, unknown code bases. The set up is a pain because cscope needs a file containing a list of names of files to parse. Also in vim, by default there are no key bindings set up - you need to run :cscope blah blah manually.

So to conclude: ctags is easier to set up and mostly works without doing much else. cscope provides more features if you have to maintain a large and mostly unknown code base, but requires more leg work.

Theory of cscope, ctags, & vim

You instruct cscope and ctags to build a database for a directory tree. These databases are a list of <variable, line_number> pairs sorted by variable. Vim does a binary search on these databases to answer your queries (e.g., find definition of variable means searching the database to find the appropriate variable=variable, then moving the cursor to that line_number).

Vim allows you to specify the order in which it searches its databases. Typically your list of databases will be ordered as such:

C-kernel: <kernel-cscope>
C-user: <ctags-stdlib, cscope-project>
C++-user: <ctags-stdlib, ctags-project>
other: <ctags-project>


      +-------+
    +-|-----+ |         +---------+           +---------+
  +-|-|----+| |         |         |  generate |  tags   |
  | | +-------+ ------->|  ctags  |---------->|  file   |
  | +-------+      |    |         |           |         |
  +--------+       |    +---------+           +---------+
  Source code      |
                   |    +---------+           +---------+
                   |    |         |  generate | cscope  |
                   +--->| cscope  |---------->|  tag    |
                        |         |           |         |
                        +---------+           +---------+

Creating cscope and ctags database for vim

In order to establish cscope and ctags database for tracing source code, we can use the run_cscope_ctags.sh script to automatically create/update database.

Usage under shell after the script is included:

Run run_cscope_ctags to create database
Run clean_cscope_ctags to clean database
Run run_cscope_ctags while database exists to update database

Click to expand run_cscope_ctags.sh

#!/bin/bash

tags_update="n"
tags_create="n"
cscope_update="n"
cscope_create="n"

function make_ctags()
{
    ctags_option="-R    \
     --exclude=.git  \
     --exclude=out   \
     --c++-kinds=+p  \
     --fields=+iaS   \
     --extra=+q ."

    ctags_option2="-R   \
     --exclude=.git  \
     --exclude=out   \
     --extra=+q ."

    if [ $1 == "update" ]; then
        ctags -o newtags $ctags_option2
        rm -f tags
        mv newtags tags
    elif [ $1 == "create" ]; then
        ctags $ctags_option2
    else
        echo Error: wrong ctags option, check the code
        exit 1
    fi  
}

function make_cscope()
{
    find `pwd` -path ./out -prune               \
     -o -name "*.aidl"   -exec echo \"{}\" \;\
     -o -name "*.asm"    -exec echo \"{}\" \;\
     -o -name "*.s"      -exec echo \"{}\" \;\
     -o -name "*.S"      -exec echo \"{}\" \;\
     -o -name "*.h"      -exec echo \"{}\" \;\
     -o -name "*.c"      -exec echo \"{}\" \;\
     -o -name "*.cpp"    -exec echo \"{}\" \;\
     -o -name "*.cc"     -exec echo \"{}\" \;\
     -o -name "*.java"   -exec echo \"{}\" \;\
     -o -name "*.xml"    -exec echo \"{}\" \;\
     -o -name "*.dtsi"   -exec echo \"{}\" \;\
     -o -name "*.dts"    -exec echo \"{}\" \;\
     -o -name "*.rc"     -exec echo \"{}\" \;\
     -o -name "*.mk"     -exec echo \"{}\" \;\
     -o -name "*.sh"     -exec echo \"{}\" \;\
     -o -name "*.kl"     -exec echo \"{}\" \;\
     > cscope.files

    if [ $1 == "update" ]; then
        cscope -bkq -i cscope.files -f newcscope.out
        rm -f cscope.out cscope.out.in cscope.out.po
        mv newcscope.out cscope.out
        mv newcscope.out.in cscope.out.in
        mv newcscope.out.po cscope.out.po
    elif [ $1 == "create" ]; then
        cscope -bkq -i cscope.files -f cscope.out
    else
        echo Error: wrong cscope option, check the code
        exit 1
    fi
    return 0
}

#
# Check ctags
#
function check_ctags()
{
    if [ -f "tags" ]; then
        read -p "tags file exists, update it? (y/n) " tags_update

        if [ $tags_update = "y" ] || [ $tags_update = "Y" ]; then
            echo "tags is to update"
        elif [ $tags_update = 'n' ] || [ $tags_update = 'N' ]; then
            echo "tags not updated"
        else
            echo "wrong option, quit"
            exit 1
        fi
    else
        read -p "tags does not exist, create new tags? (y/n) " tags_create
    fi
}

#
# Check cscope
#
function check_cscope()
{
    if [ -f "cscope.out" ]; then
        read -p "cscope.out exists, update it? (y/n) " cscope_update

        if [ $cscope_update = "y" ] || [ $cscope_update = "Y" ]; then
            echo "cscope files is to update"
        elif [ $cscope_update = 'n' ] || [ $cscope_update = 'N' ]; then
            echo "cscope files not updated"
        else
            echo "wrong option, quit"
            exit 1
        fi
    else
        read -p "cscope.out does not exist, create new cscope.out? (y/n) " cscope_create
    fi
}

#
# Create ctags and cscope
#
function create_ctags_cscope()
{
    if [ $tags_create = "y" ] || [ $tags_create = "Y" ]; then
        echo "creating tags file"
        make_ctags "create";
        echo "tags created"
    fi

    if [ $cscope_create = "y" ] || [ $cscope_create = "Y" ]; then
        echo "creating cscope.out"
        make_cscope "create";
        echo "cscope.out created"
    fi
}

#
# Update ctags and cscope
#
function update_ctags_cscope()
{
    if [ $tags_update = "y" ] || [ $tags_update = "Y" ]; then
        echo "updating tags file"
        make_ctags "update";
        echo "tags updated"
    fi

    if [ $cscope_update = "y" ] || [ $cscope_update = "Y" ]; then
        echo "updating cscope files"
        make_cscope "update";
        echo "cscope.out updated"
    fi
}

#
# Clean current database
#
function clean_ctags_cscope()
{
    rm -f cscope.* ncscope.* tags
}

#
# Main function
#
function run_cscope_ctags()
{
    check_ctags;
    check_cscope;
    create_ctags_cscope;
    update_ctags_cscope;
}

Example:

$ cd $PRJ_DIR
$ run_cscope_ctags 
tags does not exist, create new tags? (y/n) y
cscope.out does not exist, create new cscope.out? (y/n) y
creating tags file
tags created
creating cscope.out
cscope.out created

Then if you run vim in the current directory, these databases will be loaded automatically because they use standard names as: tags and cscope.out.

Seting up cscope and ctags in Vim

As we mentioned, ctags runs faster then cscope. So we should let tag file(s) are searched before cscope database(s). There is a cscope’s option to cover this, which is csto. By default is zero, which means it lets cscope databased(s) are searched first. So we need to set this value to one in $HOME/.vimrc.

if has("cscope")
    set csto=1
endif

Sometimes there are some points which clear this option, so we should set an automatic command to set it to one.

autocmd BufEnter * set csto=1

cscope usage

There are two ways to use cscope:

Use native cscope program by calling it in your shell.

$ cscope
Find this C symbol:
Find this global definition:
Find functions called by this function:
Find functions calling this function:
Find this text string:
Change this text string:
Find this egrep pattern:
Find this file:
Find files #including this file:

Use cscope by command :cs in Vim.

:cs help
cscope commands:
add  : Add a new database             (Usage: add file|dir [pre-path] [flags])
find : Query for a pattern            (Usage: find c|d|e|f|g|i|s|t name)
    c: Find functions calling this function
    d: Find functions called by this function
    e: Find this egrep pattern
    f: Find this file
    g: Find this definition
    i: Find files #including this file
    s: Find this C symbol
    t: Find this text string
help : Show this message              (Usage: help)
kill : Kill a connection              (Usage: kill #)
reset: Reinit all connections         (Usage: reset)
show : Show connections               (Usage: show)
Press ENTER or type command to continue