Vim Statusline: Word search

Question

I've searched hard for this and haven't been able to find what I'm after.

On my statusline I want a count of the number of matches that occur in the current file. The vim command below returns what I want. I need the returned number to be displayed in my statusline.

:%s/^I^I//n

vim returns: 16 matches on 16 lines

FYI Explanation: I'm working in a CSV file. I'm searching for two tab characters ( ^I^I ) because that indicates lines I still need to do work on. So my desired statusline would indicate how much work remains in the current file.

I don't know how to enter a vim command on the statusline, I know that %{} can be used to run a function but how do I run the vim search command? I've tried variations of the following, but they clearly aren't right and just end up with an error.

:set statusline+= %{s/^I^I//n}

Help me vimy one kenobi, you're my only hope!

Answer 1

The first thing to mention here is that for large files this feature would be completely impractical. The reason is that the status line is redrawn after every cursor movement, after the completion of every command, and probably following other events that I am not even aware of. Performing a regex search on the entire buffer, and furthermore, not just the current buffer, but every visible window (since every window has its own status line), would slow things down significantly. Don't get me wrong; the idea behind this feature is a good one, as it would give you an immediate and fully automated indication of your remaining work, but computers are simply not infinitely performant (unfortunately), and so this could easily become a problem. I've edited files with millions of lines of text, and a single regex search can take many seconds on such buffers.

But provided your files will remain fairly small, I've figured out three possible solutions by which you can achieve this.

Solution #1: exe :s and redirect output

You can use :exe from a function to run the :s command with a parameterized pattern, and :redir to redirect the output into a local variable.

Unfortunately, this has two undesirable side effects, which, in the context of this feature, would be complete deal-breakers, since they would occur every time the status line is redrawn:

1. The cursor is moved to the start of the current line. (Personal note: I've never understood why vim does this, whether you're running :s from a status line call or by manually typing it out on the vim command-line.)
2. The visual selection, if any, is lost.

(And there actually might be more adverse effects that I'm not aware of.)

The cursor issue can be fixed by saving and restoring the cursor position via getcurpos() and setpos(). Note that it must be getcurpos() and not getpos() because the latter does not return the curswant field, which is necessary for preserving the column that the cursor "wants" to reside at, which may be different from the column the cursor is "actually" at (e.g. if the cursor was moved into a shorter line). Unfortunately, getcurpos() is a fairly recent addition to vim, namely 7.4.313, and based on my testing doesn't even seem to work correctly. Fortunately, there are the older winsaveview() and winrestview() functions which can accomplish the task perfectly and compatibly. So for now, we'll use those.

Solution #1a: Restore visual selection with gv

The visual selection issue I thought could be solved by running gv in normal mode, but for some reason the visual selection gets completely corrupted when doing this. I've tested this on Cygwin CLI and Windows gvim, and I don't have a solution for this (with respect to restoring the visual selection).

In any case, here's the result of the above design:

fun! MatchCount(pat,...)
    "" return the number of matches for pat in the active buffer, by executing an :s call and redirecting the output to a local variable
    "" saves and restores both the cursor position and the visual selection, which are clobbered by the :s call, although the latter restoration doesn't work very well for some reason as of vim-7.4.729
    "" supports global matching (/g flag) by taking an optional second argument appended to :s flags
    if (a:0 > 1)| throw 'too many arguments'| endif
    let flags = a:0 == 1 ? a:000[0] : ''
    let mode = mode()
    let pos = winsaveview()
    redir => output| sil exe '%s/'.a:pat.'//ne'.flags| redir END
    call winrestview(pos)
    if (mode == 'v' || mode == 'V' || mode == nr2char(22))
        exe 'norm!gv'
    endif
    if (match(output,'Pattern not found') != -1)
        return 0
    else
        return str2nr(substitute(output,'^[\s\n]*\(\d\+\).*','\1',''))
    endif
    return 
endfun

set statusline+=\ [%{MatchCount('\\t\\t')}]

A few random notes:

• The use of ^[\s\n]* in the match-count extraction pattern was necessary to barrel through the leading line break that gets captured during the redirection (not sure why that happens). An alternative would be to skip over any character up to the first digit with a non-greedy multiplier on the dot atom, i.e. ^.\{-}.
• The doubling of the backslashes in the statusline option value is necessary because backslash interpolation/removal occurs during parsing of the option value itself. In general, single-quoted strings do not cause backslash interpolation/removal, and our pat string, once parsed, eventually gets concatenated directly with the :s string passed to :exe, thus there's no backslash interpolation/removal at those points (at least not prior to the evaluation of the :s command, when backslash interpolation of our backslashes does occur, which is what we want). I find this to be slightly confusing, since inside the %{} construct you'd expect it to be a normal unadulterated VimScript expression, but that's the way it works.
• I added the /e flag for the :s command. This is necessary to handle the case of a buffer with zero matches. Normally, :s actually throws an error if there are zero matches. For a status line call, this is a big problem, because any error thrown while attempting to redraw the status line causes vim to nullify the statusline option as a defensive measure to prevent repeated errors. I originally looked for solutions that involved catching the error, such as :try and :catch, but nothing worked; once an error is thrown, a flag is set in the vim source (called_emsg) that we can't unset, and so the statusline is doomed at that point. Fortunately, I discovered the /e flag, which prevents an error from being thrown at all.

Solution #1b: Dodge visual mode with a buffer-local cache

I wasn't satisfied with the visual selection issue, so I wrote an alternative solution. This solution actually avoids running the search at all if visual mode is in effect, and instead pulls the last-known search count from a buffer-local cache. I'm pretty sure this will never cause the search count to become out-of-date, because it is impossible to edit the buffer without abandoning visual mode (I'm pretty sure...).

So now the MatchCount() function does not mess with visual mode:

fun! MatchCount(pat,...)
    if (a:0 > 1)| throw 'too many arguments'| endif
    let flags = a:0 == 1 ? a:000[0] : ''
    let pos = winsaveview()
    redir => output| sil exe '%s/'.a:pat.'//ne'.flags| redir END
    call winrestview(pos)
    if (match(output,'Pattern not found') != -1)
        return 0
    else
        return str2nr(substitute(output,'^[\s\n]*\(\d\+\).*','\1',''))
    endif
    return 
endfun

And now we need this helper "predicate" function which tells us when it's (not) safe to run the :s command:

fun! IsVisualMode(mode)
    return a:mode == 'v' || a:mode == 'V' || a:mode == nr2char(22)
endfun

And now we need a caching layer that branches on the predicate result and only runs the primary function if safe, otherwise it pulls from the buffer-local cache the last-known return value that was captured from the most recent call of the primary function taking those exact arguments:

fun! BufferCallCache(buf,callName,callArgs,callElseCache)
    let callCache = getbufvar(a:buf,'callCache')
    if (type(callCache) != type({}))
        unlet callCache
        let callCache = {}
        call UnletBufVar(a:buf,'callCache')
        call setbufvar(a:buf,'callCache',callCache)
    endif
    if (a:callElseCache)
        let newValue = call(a:callName,a:callArgs)
        if (!has_key(callCache,a:callName.':Args') || !has_key(callCache,a:callName.':Value'))
            let callCache[a:callName.':Args'] = []
            let callCache[a:callName.':Value'] = []
        endif
        let i = len(callCache[a:callName.':Args'])-1
        while (i >= 0)
            let args = callCache[a:callName.':Args'][i]
            if (args == a:callArgs)
                let callCache[a:callName.':Value'][i] = newValue
                return newValue
            endif
            let i -= 1
        endwhile
        let callCache[a:callName.':Args'] += [a:callArgs]
        let callCache[a:callName.':Value'] += [newValue]
        return newValue
    else
        if (has_key(callCache,a:callName.':Args') && has_key(callCache,a:callName.':Value'))
            let i = len(callCache[a:callName.':Args'])-1
            while (i >= 0)
                let args = callCache[a:callName.':Args'][i]
                if (args == a:callArgs)
                    return callCache[a:callName.':Value'][i]
                endif
                let i -= 1
            endwhile
        endif
        return ''
    endif
endfun

For which we need this helper function which I found somewhere on the Internet years ago:

fun! UnletBufVar(bufExpr, varName )
    "" source: <http://vim.1045645.n5.nabble.com/unlet-ing-variables-in-buffers-td5714912.html>
    call filter(getbufvar(a:bufExpr,''), 'v:key != '''.a:varName.'''' )
endfun

And finally this is how we can set the statusline:

set statusline+=\ [%{BufferCallCache('','MatchCount',['\\t\\t'],!IsVisualMode(mode()))}]

Solution #2: Call match() on every line

I've thought of another possible solution which is actually much simpler, and seems to perform just fine for non-huge files, even though it involves more looping and processing at the VimScript level. This is to loop over every line in the file and call match() on it:

fun! MatchCount(pat)
    "" return the number of matches for pat in the active buffer, by iterating over all lines and calling match() on them
    "" does not support global matching (normally achieved with the /g flag on :s)
    let i = line('$')
    let c = 0
    while (i >= 1)
        let c += match(getline(i),a:pat) != -1
        let i -= 1
    endwhile
    return c
endfun

set statusline+=\ [%{MatchCount('\\t\\t')}]

Solution #3: Call search()/searchpos() repeatedly

I've written some slightly intricate functions to perform global and linewise matching, built around searchpos() and search(), respectively. I've included support for optional start and end bounds as well.

fun! GlobalMatchCount(pat,...)
    "" searches for pattern matches in the active buffer, with optional start and end [line,col] specifications
    "" useful command-line for testing against last-used pattern within last-used visual selection: echo GlobalMatchCount(@/,getpos("'<")[1:2],getpos("'>")[1:2])
    if (a:0 > 2)| echoerr 'too many arguments for function: GlobalMatchCount()'| return| endif
    let start = a:0 >= 1 ? a:000[0] : [1,1]
    let end = a:0 >= 2 ? a:000[1] : [line('$'),2147483647]
    "" validate args
    if (type(start) != type([]) || len(start) != 2 || type(start[0]) != type(0) || type(start[1]) != type(0))| echoerr 'invalid type of argument: start'| return| endif
    if (type(end) != type([]) || len(end) != 2 || type(end[0]) != type(0) || type(end[1]) != type(0))| echoerr 'invalid type of argument: end'| return| endif
    if (end[0] < start[0] || end[0] == start[0] && end[1] < start[1])| echoerr 'invalid arguments: end < start'| return| endif
    "" allow degenerate case of end == start; just return zero immediately
    if (end == start)| return [0,0]| endif
    "" save current cursor position
    let wsv = winsaveview()
    "" set cursor position to start (defaults to start-of-buffer)
    call setpos('.',[0,start[0],start[1],0])
    "" accumulate match count and line count in local vars
    let matchCount = 0
    let lineCount = 0
    "" also must keep track of the last line number in which we found a match for lineCount
    let lastMatchLine = 0
    "" add one if a match exists right at start; must treat this case specially because the main loop must avoid matching at the cursor position
    if (searchpos(a:pat,'cn',start[0])[1] == start[1])
        let matchCount += 1
        let lineCount += 1
        let lastMatchLine = 1
    endif
    "" keep searching until we hit end-of-buffer
    let ret = searchpos(a:pat,'W')
    while (ret[0] != 0)
        "" break if the cursor is now at or past end; must do this prior to incrementing for most recent match, because if the match start is at or past end, it's not a valid match for the caller
        if (ret[0] > end[0] || ret[0] == end[0] && ret[1] >= end[1])
            break
        endif
        let matchCount += 1
        if (ret[0] != lastMatchLine)
            let lineCount += 1
            let lastMatchLine = ret[0]
        endif
        let ret = searchpos(a:pat,'W')
    endwhile
    "" restore original cursor position
    call winrestview(wsv)
    "" return result
    return [matchCount,lineCount]
endfun

fun! LineMatchCount(pat,...)
    "" searches for pattern matches in the active buffer, with optional start and end line number specifications
    "" useful command-line for testing against last-used pattern within last-used visual selection: echo LineMatchCount(@/,getpos("'<")[1],getpos("'>")[1])
    if (a:0 > 2)| echoerr 'too many arguments for function: LineMatchCount()'| return| endif
    let start = a:0 >= 1 ? a:000[0] : 1
    let end = a:0 >= 2 ? a:000[1] : line('$')
    "" validate args
    if (type(start) != type(0))| echoerr 'invalid type of argument: start'| return| endif
    if (type(end) != type(0))| echoerr 'invalid type of argument: end'| return| endif
    if (end < start)| echoerr 'invalid arguments: end < start'| return| endif
    "" save current cursor position
    let wsv = winsaveview()
    "" set cursor position to start (defaults to start-of-buffer)
    call setpos('.',[0,start,1,0])
    "" accumulate line count in local var
    let lineCount = 0
    "" keep searching until we hit end-of-buffer
    let ret = search(a:pat,'cW')
    while (ret != 0)
        "" break if the latest match was past end; must do this prior to incrementing lineCount for it, because if the match start is past end, it's not a valid match for the caller
        if (ret > end)
            break
        endif
        let lineCount += 1
        "" always move the cursor to the start of the line following the latest match; also, break if we're already at end; otherwise next search would be unnecessary, and could get stuck in an infinite loop if end == line('$')
        if (ret == end)
            break
        endif
        call setpos('.',[0,ret+1,1,0])
        let ret = search(a:pat,'cW')
    endwhile
    "" restore original cursor position
    call winrestview(wsv)
    "" return result
    return lineCount
endfun

Answer 2

Maybe not exactly what you are looking for, but if you put a function like the following in your $HOME/.vimrc file you could do:

:set statusline+=%!SearchResults('^I^I')

$HOME/.vimrc

function SearchResults(q)
  redir => matches
  silent! execute "%s/".a:q."//n"
  redir END
  return substitute(matches, "^.", "", "")
endfunction

If nothing else, maybe that will get you a little closer.