How to wait for Modelsim Simulations to complete before proceeding in TCL script

Question

I am trying to execute a regression test in Modelsim. I call a TCL script which compiles my source files and launches vsim. I launch a .do file and it runs a series of testbenches which all output result files. What I am adding is an automated checker that verifies the result files match known good runs. The problem is that after launching modelsim the TCL script doesn't wait for the completion of simulation before executing the checker "results_pass.py".

set nice_pid [open "|$modelsim_dir/vsim.exe -do do_files/vsim.do -novopt -gui"]
cd ../../Scripts
set script_name "results_pass.py"
set tb_name "rcp"
call_python $script_name $tb_name
vwait forever

For those wondering why I am calling a Python script. Mostly because I know very little TCL but don't have the time to go and change the legacy TCL script into Python.

Anyway, I am confident there is something I can stick between lines 1 and 2 that will wait for some indication from modelsim that it has completed executing my .do file. Any advice is appreciated.

Answer 1

You should open the subprocess with read-write and set a fileevent to listen for readable events from the subprocess pipe. When the subprocess exits it closes its stdout and you will receive a readable event with [eof] being true when reading from this pipe.

If you are not too familiar with tcl and the asynchronous channel handling I've done a quick example that uses two tcl scripts, the client driving the server process. When the server gets "quit" on stdin it will exit. The client process sees that the channel gets closed and cleans up then exits. The key commands are fconfigure to set the channel to non-blocking and fileeventto set a procedure to be called whenever something readable happens on the pipe.

server.tcl

proc main {} {
    while {[gets stdin line] != -1} {
        if {[string match "quit" [string trim $line]]} {
            break
        }
        puts "ECHO $line"
    }
    return 0
}
main

client.tcl

proc Read {chan} {
    set len [gets $chan line]
    if {[eof $chan]} {
        puts stderr "eof received"
        set ::forever quit
        fileevent $pipe readable {}
    }
    puts stderr "RECV: $len '$line'"
}
proc main {} {
    set pipe [open |[list tclsh server.tcl] r+]
    fconfigure $pipe -blocking 0 -buffering line -translation auto
    fileevent $pipe readable [list Read $pipe]
    after 500 [list puts $pipe begin]
    after 1000 [list puts $pipe quit]
    after 2000 [list set ::forever timeout]
    vwait ::forever
    puts "exit with status '$::forever'"
    return 0
}
main

expected output

c:\src>tclsh client.tcl
RECV: 10 'ECHO begin'
eof received
exit quit

Answer 2

TCL is actually returning a channel from the open call, not a pid. There are a couple of things you can do:

1. Use exec command instead of opening a pipe because you clearly don't want asynchronous I/O.
2. Use the close command to close the pipe which I believe waits for the process to close via the wait() system call.

Answer 3

Answering my own question I wound up getting this to work before receiving responses. The other suggested solutions may be better.

First I modified my .do file adding line three below.

project open my_project.mpf
do do_files/regression.do
file delete -force "lock.txt"

Then I modified my TCL script as follows.

set nice_pid [open "|$modelsim_dir/vsim.exe -do do_files/vsim.do -novopt -gui"]

,# Create a "lock file" that is cleared at the completion of the .do file.
set lock_file "lock.txt"
set lock_PID [open $lock_file w]
close $lock_PID  

while {[file exists $lock_file]} 
    {after 1000}

cd ../../Scripts
set script_name "results_pass.py"
set tb_name "my_tb"
call_python $script_name $tb_name
vwait forever

This way the calling process waits for Modelsim to complete before proceeding.

For anyone looking to use this here is call_python proc which I found here

;#----------------------------------------------------------------------------;
;# Procedure to call python scripts. Include ".py" extention.
;#----------------------------------------------------------------------------;

proc call_python {script_name tb_name} {
    set output [exec python $script_name $tb_name]
    print $output
}

$tb_name is just an argument for my python script.

Answer 4

Have you tried VUnit?https://github.com/LarsAsplund/vunit

It is an open source VHDL test framework that already does what you want.

It can run tests in parallel as well as with different generics values.

A test bench can have multiple tests which can be run in independent simulations as well as in the same simulation.

It can emit a test report that is understood by Jenkins containing test output, runtime and status.

It comes with a VHDL library of convenience functions such as check_equal.

It has complete VHDL dependency scanning so that files can just be added and VUnit will know what to incrementally compile to speedup the edit/compile/run cycle.

Can run user defined post simulation checks to verify for instance an output file against golden data.

In addition to running in batch it can launch the test in a GUI with a single flag.

It supports Modelsim and GHDL with Aldec and NVC coming soon.

I have personally used it to manage 200+ test cases.

Disclaimer I am one of the main authors. It was created to avoid each VHDL team in the world to re-inventong the wheel with inhouse scripts of varying quality.

Answer 5

I managed this problem of automating Modelsim from Python by defining a dummy sentinal proc in Tcl. vsim is run with pipes for the I/O channels. The sentinel command is sent after every command I want to run. Then I wait to see the "sentinel" text appear in the output stream which guarantees that the previous command completed.

# Define a dummy sentinel proc
self.p.stdin.write('proc sentinel {} {}\n')

self.p.stdin.write(cmd + '\n')
self.p.stdin.write('sentinel\n')
self.p.stdin.flush()

while True:
  if self.process_done(): break  # The process died

  out.append(get_output(self.outq))
  if '> sentinel' in out[-1]: break # Stop when we see the prompt with the sentinel

You can see the full automation code here. A similar approach can be used when controlling Modelsim from a separate Tcl interpreter.