Jenkins job stuck running a .bat file trough on a remote ssh server(windows 2008 R2)

828 views Asked by At

I have a jenkins setup running on solaris. I have configured a job which runs a .bat file, using the jenkins ssh plugin, on a remote Windows 2008 R2 server having an oracle database. The .bat files does an RMAN restore in the oracle Database. I have an openssh server running on Windows 2008 R2 server. When I do a restore on smaller database, which takes less than 30 mins to run, everything works fine. But when the databse is large and the restore takes over 3 hours, the jenkins job hangs even though the rman restore srated by the job completes successfully on the server.

This is the command I run in jenkins ssh plugin command window:

/cygdrive/C/OpenSSH/bin/SSH_ImportTEST_DB.bat TEST3

This is what the .bat file does.

  1. Prepare the DB for RMAN restore(runs few sql commands as system user)
  2. Run the RMAN restore
  3. Run few more sql commands to let the Database be available for all users, reset the passwords.

Step 1 and 2 work fine but the jenkins job just shows as running step 2 even though it has completed on remote server. It never executes step 3 if the rman restore runs over 3 hours.

X:\oracle\admin\TEST3\scripts>rman auxiliary cmdfile=X:\oracle\admin\TEST3\scripts\restore_db2.sql log=Y:\oracle\NightlyBackup\Prod\restore_MSIPTST3.log 
RMAN> 2> 

This is what the restore_db2.sql contains:

DUPLICATE DATABASE TO TEST3 BACKUP LOCATION 'Y:\oracle\NightlyBackup\Prod';
exit;

I have tried the restore_db2.sql without the exit command, but that doeson't help either.

I have configured ServerAliveInterval 60 in ssh_config on the server where jenkins is running. I have also configured ClientAliveInterval 60 in sshd_config on the openssh server to rule out timeouts.

Can someone assist?

1

There are 1 answers

0
SlashGordon On

Set up your windows client as jenkins slave because cygwin openssh is not a proper way for remote jobs. In your case I would recommend the Java Web Start option. An configuration example is published on: https://wiki.jenkins-ci.org/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machines