MATLAB_NOHUP
Running MATLAB As a Noninteractive Job


This document describes several ways of running MATLAB noninteractively.

Although MATLAB is naturally an interactive program, there are times when it is appropriate, convenient or necessary to run it without interaction.

In particular, consider the case in which a user logs into a remote computer system and issues a MATLAB command which is going to take hours to complete. If the user logs out, MATLAB will terminate without completing the work. Even if the user stays logged in, the user's desktop computer might go down, or the SSH connection might be interrupted, again causing MATLAB to terminate.

It seems more natural, in such cases, to start the MATLAB job and somehow be able to ignore it until it completes. This is possible, using some simple UNIX commands. And in fact, this approach is necessary, if the MATLAB command is to be executed through a queuing system.

Case 1: The User Wants the Prompt Back

If you are working on a UNIX system, whether it's your desktop or a remote machine you have contacted by SSH, you are accustomed to the way you interact with UNIX. That is, the system prompts you for input, you issue a command, the system executes the command, and then you get a new prompt. If the command executes quickly, you don't even notice the wait for the new prompt.

But suppose you need to run a program called monster which takes 15 minutes to complete. If you simply issue this command, you will be waiting a long time for the prompt, with no way to do other work. However, UNIX offers a way to indicate that certain commands are to be carried out in the background. That is, UNIX starts working on the command, but also immediately returns a prompt, allowing you to do other work while waiting. A command that is to be executed in the background is terminated by the ampersand symbol &.

In the following example, the user types 3 commands, "date", "monster", and "date" again. Because the "monster" command is followed by an ampersand, the program begins computing, but UNIX returns a prompt immediately, which is why the date command shows that only 2 seconds have passed, although it's going to take the monster program 15 minutes to complete!

        prompt> date
        Tue Dec  1 10:09:04 EST 2009
        prompt> ./monster &
        [1] 617
        prompt> date
        Tue Dec  1 10:09:06 EST 2009
      

The number 617 is a process identifier that can be used to keep track of your job. In particular, you need that number if you want to kill the job. You would do that with the kill command:

        kill -9 617
      

Now it might seem a little strange that the "monster" job is running in the background. How can you tell if it's running? Where does its printed output go?

If the program prints to the screen, then it will still do so. So if you're running monster in the background, and you have the prompt back, then from time to time, you might see output from the program show up on your screen, sometimes mixed up with the new commands you are entering. If, on the other hand, the program only writes to files, or doesn't write to the screen, then you won't see anything. If you are wondering whether the job is still running, try typing the jobs command:

        prompt> jobs
        [1]+ Running     ./monster &
      
but eventually you will see, instead:
        prompt> jobs
        [1]+ Done     ./monster &
      

Case 2: The User Wants the Output to be Saved

Having the prompt back while your big job runs is nice, but it can be annoying to see messages from the job showing up on your screen. What's worse, those messages might be important results that you need to keep a record of. Luckily, there's a way to take care of both problems. We simply use output redirection, which sends the "standard output" of the program to a file that we name. Output redirection is accomplished using a > sign followed by the name of a file where the results are to be stored.

Assuming that we still want our job to run in the background, here is how we request that its output go to a file, perhaps named output.txt, rather than showing up on the terminal:

         ./monster > output.txt &
      

Some programs write output to "standard error", particularly if an error occurred. Output to standard error will not be redirected by the above commands; instead, it will still appear directly on the console or terminal window. If you have output to standard error, and you also want to redirect that, then you must use the 2> to redirect that output to some file, as in this example:

         ./monster > output.txt 2> error.txt &
      
I don't use output to standard error that much, and I will assume that if you do you can remember to include this extra redirection statement when you consider the examples that follow.

Case 3: The User Wants Input to Come from a File

Many programs are interactive; even if a program is not 'chatty', it may still expect to read a small number of parameter values from the user before beginning a big computation. If we run a program in the background, then the connection to the user is broken, and input cannot be entered interactively. However, if you know exactly what input you need to give, you can prepare that data in a file, and use input redirection so that the program expects its "standard input" to come from that file instead of from the user. Input redirection is accomplished using a < sign followed by the name of a file from which the data is to be read.

When both input and output redirection are used, the convention is that the input is listed first, followed by the output.

So now let us assume that our program reads an input file, perhaps named input.txt, and writes an output file, and is to be run in the background. Our command now becomes:

        ./monster < input.txt > output.txt &
      

Case 4: The User Wants to Log Out

Now suppose that the monster program is going to run for hours or days, and that the user wants to log out of the machine. The program could be running on the user's desktop machine, or on a remote machine which has been reached using ssh. In either case, if the program is running and the user logs out, the UNIX system sends a signal to the program indicating that the user is gone, and that the program should terminate immmediately.

However, in this case, we want to be able to log out, and have the program ignore the signal from the operating system. If we can manage that, then the program will continue to run (even though we're not logged in) all the way to completion, unless for some reason the computer system itself goes down.

To indicate that a command you have given is to continue to run even if you log out, you preface the command with nohup, which is short for "no hang up".

So now (take a deep breath), if we want to run our monster command in the background, with input and output files, and have it keep running even if we log out, we just have to type something like:

        nohup ./monster < input.txt > output.txt &
      

When we log in later, the program will either have finished normally, or still be running. Often, it's easy to tell if the job has completed by looking at the output file it creates. Otherwsie, if we remembered the program's process id (which was 617 in the first example), we can find out the program's status by issuing the command

        ps -p617
      
in which case, if the program is running, we might get output like this:
  PID TTY           TIME CMD
  617 ttys002    0:04.31 monster
      
while, if the program is completed, the listing will read
   PID TTY           TIME CMD
  [1]+ Done               monster
      

Case 5: The User Wants to Run a Job through a Queueing System

Now suppose that we have access to a remote computer cluster which has some kind of queueing system. To run a job, we need to create a batch file. The batch file includes some queueing instructions, followed by one or more commands that we might have typed interactively, which run the program.

Because of the nature of the queueing system, the commands we type will NOT need the initial nohup and the terminating &. So the main issues are making sure that when the batch file begins executing, we are in the right directory (where we would be if we were issuing the commands interactively) and that the input and output files get saved correctly.

A typical batch file to run our monster program, which might be called monster.sh, might look like this:

#!/bin/bash
#
#PBS -lwalltime=00:10:00
#PBS -lnodes=1:ppn=1
#PBS -W group_list=matlab
#PBS -A matlab0001
#PBS -q pmatlab_q
#
cd $PBS_O_WORKDIR
./monster < input.txt > output.txt
      
The initial PBS commands are mostly not interesting, although you can see that one PBS command is requesting a maximum of ten minutes of execution time. The first "interesting" command is the line cd $PBS_O_WORKDIR. What this does is tell the batch job to start execution in the same directory from which the batch job was submitted. The second interesting line is simply the one that runs our job, and we've seen it already several times.

To use the queueing system, we expect that the batch file monster.sh is in the same directory where the input.txt input file and the monster program are. To request that the job be executed, we type

        qsub monster.sh
      
which sends the batch file into a queue, where it will eventually be processed. Once our job has executed, the file output.txt will show up in the same directory. There are commands to talk to the queuing system and find out whether your job is waiting, running, or perhaps deferred or rejected.

Case 6: The User Wants to Run a MATLAB Job in the Background

Now it's time to deal with MATLAB. Instead of "monster", the program we want to run is called "matlab". The input file is going to be the commands that we would normally be typing interactively. Even when using MATLAB interactively, it's possible to put a list of input commands into a script file, so you may already have had some practice in putting a sequence of MATLAB commands into a file. The output file will be MATLAB's responses. The directory where we are working will need to contain the input file and and MATLAB M-files that we have written and referred to. Since the input file is likely to be a MATLAB script file, we'll give it a .m extension so we assume it's called input.m.

Let us suppose that we simply want to run a MATLAB computation that's going to take a long time, and so we want to avoid an interactive session. On top of the usual steps, we should tell MATLAB not to try to set up the X Window graphical interface. This means we don't want the initial "splash" screen that shows the MATLAB logo, and we don't want the graphical user interface. The arguments to MATLAB that do this are -nosplash and -nodisplay.

Our command to run MATLAB noninteractively would be:

        nohup matlab -nosplash -nodisplay < input.m > output.txt &
      

This command should allow us to log out of our desktop, or from the remote system, while having MATLAB continue to execute the commands in input.m, writing the output to output.txt.

Case 7: The User Wants to Run a MATLAB Job through a Queueing System

Now suppose that we are using a remote computer system with a queueing system. The batch file we would use would look like this:

#!/bin/bash
#
#PBS -lwalltime=00:10:00
#PBS -lnodes=1:ppn=1
#PBS -W group_list=matlab
#PBS -A matlab0001
#PBS -q pmatlab_q
#
cd $PBS_O_WORKDIR
matlab -nosplash -nodisplay < input.m > output.txt
      

We will assume the batch file is called monster.sh, that this file is in the same directory at input.m. In that case, the user can submit this job to the queueing system by typing qsub monster.sh.

You can return to the HTML web page.


Last revised on 04 December 2009.