Interact with the system when Perl isn’t enough

Usually, you want to do as much of your work inside your Perl program as you can. CPAN has a wealth of modules that accomplish many tasks, including those that you normally do very easily from the command line. In the cases where you can’t find a module, you might not be able to improve on the command-line siutation for complicated tasks. Perl is, after all, a glue language, so you don’t always have to do everything in Perl. When you decide to use an external program, there are many Perl features waiting to help, although you have to choose the right one for the job.

Backticks capture output

One of the easiest and most common methods of running a command on the underlying operating system is to simply put the command in backticks. Whatever text you put in in the backticks is executed on the system. The backticks also trigger the same interpolation that double-quotes do, so you can place variables in the backticks and they will be converted into text before Perl executes the command.

If you use the backticks in scalar context, Perl returns all of the standard output (but not standard error!) as a single, possibly multi-line string. If you use it in list context, then Perl returns a list that has one line per element:

# Full directory listing in a single scalar
my $output = `ls -l`;

# Full directory listing, one line per list item
my @output = `ls -l`;

The backticks have a generalized quoting syntax too. The qx{} quoting is the same as the literal `` but allows you to choose the delimiter (see perlop). These are all the same code:

my $output = `ls -l`;

my $output = qx{ls -l};

my $output = qx(ls -l);

Remember, the backticks return only the standard output, so you have to do more work to grab the standard error. If your particular shell command does not support merging output streams, you might need a shell feature to tell it to combine standard error (conventionally, file descriptor 2), with standard output (file descriptor 1). The syntax is a bit odd because you have to use a bit of Bourne shell magic (and if you aren’t using that shell, you’ll have a different goofy syntax):

my $output = `scp xyz\@example.com:x.dat ./ 2>&1`

Detecting errors from external commands

In modern Perl, you can mostly skip this section because you should use IPC::System::Simple, but we don’t talk about that until the end of this Item. If you can’t use that module, you just have a little extra work to do. For the most part, this section applies to Perl’s interaction with all external commands, not just those you fire off from backticks.

If there is an error with an external command (not just through backticks), Perl sets the child error special variable, $?, with the error value. If $?, you had a problem with the last external command. If $? is false, then your command exited successfully. If you prefer the English, you can use
the $CHILD_ERROR variable instead of $?:

my $output = `ls -l`;
die "something went horribly wrong' if $CHILD_ERROR;

The value in $? is acutally more than a single value. It’s two bytes, and you need to look at certain bits to extract values:

if ( $? == -1 ) {
	print "Command failed to execute: $!\n";
	}
elsif ( $? & 127 ) {
	printf "The child died with signal %d, %s a coredump\n",
	( $? & 127 ), ( $? & 128 ) ? 'with' : 'without';
	}
else {
	printf "child exited with value %d\n", $? >> 8;
	}

Fire and forget with system

If you don’t care about the output, you can use system to start a command. It’s output goes to the same place your Perl output goes to (e.g. to the screen):

system( 'ls', '-l' )
	and die 'something went horribly wrong';

Your program waits (“blocks”) while the external command runs, and your program only continues once the external command completes.

The return value from system is the exit value from the external command. Each command can define its own exit values, but most commonly they return 0 for success or a non-zero number to indicate a particular sort of failure. At first glance the and looks odd, but that’s because an error from ls returns a value that Perl considers true.

If you are using autodie (Item 27: Use autodie to simplify error handling), you don’t have to do that work yourself.

Turn into other programs with exec

The exec function is similar to system, but it turns your program into that other process. If you can run that command, you never come back to your Perl program. There’s usually no point in continuing if your exec fails:

unless( exec( 'myprogram.sh', $@ ) ) {
	die "Could not exec! Oh noes!\n"
	}

That’s not really Perl programming, but many people use exec after they set up an environment or do other work to set up the situation. Essentially, this is a wrapper program:

my $line = ;

$ENV{DEBUG} = 1 if $line =~ /debug\s+on/i;

exec( 'myprogram.sh', $@ );

Use IPC::System::Simple to make life easier

Running external commands from within a Perl program is
easy enough, but you can easily get confused about whether or not those
commands succeeded or failed. IPC::System::Simple provides its own versions of system that dies when it encounters an error:

use IPC::System::Simple qw(system);
use Try::Tiny;

try {
	system( 'ls', '-l', $ARGV[0] );
	}
catch {
	print $_;
	};

When you run this command, you see that IPC::System::Simple figures out the success or failure for you (much like autodie):

% perl ipc_system_simple.pl /does/exist
-rw-rw-r-- 1 buster  staff  555 Jan  1 00:00 /does/exist
% perl ipc_system_simple.pl /does/not/exist
ls: /does/not/exist: No such file or directory
"ls" unexpectedly returned exit value 1 at ipc_system_simple.pl line 5

IPC::System::Simple also has a replacement for backticks, which it calls capture. You could run the ps command to get a list of running processes then look for the line with your program ($0 is the name of the currently running Perl program):

use IPC::System::Simple qw(capture);

my @processes = capture( 'ps', 'ux' );

print grep { /$0/o } @processes;

This program just prints the ps line that applies to itself:

% perl ipc_system_simple_capture.pl
buster 74442   0.0  0.1   601344   2876 s000  S+   11:08 AM 0:00.04 perl ipc_system_simple_capture.pl

IPC::System::Simple‘s system and capture commands behave much like Perl’s built-in system command, so they might invoke an underlying shell. If you want to prevent an shell interaction (i.e. don’t interpret metacharacters), you can use the systemx and capturex versions instead. This prevents you from accidently starting a command that does more than you wanted:

use IPC::System::Simple qw(systemx);
systemx("ps ux | rm -rf /");

The systemx thinks its argument a file with the literal characters you specified in the argument, and fails instead of trying to remove all of your files:

% perl ipc_system_simple_systemx.pl
"ps ux | rm -rf /" failed to start: "No such file or directory" at ipc_system_simple_system x.pl line 2

Use IPC::Run3 to connect filehandles to external processes

Backticks, system, exec, and IPC::System::Simple are all fine ways of running processes in Perl; however, they don’t do a great job in working with output streams. What if you want to write to an external program, or if you want to both read and write to the same external process? In those cases, you want the IPC::Run3 module. This module is the modern successor to the core module IPC::Open3, which though more flexible, is also more complex to use.

IPC::Run3 is best served by an example. In this example you are going to send some input to the sort command and capture the standard input, and standard output.

With IPC::Run3, you can define a scalar variable ($stdin) that represents the input, and the scalar variables where it should put the output ($stdout and $stderr):

use IPC::Run3;
my @command = qw(sort -n);

my $stdin   = "5\n34\n32\n12\nhi\n";
my ( $stdout, $stderr ) = ( '', '' );

run3 \@command, \$stdin, \$stdout, \$stderr;

die "something went horribly wrong" if $?;

print "STDOUT\n$stdout\nSTDERR\n$stderr\n";

You can even do fancy things like bypass writing to a scalar and directly write to a file. You can also choose to ignore any or all of the standard filehandles and instead just run the command directly, though it seems like IPC::Run3 might be more than you are needing in that case.

Things to remember

• The backticks capture output.
• The system command does not capture output
• A successful exec never returns
• IPC::System::Simple handles most of the shell and error details for you.
• IPC::Run3 allows you to work with input and output at the same time