1. PHP Pie?

    I've often had to manipulate large blobs of text—no, make that many files containing large blobs of text.

    Of course, my IDE can usually handle simple search-and-replace operations, I appreciate the simplicity of the command line interface, on most occasions.

    That's one of the reasons I love working in a unixy environment, I think. There's a bunch of utilities that embrace the command line and take simple input and deliver equally simple output. I've employed sed and awk, in the past, and I still use them to perform some very simple parsing. For example, I can often be found doing something like ps auxwww | grep ssh | awk {'print $2'} to get a list of ssh process IDs, for example.

    But almost anyone who's ever been enlightened to perl pie delights in its power. In a nutshell, I can do something like perl -p -i -e 's/foo/oof/g' somefile from the command line, and perl will digest every line of somefile and perform the substitution. Perl is very well suited to this type of operation, what with its contextual variables and all.

    I updated the code a little, below. You now must explicitly set $_.

    Read on for my PHP-based solution (lest planet-php truncate my post). I've often found myself looking for a PHP equivalent. Not to do simple substitutions, of course, but complex ones. And since I'm most comfortable with PHP, and a I have a huge library of snippets that I can dig out to quell a problem that I may have solved years ago, I've been meaning to fill this void for a while.

    Tonight, I had to come home from a dinner party, early, because my daughter was sick. Too bad, it looked like it was going to be an amazing feast, but I digress. The home-on-a-Saturday-night time left me with a bit of free time to solve one of the problems that's been floating around in my head for who-knows-how-long.

    Thus, I'm happy to present my—at least mostly—working PHP pie script.

    // Change the shebang line above to point at your actual PHP interpreter
    $interpreter = array_shift($_SERVER['argv']);
    $script = array_shift($_SERVER['argv']);
    $files = array_filter($_SERVER['argv']);
    if (!$script) {
    	fwrite(STDERR, "Usage: $interpreter <script> [files]\n");
    	fwrite(STDERR, "  Iterates script over every line of every file.\n");
    	fwrite(STDERR, "  \$_ contains data from the current line.\n");
    	fwrite(STDERR, "  If files are not provided, STDIN/STDOUT will be used.\n");
    	fwrite(STDERR, "\n");
    	fwrite(STDERR, "  Example: ./pie.php '$_ = preg_replace(\"/foo/\",\"oof\",\$_);' testfile\n");
    	fwrite(STDERR, "    Replaces every instance of 'foo' with 'oof' in testfile\n");
    	fwrite(STDERR, "\n");
    // set up function
    $func = create_function('$_', $script .';return $_;');
    if (!$files) {
    	// no files, use STDIN
    	$buf = '';
    	while (!feof(STDIN)) {
    		$buf .= $func(fgets(STDIN));
    	echo $buf;
    } else {
    	foreach ($files as $f) {
    		if (!is_dir($f) or !is_writable($f)) {
    			fwrite(STDERR, "Can't write to $f (or it's not a file)\n");
    		$buf = '';
    		foreach (file($f) as $l) {
    			$buf .= $func($l);
    		file_put_contents($f, $buf);

    Hope it helps someone out there.

    Update: I've had some people ask me why I'm reinventing the wheel. I did cover this above—I have plenty of existing PHP code snippets, and almost no perl. I also am very comfortable in PHP, but it's been years since I've been comfortable in perl.

    Here's an example of something I hacked up, today. I can (relatively) easily turn this:

    dmesg | tail -n5

    ... which returns this:

    [17214721.004000] sdc: assuming drive cache: write through
    [17214721.004000]  sdc: sdc1
    [17214721.024000] sd 7:0:0:0: Attached scsi disk sdc
    [17214721.024000] sd 7:0:0:0: Attached scsi generic sg1 type 0
    [17214722.464000] FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!

    (the first field is the time since boot... useless for my feeble human brain)


    dmesg | ./pie.php 'static $prev = false; static $boot = false; if (!$boot) {
    list($boot) = explode(" ", file_get_contents("/proc/uptime"));
    $boot = time() - (int) $boot;} if (!$_) return; list($ts, $log) = explode(" ", $_, 2);
    $ts = str_replace(array("[","]"), array("",""), $ts); $_ = date("H:i:s", $boot + $ts);
    if ($prev && ($diff = round($boot + $ts - $prev, 2))) $_ .= " (+". $diff .")"; 
    $_ .= " ".$log; $prev = $boot + $ts;' | tail -n 5

    (line breaks added for easier reading)... which returns:

    17:07:44 sdc: assuming drive cache: write through
    17:07:44  sdc: sdc1
    17:07:44 (+0.02) sd 7:0:0:0: Attached scsi disk sdc
    17:07:44 sd 7:0:0:0: Attached scsi generic sg1 type 0
    17:07:45 (+1.44) FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!

    That's the sort of thing I wouldn't be comfortable doing in perl, but I hacked up on the command line in PHP.

    8 Responses

    Feed for this Entry
    • Why re-invent the wheel? ;-)

    • Humm interesting, however a simple editor like VIM does this very nicely and you can even specify a line range :)

      Here is how to do it in vim


      This will replace all the stuff from line 10 to 50 with the keyword asked. It could also be handy to reaplce in all open buffers (files) but i got too lazy to dig into. I keep this question with his Vim Presntation :)


    • @Yann: But vim doesn't allow this from the command line, or am I missing something?
      @Ilia: The subtle but important difference to perl -p -i -e is that you can run [i]any[/i] PHP expression on each line of the processed file. Most processing could be done with Perl too of course, but some native PHP functions could become very handy.

    • Yep it works with vim in command line ( i never tought doing this before)

      In command line you just have to do this.
      vim -c :10,50s/test/work/g test.txt -c :wq

      I guess a php script could be usefull too.

    • anonymous

      2007 Feb 20 09:05

      > a simple editor like VIM

      erm, if Vim is 'simple' then what is complex ?

      anyway :

      - Vim does not run on windows (well, with cygwin it could)...

      - PHP does (without cygwin)

      and :

      - Vim is as verbose and user-friendly as an old A4 scansheet of sandy newspaper used for toilet-paper replacement.

      - PHP is as verbose and user-friendly as VIM, but still more user-friendly than perl and its s/wtf/omg/g hideous syntax.

      so :

      - Your comment assumes everyone is using gnu linux, and of course Vim as the preferred text editor

      - Reality is there are MacOS, Windows, and of course tons of editors (including EMACS) that can perform that job too.

      hint :

      do not fear php

    • FYI, entry updated.


    • Tobias Struckmeier

      2007 Feb 22 11:06

      Yes VIM is simple.

      There is GVIM for windows. Even in a "simple" mode for people that are not yet used to the VIM commands.

      What verbosity do you need? Vim is able to mark the replacements, ask for every single and so on...

      And on windows I also prefer VIM (GVIM).

      MacOS I think textmate does the job (Compare it with vim and you find many functions that are very closely)

      You should try using vim for a while. Then youll be able to harnest its powers.

    • anonymous

      2009 Apr 01 20:06

      It is impossible to do substitutions using (G)VIM on a file, say 1GB big....and thus perl -p-i-e comes to the rescue :)