Category: PHP

PHP: Hypertext Preprocessor

Development

SPDO Wiki and Updates

I updated SPDO some today – I added in MySQL support for the Python version, and changed the names of the objects and files to match the new moniker. Speaking of which, I have decided on “Simplified (Python|PHP) Database Object” – so yeah, SPDO is the official name now. (Of course I keep seeing it “speedo”).

I also started a Wiki which will be updated with documentation, samples, etc as I make the time. I am hoping to get a bunch done tomorrow afternoon and this weekend.

The version 0.9 files are spdo-py.tar.bz2 and spdo-php.tar.bz2

Development

New Project: SPDO

I have just posted the (embarrassingly empty) page for my new pet project: SPDO (Sjan’s PDO) – in two flavors: PHP and Python.

There are only about a thousand PDOs out there, and perhaps a dozen or so of them are really functional and (in a few cases) very polished pieces of work. So why am I messing with writing my own? A couple of reasons:

  1. I like to have coding signatures that are consistent, whether I am working with PostgreSQL, MySQL or (heaven help us all) SQLite.
  2. I like to have coding signatures that are (reasonably) consistent across different languages – in this case Python and PHP.
  3. I wanted to take advantage of Prepared Statements where I could, even though the PHP implementations of those are pretty weak (especially in the case of MySQL).

Currently implemented in

  • Python:
    • PostgreSQL (with prepared statements)
    • SQLite (no prepared statements).
  • PHP
    • PostgreSQL (with prepared statements)
    • MySQL (with prepared statements)
    • SQLite (no prepared statements)

Here’s an example of how they work (in most simplistic terms):

#in Python
from pyDB import *
db = pyDB('mysite')
newid = db.insert('INSERT INTO test (name, value) VALUES (?,?)',['foo','bar'])
update_count = db.update('UPDATE test SET value=? WHERE id=?',['baz',newid])
results = db.select('SELECT * FROM test')
for row in results:
    for i in row:
        print "\t", i,"\t", row[i]
delete_count = db.delete('DELETE FROM test WHERE id=?',[newid])
//in PHP
require_once('phpdb.php');
$db = new phpDB('test');
$newid = $db->insert('INSERT INTO test (name, value) VALUES (?,?)',array('foo','bar'));
$update_count = $db->update('UPDATE test SET value=? WHERE id=?',array('baz',newid));
$results = db->select('SELECT * FROM test');
foreach($results as $row)
{
    foreach ($row as $key=>$val)
    {
        print "\t$key\t$val";
    }
}
$delete_count = $db->delete('DELETE FROM test WHERE id=?',array($newid));

The page with links to the code is in the list up top, and everything is MIT license. Enjoy.

Development

Learning things the hard way

These are some things I learned the hard way over the past few weeks. Ignore at your peril!

  • When your setup requires a different database layout (meaning differently named and/or organized databases and/or servers) for development than for live deployment, you would do well to try to mimick the live layout in your development environment before beginning. (This one hurt, bad.) In fact, when you find a need for your application to test for current environment (development vs production), you may want to be extra certain that you really need to set up your development structure that differently.
  • When your test & deployment environment calls for a server external to the existing systems and database servers, make sure that you:
    1. Have a clear understanding of what packages/versions that server will need.
    2. Have a clear understanding of how long it will take to get the hardware in place.
    3. Have a working copy in a VM on the same OS that the server will be deployed on before you say ok to anything.
    4. Ask, beg, plead, whatever, for a copy of that server (even on desktop hardware) to be set up for the testing. If you cannot test against real machines in a real environment for any kind of multi-server deployment, all bets are off. (Um, yeah, again, ouch.)
  • While the occasional 12+ hour day for developers is inevitable, too many of them in a row will result in more mistakes and coding errors of the most simplistic sort.
  • When you discover that you need to add functionality to an integral part of the software (like, say, the primary db connection class) do everything within your power to ensure that it still “works the same” – that is, add any modifications in such a way that they do not break existing code. This is not always easy to tell, but usually a simple trip around the application will tell you quickly if you have broken something as basic as this. (No, I didn’t break it, at least not anywhere other than locally. I actually took this step, many times and saved myself great embarrassment.)
  • Finally, when switching between languages (PHP, Java, Bash, Javascript, etc etc) you may find yourself looking at code that should work, but doesn’t. Chaining calls in PHP seems to be iffy at best. For example:
    • This does not work, although it seems it should. The $db is an instance of a legacy database connection class used for the project, do_query returns an array of results, and do_query with a second argument sets the value of the field named as the key for each row:list($account) = array_keys($db->do_query(“SELECT account_id FROM reports WHERE store_db=’$store’ LIMIT 1”, ‘account_id’));
    • while this does, using the same class and method:$data = $db->do_query(“SELECT account_id FROM reports WHERE store_db=’$store’ LIMIT 1”, ‘account_id’);
      list($account) = array_keys($data);
      // this also works:
      // $account = $data[0][‘account_id’];
    • Really, sometimes an extra line is worth not having the headache.
PHP

MVCFDH: One way to do MVC in PHP

Aaron Saray, over at 102Degrees, put up an interesting article today on MVCFDH, what he calls his way of building MVC architecture in PHP.

The short version is Model-View-Controller (which we are all familiar with) and adding the ‘Front Controller,’ ‘Dataobject’ and ‘Helper’ pieces. I won’t go into details here about what all this means, you can read it over there. The interesting thing is, without really having given it that kind of thought, that is pretty much the way I have always pieced together any kind of MVC work I have done, regardless of the language.

I think there might be a minor difference, though, at least in what we call things. I tend to think of them as ‘Dispatcher’ (don’t jump my case for calling it that, it works in my head!) rather than ‘Front Controller’, ‘Libs’ rather than ‘Helpers’ and ‘Dataobjects?’ I call them ‘Data Objects.’ :P

PHP

Using the COM class for PHP backgrounding in Windows

I was having a difficult time finding a reliable way to run a background PHP process in Windows, when that was called from an active PHP page. In the *nix world it is relatively simple: by using shell_exec() (or the bactick operator) you can redirect the output to another stream or file and the process will run in the background with no blocking. In Windows, however, this doesn’t seem to work well (or at all, depending on what you are calling via shell_exec()). I did find the answer, though, from piecing together info from the PHP documentation for shell_exec() and the COM class.

That, with a little trial and error and I was able to get a PHP page to fire off a command-line PHP process to run an import of several years data into a new reporting schema. Since this import relies on some serious data manipulation it has a tendency to time out for large data sets. So, I set up the command line script to run six months worth of data and before it exits it starts a new background process for the next six months of data. In this way I was able to complete a many-hour process without worrying about timeouts. I did notice that running in the background (actually in an “invisible” command shell) that the process ran slower than when running in the foreground. This was acceptable, however, since the page returns immediately while the processing begins and the application is still usable while the process is running.

Here is how I call it from the page:

if(isset($_SERVER['PWD'])) { // *nix
    $basepath = dirname(__FILE__).'/';
    $php = 'php';
} else {
    $basepath = dirname(__FILE__).'\\';
    // edit to match your installed target environment
    $php = "C:\\php516\\php.exe";
}
ignore_user_abort(true);
set_time_limit(0);
$arg1 = 'foo';
$arg2 = 'bar';
$runCommand = "$php -q {$basepath}my_background_running.php $arg1 $arg2";
if(isset($_SERVER['PWD'])) { // *nix
    // *nix: Use the backtick operator or shell_exec()
    $nullResult = `$runCommand > /dev/null &`;
} else { // Windows: use the php COM class
    // WScript.Shell gives you the command line
    $WshShell = new COM("WScript.Shell");
    $oExec = $WshShell->Run($runCommand, 7, false);
}

I do the same from the background script to call itself recursively right before it exits.

I didn’t find the Microsoft documentation for the Windows Script Host Shell until today during lunch. I found the location in the Practical PHP Programming Online Book where he says to use the Google search “wshell object” msdn which will result in the first link pointing at the MSDN documentation. (I have a feeling it moves around quite a bit, since every link I have run across up to now that points directly at the documentation results in a nice 404 error page at Microsoft.)

Development

Problems arising from PHP type casting in ==

While trying to work through the issues I mentioned in the last post I started doing some serious digging and testing. Here is what I have found.

PHP seems to use == for determining equivalance when performing array_search, in_array and switch, while using either === or strcmp when doing array_key_exists.

The result of this is that array_search and in_array will return improper results when used on an array with mixed string and integer values. (Another thing I found, that may or may not be related, is that array keys will be cast from strings to integers when those strings are valid integer values.)

array_search() with mixed types

$one = array (
  'abc',
  'abc1',
  111,
  '111b',
  2,
  '2xyz',
  '123a',
  123
);
$two = $one;
for ($i = 0; $i < count($one); $i++) {
  $xkey = array_search($one[$i], $two);
  if(strcmp(strval($one[$i]), strval($two[$xkey])) != 0) {
    // This should NEVER be reached, but it is, often!
    $eq = 'FALSE';
  } else {
    $eq = 'true';
  }
}
Row $one $two Correct? Found Notes
0 abc abc true 0 abc == abc : array_search($one[0], $two) where $one[0] = string(3) “abc”
1 abc1 abc1 true 1 abc1 == abc1 : array_search($one[1], $two) where $one[1] = string(4) “abc1”
2 111 111 true 2 111 == 111 : array_search($one[2], $two) where $one[2] = int(111)
3 111b 111b FALSE 2 111b == 111 : array_search($one[3], $two) where $one[3] = string(4) “111b”
4 2 2 true 4 2 == 2 : array_search($one[4], $two) where $one[4] = int(2)
5 2xyz 2xyz FALSE 4 2xyz == 2 : array_search($one[5], $two) where $one[5] = string(4) “2xyz”
6 123a 123a true 6 123a == 123a : array_search($one[6], $two) where $one[6] = string(4) “123a”
7 123 123 FALSE 6 123 == 123a : array_search($one[7], $two) where $one[7] = int(123)

array_search() with all strings

$one = array (
  'abc',
  'abc1',
  '111',
  '111b',
  '2',
  '2xyz',
  '123a',
  '123'
);
$two = $one;
for ($i = 0; $i < count($one); $i++) {
  $xkey = array_search($one[$i], $two);
  if(strcmp(strval($one[$i]), strval($two[$xkey])) != 0) {
    // This should NEVER be reached, and with all strings it isn't.
    $eq = 'FALSE';
  } else {
    $eq = 'true';
  }
}
Row $one $two Correct? Found Notes
0 abc abc true 0 abc == abc : array_search($one[0], $two) where $one[0] = string(3) “abc”
1 abc1 abc1 true 1 abc1 == abc1 : array_search($one[1], $two) where $one[1] = string(4) “abc1”
2 111 111 true 2 111 == 111 : array_search($one[2], $two) where $one[2] = string(3) “111”
3 111b 111b true 3 111b == 111b : array_search($one[3], $two) where $one[3] = string(4) “111b”
4 2 2 true 4 2 == 2 : array_search($one[4], $two) where $one[4] = string(1) “2”
5 2xyz 2xyz true 5 2xyz == 2xyz : array_search($one[5], $two) where $one[5] = string(4) “2xyz”
6 123a 123a true 6 123a == 123a : array_search($one[6], $two) where $one[6] = string(4) “123a”
7 123 123 true 7 123 == 123 : array_search($one[7], $two) where $one[7] = string(3) “123”

in_array() and array_key_exists()

$array = array('111'=>'111', '11b'=>'11b', '222b'=>'222b','2x22'=>'2x22');
$keys = array_keys($array);
$searches = array('111b',222,11,'222b',2);
foreach ($searches as $search) {
  $ia = (in_array($search, $array))?'true':'false';
  $ake = (array_key_exists($search, $array))?'true':'false';
  if ($search === '222b') {
    // This is the only place where either should return true
    $iaf = ($ia == 'true')?" class=\"$true\"":" class=\"$false\"";
    $akef = ($ake == 'true')?" class=\"$true\"":" class=\"$false\"";
    $notes = "** Both should be true **";
  } else {
    $iaf = ($ia == 'false')?" class=\"$true\"":" class=\"$false\"";
    $akef = ($ake == 'false')?" class=\"$true\"":" class=\"$false\"";
    $notes = "Both should be false";
  }
}

Notice how the array keys are cast to type int in both the original array and in array_keys.

$array $keys
array(4) {
  [111]=>
  string(3) "111"
  ["11b"]=>
  string(3) "11b"
  ["222b"]=>
  string(4) "222b"
  ["2x22"]=>
  string(4) "2x22"
}
array(4) {
  [0]=>
  int(111)
  [1]=>
  string(3) "11b"
  [2]=>
  string(4) "222b"
  [3]=>
  string(4) "2x22"
}
Search Item in_array array_key_exists Notes
111b false false Both should be false
222 true false Both should be false
11 true false Both should be false
222b true true ** Both should be true **
2 true false Both should be false

So, it appears that array_key_exists() uses either or === strcmp() while in_array() uses ==

NOTE: Calling array_key_exists() with a string value ‘111’ will return true for an item with a key of int 111. This is not the same behavior as ===, but is the same behavior as strcmp() which must be what is used internally for array_key_exists().

The difference between the 3 operations is clear:

$a $b $a == $b $a === $b strcmp(strval($a),strval($b))
int(123) string(4) “123b” bool(true) bool(false) int(-1)

Another area where this becomes an issue is in switch statements. Take the following, for example:

switch()

$array = array(111,'222b');
foreach($array as $val)
{
  $row = ($row == 'row')?'offset_row':'row';
  $false = ($false == 'false')?'offset_false':'false';
  $true = ($true == 'true')?'offset_true':'true';
  switch($val)
  {
    case '111b': // this displays
      $match = '111b';
      $f = " class=\"$false\"";
      $notes = "Incorrect: should have fallen through to next case";
      break;
    case 111: // never makes it here even tho this is correct
      $match = 111;
      $f = " class=\"$true\"";
      $notes = "** Correct **";
      break;
    case 222: // this displays
      $match = 222;
      $f = " class=\"$false\"";
      $notes = "Incorrect: should have fallen through to next case";
      break;
    case '222b': // never makes it here even tho this is correct
      $match = '222b';
      $f = " class=\"$true\"";
      $notes = "** Correct **";
      break;
    default:
      $match = 'no match';
      $f = " class=\"$false\"";
      $notes = "Incorrect: should have matched";
      break;
  }
}
Search Item Match Notes
111 111b Incorrect: should have fallen through to next case
222b 222 Incorrect: should have fallen through to next case
Development

PHP array_search implicit cast of search term

There is an error in the values that array_search returns when searching on an array that has a mix of numeric values (123) and alpha-numeric mixed strings that start with the same and follow with alpha characters (‘123a’).

The results are actually kind of bizarre, but explainable by a bug in PHP’s equivalence test. When testing for equivalence (using ==) PHP determines that 123 == ‘123xyz’. PHP casts the string to an integer when doing the comparison (so ‘123xyz’ becomes 123). This is documented in bugs.php.net (http://bugs.php.net/bug.php?id=23110) – but this leads to problems: both switch and array_search use == for comparison.

So, using:

$one = array (
  'abc',
  'abc1',
  111,
  '111b',
  2,
  '2xyz',
  '123a',
  123
);
$two = $one;

foreach($one as $val)
{
  $key = array_search($val, $two);
  if ($key !== false) {
    echo "$val == {$two[$key]} \n";
    if (strcmp(strval($val), strval($two[$key])) == 0) {
      echo "strcmp returns true";
    } else {
      echo "strcmp returns false";
    }
  } else {
    echo "$val not found \n";
  }
}

results in:

abc == abc -- strcmp returns true
abc1 == abc1 -- strcmp returns true
111 == 111 -- strcmp returns true
111b == 111 -- strcmp returns false
2 == 2 -- strcmp returns true
2xyz == 2 -- strcmp returns false
123a == 123a -- strcmp returns true
123 == 123a -- strcmp returns false

This becomes a real problem when you can’t be sure that the values in an array are all of the same type. However, if you are sure that all the values in the array are of type string then array_search works flawlessly.

I am still unsure how to work around this, however, I think having a version of array_search that doesn’t do an implicit cast on the search value would be of great use.

PHP

PHP File IO concurrency issues with Windows

A project that I am currently dealing with at work involves writing a large number of files to disk on Windows via PHP, and then pulling them into a database via a LOAD DATA INFILE call. The problem I am running into is that the file writes are backgrounded by the system while the PHP script keeps trucking along, pushing the PHP system resources through the roof while paging most of the running processes, further slowing down the disk IO. Finally, once the scripts have have started and backgrounded all these file writes, the next script can’t read in the files to load them into the database.

So far I haven’t found any hints on this anywhere yet. If any of my readers know of any file IO tricks for PHP on Windows please let me know! (I’m talking to both of you!)

Edit: It seems the issue was caused by having XDebug running while trying to write the files. I thought I had turned it off, but I hadn’t so it was writing cachegrind files while the scripts were trying to write their files, and well, it wasn’t pretty.

Read More

Apache

Apache 2.2.6, PHP 5.2.4 and MySQL 5.0.45 on OS X

I got tired of looking for a way to replace the Apache/PHP that Apple packages with OS X (without breaking anything else in the process) so I decided to install Apache 2.2 and PHP5 in their own location to avoid stepping on the Apple package toes.

Since I do a great deal of development again MySQL I needed to install that as well, and figured that I would probably need the GD functionality as well so I grabbed libjpeg and libpng to make those work as well. This is the step-by-step.

(Props to James Pelow and his article from last year, from which I borrowed the configure command lines and configuration modifications, as well as the idea of installing the whole mess in /apache2.)

Download the latest MySQL (I used the package version) from MySQL.

Installation is straightforward following the same methods as any other Mac installer.

Download and install libjpeg and libpng – from Ethan Tira-Thompson (this is also in a Mac installer which contains both libraries in one installer).

Download the latest Apache httpd server (Unix source) from Apache

in the terminal:

tar -xzvf httpd-2.2.6.tar.gz && cd httpd-2.2.6

./configure

--prefix=/apache2

--enable-module=most

--enable-shared=max

make

sudo make install

sudo mkdir /apache2/php

Download the latest PHP from PHP

tar -xzvf php-5.2.4.tar.gz && cd php-5.2.4

./configure

--prefix=/apache2/php

--with-zlib

--with-xml

--with-ldap=/usr

--enable-cli

--with-zlib-dir=/usr

--enable-exif

--enable-ftp

--enable-mbstring

--enable-mbregex

--enable-dbx

--enable-sockets

--with-iodbc=/usr

--with-curl=/usr

--with-mysql=/usr/local/mysql

--with-gd

--with-jpeg-dir=/usr/local

--with-png-dir=/usr/local
--with-apxs2=/apache2/bin/apxsmake

sudo make install

sudo cp php.ini-dist /apache2/php/lib/php.ini

Now to make your Apache2.2 a little more ‘Mac’ – you can point it at the Mac web shared files folder, change the user and group and change the location for user files to match the Mac folder system.

Edit httpd.conf (I use nano, you can use any flat text editor like nano, pico, vi, emacs or even BBedit)

sudo nano -w /apache2/conf/httpd.conf

The changes to httpd.conf I made:
I changed

User daemon
Group daemon

to

User www
Group www

and

DocumentRoot "/apache2/htdocs"

to

DocumentRoot "/Library/WebServer/Documents"

and

<Directory "/apache2/htdocs">

to

<Directory "/Library/WebServer/Documents">

and added

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

DirectoryIndex index.html index.php

Edit httpd-userdir.conf

sudo nano -w /apache2/conf/extra/httpd-userdir.conf

The changes to httpd-userdir.conf I made:
I changed

UserDir public_html

to

UserDir Sites

To start and stop the server:
MySQL comes with a Preference Pane that allows you to start and stop it there. To start and stop Apache you need to first make sure that the default Apache shipped with OS X is stopped.

sudo /apache2/bin/apachectl start
sudo /apache2/bin/apachectl stop

I only ran into one issue, when trying to start the server I ran against the following error message (and no running server, of course):

httpd: Syntax error on line 53 of /apache2/conf/httpd.conf:
Cannot load /apache2/modules/libphp5.so into server:
Library not loaded: /usr/local/mysql/lib/mysql/libmysqlclient.15.dylib
Referenced from: /apache2/modules/libphp5.son  Reason: image not found

To fix this I did the following:

cd /usr/local/mysql/lib
sudo mkdir /usr/local/mysql/lib/mysql

for i in `ls ./l*`; do sudo ln -sf /usr/local/mysql/lib/$i /usr/local/mysql/lib/mysql/$i; done

This creates soft links in the directory that libphp5.so is looking for the MySQL libraries.

Then it started right up! Wheee! (I did a quick test by dropping PhpMyAdmin into the /Library/WebServer/Documents folder and browsed to it – the whole Apache/PHP/MySQL is working correctly)

Technorati Tags: , , ,

Read More

Apache

Useful custom 403 and 404 error pages with PHP

While this is certainly nothing new, it seems to be too often overlooked. Apache allows an ErrorDocument Directive in the configuration that will point at a custom document. Using this can have some benefits to the user and to the site administrator.

While Apache allows for error documents located at a remote URL (ie anything starting with http://) this causes Apache to send a redirect to the browser, even if the document resides on the same server. This is not a good idea, as the documentation points out.

This has several implications, the most important being that the client will not receive the original error status code, but instead will receive a redirect status code. This in turn can confuse web robots and other clients which try to determine if a URL is valid using the status code. In addition, if you use a remote URL in an ErrorDocument 401, the client will not know to prompt the user for a password since it will not receive the 401 status code. Therefore, if you use an ErrorDocument 401 directive then it must refer to a local document.

Using a local document for handling errors, however, gives you the ability to override the default Apache messages, which are often replaced by the browser with their own, internal error messages (MSIE, I’m talking about you.) Besides giving you the ability to match the error page to your site, you can use some simple PHP to make it more informative for both the end user and the site admin. Instead of just saying “File so-and-so doesn’t exist, sorry” you can make a page that allows the user to send a message to the admin. If you wish, you can have the page automatically mail the information, although that can quickly lead to hundreds of emails as users mis-type urls, spiders follow old links, and scripts search your LAMP site for IIS vulnerabilities. Trust me on that one, it’s a bad idea that won’t outlive the weekend.

With that in mind here a couple samples that you can build from.

Sample 403 error page:

<?php
print "<html>
<head>
<title>Sample 403 Error Document</title>
</head>
<body>"
$server = $_SERVER['SERVER_NAME'];
$uri = $_SERVER['REQUEST_URI'];
$bad_link = $server.$uri;
// Note that the referer cannot be completely trusted
// as some agents either do not set a referer or allow
// the user to modify the referer at will. It is, however,
// often useful for troubleshooting.
$referer = $_SERVER['HTTP_REFERER'];
$remote = $_SERVER['REMOTE_ADDR'];
print "<h1>403: Forbidden</h1>
<p> </p>";
if ($uri == '/403/403.php') { 
	print "<p>>You have reached the custom 403 error page for mysite.com. Was it everything you were hoping for?</p>";
}
else if (substr($uri, -1, 1) == '/') {
    print "<p>Sorry, this directory cannot be browsed.</p>
    <p>If you received this message by clicking on a link on this website, please <a href=\"mailto:webmaster@mysite.com?subject=403: Bad Directory Link&body=$bad_link from $referer\">report it to the webmaster</a>.</p>";
}
else {
    print "<p>You have attempted to access a resource ($uri) for which you do not have the proper authorization or which is not available from your location.</p>
    <p>If you received this message by clicking on a link on this website, please <a href=\"mailto:webmaster@mysite.com?subject=403 Error&body=$bad_link from $referer reached by $remote\">report it to the webmaster</a>.</p>";
}
print "</body>
</html>
";
?>

Sample 404 error page:

<?php
print "<html>
<head>
<title>Sample 403 Error Document</title>
</head>
<body>"
$server = $_SERVER['SERVER_NAME'];
$uri = $_SERVER['REQUEST_URI'];
$bad_link = $server.$uri;
// Note that the referer cannot be completely trusted
// as some agents either do not set a referer or allow
// the user to modify the referer at will. It is, however,
// often useful for troubleshooting.
$referer = $_SERVER['HTTP_REFERER'];
print "<h1>404: File Not Found</h1>
<p> </p>";
if ($uri == '/404/404.php') {
    print "<p>You have reached the custom 404 error page for mysite.com. Was it everything you were hoping for?</p>";
}
else {
    print "<p>Sorry, that file ($uri) does not seem to exist.</p>
    <p>If you received this message by clicking on a link on this website, please <a href=\"mailto:webmaster@mysite.com?subject=Bad Link&body=$bad_link from $referer\">report it to the webmaster</a>.</p>";
}
print "</body>
</html>
";
?>

Of course you would make sure the styles, links, mailtos, site name, etc are right for your site, but this gives you an idea.

Technorati Tags: , ,