Author: sjan

PHP

PHP File IO concurrency issues with Windows

A project that I am currently dealing with at work involves writing a large number of files to disk on Windows via PHP, and then pulling them into a database via a LOAD DATA INFILE call. The problem I am running into is that the file writes are backgrounded by the system while the PHP script keeps trucking along, pushing the PHP system resources through the roof while paging most of the running processes, further slowing down the disk IO. Finally, once the scripts have have started and backgrounded all these file writes, the next script can’t read in the files to load them into the database.

So far I haven’t found any hints on this anywhere yet. If any of my readers know of any file IO tricks for PHP on Windows please let me know! (I’m talking to both of you!)

Edit: It seems the issue was caused by having XDebug running while trying to write the files. I thought I had turned it off, but I hadn’t so it was writing cachegrind files while the scripts were trying to write their files, and well, it wasn’t pretty.

Home

New Year, time to (finally) catch up (?)

As we start the new year I find that I am sadly fallen behind on things like this blog, social networking, reading and news aggregation, and generally anything unrelated to work or the daily functioning of the household. In other words, I have been working too much and relaxing too little. Well, that will have to change. Of course, that means I have to get back into a routine, which seems to be difficult when making major changes, such as, oh, starting a new job in a new town.

OS X

Leopard – First Run

After waiting all this time for a multi-desktop environment I was excited to get and install my copy of 10.5 today. Even though it isn’t ready for some because of program incompatibilities (sorry, Ted) I hadn’t run across anything to make me think it wasn’t ready for my use. And in truth, there isn’t anything ‘wrong’ with Leopard. It just turns out that I hadn’t gotten the word that I would have to upgrade Parallels to build 5160 in order to use it on Leopard. And since I primarily want the multiple desktops (“Spaces”) to make it easier to run VMs that is in an issue. So, I am downloading the update now. Since it isn’t going to cost me anything to upgrade Parallels, that’s ok. For me, running a VM in full-screen mode while still having access to my Mac desktop is important. While I am at the desk I can do that by hooking up another monitor. When I’m not at the desk, though, that isn’t possible. What I sincerely hope is that I can run my guest OS fullscreen on one desktop while coding on another and browsing on another and wasting time (chatting and playing mindless puzzle games) on the fourth. This was my MO when I was working with VMWare on a Linux machine with XFCE. So, now I plan on doing the same on my Mac. After all, when was the last time a window manager for BSD didn’t have multiple desktops? (Ok, I know, OS X is more than a WM, but you get the drift.)

Technorati Tags: OS X, Leopard, Parallels

Subversion

WordPress Update headaches – resolved???

I had a few headaches updating to WordPress 2.3. Not surprising since the category related tables were removed to make room for native tagging, and I have been relying on the Category Tagging plug-in. Ah well, after some hand sql work I was able to populate the tags to match the categories.

Now it can only get better, right? I have moved to using subversion to pull the latest stable release, so that takes a big chunk of time out of the equation, and now my tagging will happen at the time of posting, rather than being a pseudo-tag solution pulled from post categories. Or will it? I wonder if the tags tab in ScribeFire will do the trick?

We shall see.

The 3.0 version of the Google Sitemap generator seems to not be working, however, but I may have b0rked something there myself, and I will try again later.

Oh, and those sql statements?

If I remember correctly (I didn’t keep notes like I should have):

INSERT INTO wp_term_taxonomy (term_id, taxonomy, count) SELECT term_id, ‘post_tag’, count FROM wp_term_taxonomy WHERE taxonomy=’category’;

INSERT INTO wp_term_relationships (object_id, term_taxonomy_id) SELECT rel.object_id, tax.term_taxonomy_id FROM wp_term_relationships AS rel, wp_term_taxonomy AS tax, wp_term_taxonomy AS ttx WHERE rel.term_taxonomy_id=ttx.term_taxonomy_id AND ttx.term_id = tax.term_id AND tax.taxonomy=’post_tag’;

I could have brought the descriptions across from the categories, too, but I didn’t think about it until just now.

Edit: I found the issue with the site-map generator. It required a tweak to the memory limits in my php.ini :/

Alaska

Back up and running

We have moved into our new home (sort of – most everything is still in boxes) but we are back up online and rockin’ again. We are now located in Fairbanks, just 105 miles south of the Arctic Circle. And I’ll tell you what – this is a hip little town with a happening music and arts scene and a fantastic University. You should come visit some time, you won’t be disappointed. In the summer there are all sorts of things happening for the tourists, and in the winter (and late autumn and early spring) the Aurora just kicks!

Apache

Gentoo Apache 2.2 update and 403 errors

After upgrading my entire system, moving from Apache 2.0.x to 2.2.6 I ran into an interesting problem (actually, a couple.) Some of the configurations have moved, and things that used to be in the Gentoo dist httpd.conf have been broken out into config files in the /etc/apache2/modules.d/ directory. So, copying my old vhosts file in was not a good idea. And doing a merge of the old and new httpd.conf files was also a mistake. Once I figured out that I had the Listen 80 directive in one file and Listen 192.168.1.10:80 in another I understood why it failed to start. So, I fixed all the configs to match the new setup, and tried again. It started up just fine, and seemed okay, until I tried to connect. I kept getting 403 errors.

I went through the standard checks, checking the file permissions, .htaccess settings, and so on, to no avail. A quick Google search pointed me to the fix from Victor Trac. He found the offending bit in the new broken out config in /etc/apache2/modules.d/00_default_settings.conf where it contains:

<Directory />

        Options FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
</Directory>

The fix is either to change that to Allow from all and define Deny where needed in each virtual host or to override it in every virtual host. Since I tend to set up my hosts with the idea that the server allows everything and it is up to the host to deny where needed I chose the first option, reloaded Apache and everything is sweet again.

Edit:

I found I was having an error with the RewriteRules after switching from Apache 2.0.x to 2.2.x – I found the fix on the Gentoo forums, which required adding an extra RewriteCond line in the .htaccess file.

The old .htaccess: and the new:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /blog
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L]
</IfModule>
# END WordPress

and the new:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /blog
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond $1 !\.php$
RewriteRule . /blog/index.php [L]
</IfModule>
# END WordPress

Technorati Tags: Apache 2.2, 403, Gentoo

Gentoo

Getting caught up

While things have been hectic preparing for the big move, some things have fallen by wayside. Like updating the server (ok, I can’t just blame that on moving, it hasn’t been done in a while) and updating the MySpace crossposter from Roderick Russell to version 2.0. That part, at least, was a piece of cake, and the options panel makes setup much easier.

It turns out the server was still running on the Gentoo 2006.0 profile and in order to bring things up to date I needed to get it to the 2007.0 profile. For anyone who hasn’t tried this, here is the easy way: run emerge –sync and then change the symlink /etc/make.profile to point to the newer profile (in this case /usr/portage/profiles/default-linux/x86/2007.0 – after that run emerge -u portage and then revdep-rebuild (in order to use revdep-rebuild you need to have the gentoolkit installed, which can be done via emerge gentoolkit. (It takes about 5 minutes to download and build.)

After running revdep-rebuild it was time to run the dreaded emerge -auv world. Running verbose shows clearly what is getting replaced, updated, added, etc and you get the nice red blocking messages.

In my case, pecl-zip was blocking php-5.2.4, and no amount of forcing was going to help, so for the duration of the upgrade I unmerged pecl-zip. I also had some blocking in that the current version of Postfix was not compatible with the new version, so the choice is to unmerge Postfix and then emerge (too long of a down-time) or by doing a FORCE_UPGRADE=1 emerge –buildpkgonly postfix followed by /etc/init.d/postfix stop and then emerge –usepkgonly postfix followed by a quick etc-update to merge the config file with the new configs and then restart the server with /etc/init.d/postfix start. Total downtime for the mail server: 2 minutes.

I am currently updating the last of the software (mostly admin and gnu tools) and then will build a new kernel. Of course, since there is a newer version of GCC included in the updates, everything that wasn’t updated will get rebuilt with another call to revdep-rebuild before I configure and compile the new kernel.

It’s not like any of this is difficult, especially when most of the ebuilds are so good about giving you help in their error messages (if there are errors) and often tell you in the message how to fix it step-by-step. Now, if I could just find the build that reminds me every week or so to run emerge –sync and check for updates. Hmmm – I might even put that in a weekly or daily cron job:

#!/bin/bash
#update notifier
usr/bin/emerge --sync > /dev/null 2>&1 || true --nospinner && /usr/bin/emerge -puv world

Then it is just a matter of seeing what shows up as needing to be updated, and I can compare that with messages from the Gentoo Linux Security Advisories to determine which can wait and which are needed right now.

Technorati Tags: Gentoo

Apache

Apache 2.2.6, PHP 5.2.4 and MySQL 5.0.45 on OS X

I got tired of looking for a way to replace the Apache/PHP that Apple packages with OS X (without breaking anything else in the process) so I decided to install Apache 2.2 and PHP5 in their own location to avoid stepping on the Apple package toes.

Since I do a great deal of development again MySQL I needed to install that as well, and figured that I would probably need the GD functionality as well so I grabbed libjpeg and libpng to make those work as well. This is the step-by-step.

(Props to James Pelow and his article from last year, from which I borrowed the configure command lines and configuration modifications, as well as the idea of installing the whole mess in /apache2.)

Download the latest MySQL (I used the package version) from MySQL.

Installation is straightforward following the same methods as any other Mac installer.

Download and install libjpeg and libpng – from Ethan Tira-Thompson (this is also in a Mac installer which contains both libraries in one installer).

Download the latest Apache httpd server (Unix source) from Apache

in the terminal:

tar -xzvf httpd-2.2.6.tar.gz && cd httpd-2.2.6

./configure

--prefix=/apache2

--enable-module=most

--enable-shared=max

make

sudo make install

sudo mkdir /apache2/php

Download the latest PHP from PHP

tar -xzvf php-5.2.4.tar.gz && cd php-5.2.4

./configure

--prefix=/apache2/php

--with-zlib

--with-xml

--with-ldap=/usr

--enable-cli

--with-zlib-dir=/usr

--enable-exif

--enable-ftp

--enable-mbstring

--enable-mbregex

--enable-dbx

--enable-sockets

--with-iodbc=/usr

--with-curl=/usr

--with-mysql=/usr/local/mysql

--with-gd

--with-jpeg-dir=/usr/local

--with-png-dir=/usr/local

--with-apxs2=/apache2/bin/apxsmake

sudo make install

sudo cp php.ini-dist /apache2/php/lib/php.ini

Now to make your Apache2.2 a little more ‘Mac’ – you can point it at the Mac web shared files folder, change the user and group and change the location for user files to match the Mac folder system.

Edit httpd.conf (I use nano, you can use any flat text editor like nano, pico, vi, emacs or even BBedit)

sudo nano -w /apache2/conf/httpd.conf

The changes to httpd.conf I made:
I changed

User daemon

Group daemon

User www

Group www

and

DocumentRoot "/apache2/htdocs"

DocumentRoot "/Library/WebServer/Documents"

and

<Directory "/apache2/htdocs">

<Directory "/Library/WebServer/Documents">

and added

AddType application/x-httpd-php .php

AddType application/x-httpd-php-source .phps

DirectoryIndex index.html index.php

Edit httpd-userdir.conf

sudo nano -w /apache2/conf/extra/httpd-userdir.conf

The changes to httpd-userdir.conf I made:
I changed

UserDir public_html

UserDir Sites

To start and stop the server:
MySQL comes with a Preference Pane that allows you to start and stop it there. To start and stop Apache you need to first make sure that the default Apache shipped with OS X is stopped.

sudo /apache2/bin/apachectl start

sudo /apache2/bin/apachectl stop

I only ran into one issue, when trying to start the server I ran against the following error message (and no running server, of course):

httpd: Syntax error on line 53 of /apache2/conf/httpd.conf:

Cannot load /apache2/modules/libphp5.so into server:

Library not loaded: /usr/local/mysql/lib/mysql/libmysqlclient.15.dylib

Referenced from: /apache2/modules/libphp5.son  Reason: image not found

To fix this I did the following:

cd /usr/local/mysql/lib

sudo mkdir /usr/local/mysql/lib/mysql

for i in `ls ./l*`; do sudo ln -sf /usr/local/mysql/lib/$i /usr/local/mysql/lib/mysql/$i; done

This creates soft links in the directory that libphp5.so is looking for the MySQL libraries.

Then it started right up! Wheee! (I did a quick test by dropping PhpMyAdmin into the /Library/WebServer/Documents folder and browsed to it – the whole Apache/PHP/MySQL is working correctly)

Technorati Tags: Apache 2.2, PHP5, MySQL 5, OS X

Alaska

Making moves

I didn’t want to post anything about this until I had given notice at work, so here it is. I was contacted by a head-hunter about a position in Fairbanks. I hadn’t planned on changing positions at this time, but the opportunity was too good to pass up. The position is as a PHP developer with a company in Fairbanks, Alaska. They would like me to start on the 17th of this month, and I am trying to get everything together to make that happen.

The house is currently in underwriting and should close soon, and we are getting everything packed and ready to roll. I will definitely miss working with the crew I am leaving behind, but I am looking forward to learning lots of new (to me) things, including interfacing with touch screens and some serious AJAX.

The downside of all this is that both this site and talkingfox.com will be down for anywhere from one day to a week (or if we have difficulty getting an apartment lined up possibly longer – although I don’t want to think about that right now.)

Hardware

Sun opens UltraSPARC T2

Sun announced their entrance into the commodity silicon business with the UltraSPARC T2 (or Niagara 2), which is their fastest processor to date, delivering 89.6 Ghz of parallel computing on a single chip. They have signed their first OEM agreement with Marvell, and more are sure to follow.

In a surprising move, Sun will be releasing the blueprints and core design files and test suites for the T2 under the GPL.

Technorati Tags: Sun, UltraSPARC T2, Hardware

evardsson.com: stuff that w0rks