Archive for July, 2012

Symfony2 + Propel 1.6 with Memcache

July 4th, 2012 by Sjan Evardsson

I was looking for a way to put the Propel 1.6 “Instance Pool” into Memcache for a Symfony2 project I am working on, and I have managed it. Here is what I did; it may help you, but all the usual caveats apply. (Your mileage may vary. Use only under the direct supervision of your doctor. Do not allow children to use unsupervised. Not to be taken internally.)

What you have:

I am assuming that you currently have Symfony2 set up with Propel 1.6 as your ORM. If not there are guides (http://symfony.com/doc/current/book/installation.html, http://www.propelorm.org/cookbook/symfony2/working-with-symfony2.html) to help you. Do that first, then come back. I’ll wait.

The setup I am using assumes you will have multiple servers behind a load balancer of some sort, along with a memcache server that is accessible to all of them.

This is a sample of such a setup.

What you need in Symfony2:

Now that you have Symfony2 and Propel 1.6 set up, you need to create a new Symfony bundle to handle your Memcache connections. (You will see the need for this soon.)

What I have done is create a bundle which will contain all my shared utilities among all the apps built on this Symfony2 instance. For the sake of this article we will call the bundle “MyApp.” (I am not going to go into detail on how to create a Symfony2 bundle – the Documentation is your friend.)

Once we have created a bundle and registered it with app/AppKernel.php by adding

new MyApp\MyAppBundle\MyAppBundle(),

in the $bundles array under registerBundles(), we need to create a class to read configurations from the app/config directory (config_dev.yml, config_test.yml and config_prod.yml) so that our Memcache settings can be configured by environment. Note that we are not adding any routes for this bundle, it is used for utility stuff only, not for routable pages.

For this we will also need to make sure that we have MyAppExtension and Configuration in place in src/MyApp/MyAppBundle/DependencyInjection.

This gives us access to anything in the config_XXX.yml files under the “my_app” node.

The configuration class we will call “MyAppConfiguration” and will place this in the src/MyApp directory.

Now that we have a class that can read our own special configurations we need to add a class to handle the Memcache connection.

For this example we will create this class as Poolcache\CacheHandler. In order to do that we add a directory in src/MyApp called Poolcache and create the CacheHandler.php there. This means we can now set up the Memcache server locations based on the environment. For example, in config_dev.yml you might include:

my_app:
    cache:
        server: 127.0.0.1
        port:   11211
        expire: 120

While in config_prod.yml you might use:

my_app:
    cache:
        server: mymemcache.mydomain.dom
        port:   11211
        expire: 3600

(All the sample files can be downloaded here.)

Creating the Propel behavior:

Now that we have Symfony2 set up the way we need, we can add the Propel behavior. In the vendor/propel/generator/lib/behavior directory we will be adding MemcachedPoolBehavior.php.

This class uses the parser to replace some of the code in the generated classes.

<?php
require_once __DIR__.'/../util/PropelPHPParser.php';
class MemcachedPoolBehavior extends Behavior
{

    /**
     * Filter to add the CacheHandler class to the Peer objects so they
     * can use Memcached or whatever other cache you want to use
     */
    public function peerFilter(&$script)
    {
        $keyname = $this->getTable()->getPhpName();
        $newAddInstanceToPool = "
    public static function addInstanceToPool(\$obj, \$key = null)
    {
        if (Propel::isInstancePoolingEnabled()) {
            if (\$key === null) {
                \$key = (string) \$obj->getId();
            } // if key === null
            \$cache = \\MyApp\\Poolcache\\CacheHandler::getInstance();
            \$pool = \$cache->getPool('%s');
            \$pool[\$key] = \$obj;
            \$cache->setPool('%s', \$pool);
            /*self::\$instances[\$key] = \$obj;*/
        }
    }
    ";
        $newAddInstanceToPool = sprintf($newAddInstanceToPool, $keyname, $keyname);
        $parser = new PropelPHPParser($script, true);
        $parser->replaceMethod('addInstanceToPool', $newAddInstanceToPool);
        $script = $parser->getCode();

        $newRemoveInstanceFromPool = "
    public static function removeInstanceFromPool(\$value)
    {
        if (Propel::isInstancePoolingEnabled() && \$value !== null) {
            if (is_object(\$value) && \$value instanceof %s) {
                \$key = (string) \$value->getId();
            } elseif (is_scalar(\$value)) {
                // assume we've been passed a primary key
                \$key = (string) \$value;
            } else {
                \$e = new PropelException(\"Invalid value passed to removeInstanceFromPool().
                    Expected primary key or %s object; got \" .
                    (is_object(\$value) ? get_class(\$value) . ' object.' : var_export(\$value,true)));
                throw \$e;
            }
            \$cache = \\MyApp\\Poolcache\\CacheHandler::getInstance();
            \$pool = \$cache->getPool('%s');
            unset(\$pool[\$key]);
            \$cache->setPool('%s', \$pool);
            /*unset(self::\$instances[\$key]);*/
        }
    }
    ";

        $newRemoveInstanceFromPool = sprintf($newRemoveInstanceFromPool, $keyname, $keyname, $keyname, $keyname);
        //$parser = new PropelPHPParser($script, true);
        $parser->replaceMethod('removeInstanceFromPool', $newRemoveInstanceFromPool);
        $script = $parser->getCode();

        $newGetInstanceFromPool = "
    public static function getInstanceFromPool(\$key)
    {
        if (Propel::isInstancePoolingEnabled()) {
            \$cache = \\MyApp\\Poolcache\\CacheHandler::getInstance();
            \$pool = \$cache->getPool('%s');
            if (isset(\$pool[\$key])) {
                return \$pool[\$key];
            }
        }
        return null; // just to be explicit
    }
    ";

        $newGetInstanceFromPool = sprintf($newGetInstanceFromPool, $keyname);
        //$parser = new PropelPHPParser($script, true);
        $parser->replaceMethod('getInstanceFromPool', $newGetInstanceFromPool);
        $script = $parser->getCode();

        $newClearInstancePool = "
    public static function clearInstancePool()
    {
        \$cache = \\MyApp\\Poolcache\\CacheHandler::getInstance();
        \$cache->clearPool('%s');
        /*self::\$instances = array();*/
    }
    ";

        $newClearInstancePool = sprintf($newClearInstancePool, $keyname);
        //$parser = new PropelPHPParser($script, true);
        $parser->replaceMethod('clearInstancePool', $newClearInstancePool);
        $script = $parser->getCode();
    }

}

Notice that we are calling the static getInstance method on the cache handler class we created earlier, and using that to move the instancePool from the classes static properties to the memcached server.

Final steps:

Back in Symfony2, add the following to app/config/propel.ini:

#memcaching instance pool
propel.behavior.memcachedpool.class = behavior.MemcachedPoolBehavior

And in your Symfony2 application bundles, when you set your Resources/config/schema.xml for Propel you can add <behavior name="memcachedpool" /> to place the instance pool into memcache. Like so:

<?xml version="1.0" encoding="UTF-8"?>
<database name="default" namespace="Acme\HelloBundle\Model" defaultIdMethod="native">

    <table name="book">
        <column name="id" type="integer" required="true" primaryKey="true" autoIncrement="true" />
        <column name="title" type="varchar" primaryString="1" size="100" />
        <column name="ISBN" type="varchar" size="20" />
        <column name="author_id" type="integer" />
        <foreign-key foreignTable="author">
            <reference local="author_id" foreign="id" />
        </foreign-key>
        <behavior name="memcachedpool" />
    </table>

    <table name="author">
        <column name="id" type="integer" required="true" primaryKey="true" autoIncrement="true" />
        <column name="first_name" type="varchar" size="100" />
        <column name="last_name" type="varchar" size="100" />
        <behavior name="memcachedpool" />
    </table>

</database>

This means you can pick and choose which classes are maintained in Memcache as well, so that not every class needs to be there.