It Stops Here

April 21st, 2008

In some ways I miss spam. Of course I loathe it, and I believe that if there’s a hell, there’s a special part of it set aside for the evil Viagra-peddling scumbags that blight our inboxes. But now that I’ve filtered most of it out, it becomes starkly apparent how few actual real people want to mail me on a daily basis. I used to cycle through through screen after screen of penile bling painrelief awfulness and hate every second, but at least there was the possibility, the merest chance, that someone had decide to publish that short story, or offer me a fantastic tech book deal. Nowadays I must come to terms with my relative unpopularity at a single glance.

Nevertheless, I thought I’d release my mail quarantine package for any other sufferers who happen to use a procmail-like mail filter. It’s called ISH (for It Stops Here), and it works for me.

stickleback tricks part 2: providing default functionality for an extension point

March 22nd, 2008

In part 1 of this short series I looked briefly at mechanisms for passing parameters to plugins. In this piece I’m going address another of Jun’s points. This time: how to provide a default implementation of certain extension point interface functions.

First, let’s break down what we want to achieve here. stickleback defines its extension points in two ways. Firstly an extension point is associated with a plugin in XML metadata, that is via the <epoint> element in the sbplugin.xml file which must be present in a pluginset. The second aspect to an extension point is its interface. It is this that enforces the functionality that an extending plugin must provide. Here is the extension point interface in my ongoing example:

interface DoThisPleasePlugins extends stickleback_Plugin {
    function sayHello( $requiredvalue );
    function sayGoodbye();
}

Now all plugins that extend the MyAppPlugins host plugin along the DoThisPleasePlugins extension point are forced to support sayHello() and sayGoodbye(). In a non-trivial example though, it’s likely that you’ll want to provide some core functionality for your plugin authors above and beyond the simple interface they must implement.

Let’s say, for example, that you would like to provide an extendable implementation of the DoThisPleasePlugins interface. The functionality I provided for the someone plugin last time looks like it might be generically useful. By making it available as the optional default behavior for all plugins I can save a lot of duplication. By convention, I use the name of the extension point interface followed by Impl — short for implemented or implementation. Here is DoThisPleasePluginsImpl class. I drop it into the MyAppPlugins directory.

abstract class DoThisPleasePluginsImpl implements DoThisPleasePlugins {

    function sayHello( $requiredvalue ) {
        $this->doPrint("$requiredvalue\n");
    }

    function sayGoodbye() {
        $reg = stickleback_PluginRegistry::getInstance();
        $desc = $reg->getPluginDescriptor( $this );
        $bye = $desc->parameter("goodbye");
        $this->doPrint((is_null( $bye ) ? "goodbye" : $bye)."\n");
    }

    abstract function doPrint( $str );
}

Note that I would usually name a class in a pluginset directory as if the pluginset were its package. So I could have called this class MyAppPlugins_DoThisPleasePluginsImpl — but that’s such a mouthful, that I’ve decided to forego the convention. stickleback does not enforce any naming conventions for classes or extension points, and it doesn’t care where a class or interface is found, so long as it is found. It makes sense though to put a stub plugin in the relevant pluginset directory for ease of distribution.

So The functionality here should be familiar from part 1. sayHello() requires a value, and sayGoodbye() acquries a value via a parameter in the sbplugin.xml file. The difference is that I’ve moved it into an abstract base class. Notice also that I’ve deployed the template method pattern. In other words both my implemented methods call an abstract method: doPrint(). This is a typical trick. It allows common behavior to be shared, but requires extending classes to provide their own implementation of a small part of the shared workflow.

Now that I’ve provided an abstract base class, the someone plugin can be much simplified:

require_once( "MyAppPlugins/DoThisPleasePluginsImpl.php" );

class someone extends DoThisPleasePluginsImpl {
    function doPrint( $str ) {
        print "someone says $str";
    }
}

So now, instead of implementing the DoThisPleasePlugins extension point interface, all the someone plugin class has to do is to extend the DoThisPleasePluginsImpl class. I could choose to override sayHello() or sayGoodbye(), but in this case all I do is implement the doPrint() method. In this way we can vastly simplify the process of building a plugin for our users. In fact, when building pluggable applications I almost always work in this way, providing much default functionality. Of course a plugin author could chose to provide their own implementation — that’s their privilege, but design your abstract plugin stubs right, and most plugin authors will gladly use them.

Finally, then, here is the script’s new output:


$ php MyApp.php
someone says wotcha
someone says cheerio

stickleback tricks part 1: passing parameters to plugins

March 21st, 2008

Commenter Jun posted a really handy set of questions about stickleback. These questions were posed off the back of my stickleback quickstart articles, so I’ll extend the code I used there.

So, to kick off, how can we provide additional input parameters and objects to the extender plugin?

The simplest way of doing this of course is for the extension point interface to demand input values through type hinting. This forces the host plugin to provide the required parameter on calling the plugin’s implemented method. First of all I amend the interface that enforces the extension point so that sayHello() requires an argument:

interface DoThisPleasePlugins extends stickleback_Plugin {
    function sayHello( $requiredvalue );
    function sayGoodbye();
}

Then I implement my extender plugin so that it respects the new interface. Note that the new argument in the sayHello() declaration is not the only change here. The DoThisPleasePlugins interface now extends from stickleback_Plugin — this means that any class that implements DoThisPleasePlugins is automatically of type stickleback_Plugin — this will become significant later on.

class someone implements DoThisPleasePlugins {

    function sayHello( $requiredvalue ) {
        print "$requiredvalue\n";
    }
    //....

Finally any client must now provide an argument when it calls DoThisPleasePlugins::sayHello(). Here’s and amended version of MyApp::execute().

    function execute() {
        $reg = stickleback_PluginRegistry::getInstance();
        $desc = $reg->getPluginDescriptor( 'MyAppPlugins' );
        $plugins = $desc->getExtendingPlugins( "DoThisPleasePlugins" );
        foreach ( $plugins as $plugin ) {
            $plugin->sayHello( "wotcha" );
            $plugin->sayGoodbye();
        }
    }

I pass the string “wotcha” to the plugin. Here’s what we get when run the application:

$ php MyApp.php
wotcha
goodbye

Really, this is just standard coding — which is no surprise, since stickleback uses object-oriented features of PHP such as interfaces, type hinting and inheritance.

You can however use stickleback itself to parameterize plugins. This is particularly useful since a single plugin class can be used in multiple plugins. By passing different values to your plugin objects through the sbplugin.xml document you can construct plugins objects that behave in varying ways.

Here is plugins/someone/sbplugin.xml .

<?xml version="1.0" standalone="yes"?>
<pluginset
    name="someone"
    xmlns='http://developer.yahoo.com/xmlns/stickleback/sbconfig'>
    <plugin name="someone" type="someone">
        <honors plugin="MyAppPlugins" epoint="DoThisPleasePlugins" />
        <param name="goodbye" value="cheerio" />
    </plugin>
</pluginset>

As we’ve seen this pluginset defines a single plugin, someone which extends its host MyAppPlugins at extension point (ie interface) DoThisPleasePlugins. I have added a single line:

        <param name="goodbye" value="cheerio" />

By the magic of stickleback, this parameter becomes available to a plugin’s descriptor. A plugin can easily get access to its own plugin descriptor, and through that can acquire any parameters. Here’s the someone class:

class someone implements DoThisPleasePlugins {

    function sayHello( $requiredvalue ) {
        print "$requiredvalue\n";
    }
    function sayGoodbye() {
        $reg = stickleback_PluginRegistry::getInstance();
        $desc = $reg->getPluginDescriptor( $this );
        $bye = $desc->parameter("goodbye");
        print (is_null( $bye ) ? "goodbye" : $bye)."\n";
    }
}

In MyApp I passed a string name to stickleback_PluginRegistry::getPluginDescriptor() but the method also accepts an plugin object — allowing plugins to acquire their own descriptors with ease. At the time of this writing a plugin must implement the stickleback_Plugin class for this to work, but this requirement will probably be relaxed in future versions. The someone plugin is now parameterized. Any value addded to the relevant plugin/param element in sbplugin.xml will be picked up by someone::goodbye().

Here’s the full output now:

$ php MyApp.php
wotcha
cheerio

Next stickleback trick? I’ll show you how to save duplication by providing a default implementation for an extension point.

openr3: a new r3 site

February 23rd, 2008

Although it’s a pleasure to blog about my projects at Yahoo! they do somewhat dominate these pages at times. In order to encourage an open source community around these tools, a group of us are building a resource and information site over at openr3.com. There is very little there at the time of this writing, but we have a bunch of potential contributors, and a lot to talk about. We should be getting input from core developers, and I hope from some of the many web developers who use r3. This power user perspective has been missing from getinstance, and from other articles, and I hope it will set openr3.com apart.

A side effect of all this is that these pages will be freed up for more general content. More stick man cartoons anyone? Suit yourself.

r3 on Yahoo! Developer Network

February 14th, 2008

My project at Yahoo! is r3 — a flexible tool for localizing sites, configuration files and applications (across markets, cobrands, languages, or anything you like really). When we learned that Yahoo! was happy to support the release of certain properties to the open source community we were quick to volunteer r3 and stickleback (the plugin engine which drives r3’s interfaces and much else besides). We got the go-ahead after some scrutiny, and have lurked in the softest of launches for many months now.

Yesterday, though, we went official. You can now find full documenation at developer.yahoo.com. There are some more details in Norm’s announcement on the YDN blog.

Behind the scenes at the moment we’re labouring on some major new features. A vastly improved UI for one thing, some Web Services support, and mutable dimensions — a major new release is expected in about a month (though you’ll probably see point releases sooner).

no comment

February 7th, 2008

Roger Lancefield who left a really nice comment about PHP Objects Patterns and Practice in these pages recently posted a yahoosoft logo on his blog. Since I am officially barred from advancing any substantive opinion on the topic I’ll confine myself to two points:

  1. microhoo sounds better
  2. I for one welcome our new overlords

stickleback and r3 1.3.5 launched to Open Source by Yahoo!

January 31st, 2008

Yesterday we released stickleback 1.3.5 and r3 1.3.5.

Mainly a refactoring release, this provides significant improvements in database speed and template generation.

Not sure what r3 is? Check out this quick start.

We’ll be releasing again in the next few days with an important enhancement (embeddable r3 API). Also expect some more announcements in the next couple of weeks.

PHP Dogfood - cheap and cheerful profiling

January 16th, 2008

I spend a lot of my time debugging and profiling, and tend to write the same scriptlets over and over again. Clearly I should heed my own advice and put things into a package or two.

Dogfood is just that — a bunch of low rent tools for doing simple stuff. In particular: measure the time between two debug statements, write comments to standard error (very handy when output buffering swallows your ‘hello world’ messages).

It also provides method profiling — outputting the duration of every method in a process, both in the method’s own right and including the time taken by methods further down the stack.

This is from the notes:

D::bleat( String ) - debug message to STDERR, with line number,
                     duration and mem usage info included
D::stderr( String) - message to stderr
D_Profile::start() - profile a script -- need to invoke like
                     this from global scope:

                     require( "D/Profile.php" );
                     declare( ticks=1 );
                     D_Profile::start();

Download Dogfood-1.0.0.tgz or:


sudo pear install http://www.getinstance.com/pkg/Dogfood-1.0.0.tgz

This code is a slightly polished version of code to be discussed in an upcoming php|architect article.

pluggable CLI apps with stickleback — part 3

January 2nd, 2008

In part 2 I showed you how to set up a basic CLI environment using stickleback. Now it’s time to add a command.

All stickleback plugins have an XML component and an existence in code (though in some cases you will be able to reuse existing classes, and configure them exclusively with the relevant XML entry). Here I add the XML for an Add command:

    <plugin name="calccli_Add" type="calccli_Add">
        <param name="cmd-aliases" value="add" />
        <param name="arg-spec" value="left right" />
        <param name="describe" value="add left to right" />
        <offers epoint="sbcli_flag" />
        <honors plugin="calccli" epoint="sbcli_cmd" />
    </plugin>

Most of this should be familiar by now if you’ve been reading along. I’m extending the calccli command I created previously. I have included a new <param> element, however. arg-spec defines the arguments a command requires. You should name the arguments, and separate them by spaces. You can all the modifiers ‘?’ (zero or one), ‘+’ one or more, or ‘*’ zero or more, after a named argument if you need to. In this case I require exactly two arguments: left and right. Let’s create the calccli_Add class to which this command refers.

require_once("Calculator.php");

class calccli_Add extends sbcli_cmdImpl {

    function doHandleEvent( sbcli_Event $event, array $args ) {
        $left  = $args['left'][0];
        $right = $args['right'][0];
        $event->addOut("$left + $right = ".Calculator::add($left,$right));
    }
}  

I extract the left and right values from the $args array argument. Why didn’t I bother to check that the keys existed? Because stickleback takes care of that for me, silly. I add this class to a file called Add.php and drop it into the calccli pluginset directory, and I’m done.

Now to prove that stickleback enforces the required arguments:

$ ./calc add
ERROR: command: 'add' argument 'left' required

usage:calc add <left> <right>
add left to right

In addition to enforcing the argument list, sbcli provides usage information, using the describe and arg-spec parameters.

Finally, let’s provide the command with the requisite number of arguments.

$ ./calc add 4 6
welcome to calc
4 + 6 = 10

In Part 4, I will wrap up by adding support for flags.

pluggable CLI apps with stickleback — part 2

December 22nd, 2007

sbcli participants on the filesystem In part 1 I introduced the sbcli pluginset. In order to use it, you need to create a point of entry for your users. Here’s a simple script in a file named calc:

#!/usr/bin/php5
<?php

require_once("stickleback/PluginRegistry.php");
$localdir = dirname(__FILE__) . DIRECTORY_SEPARATOR;

$reg = stickleback_PluginRegistry::getInstance();
$reg->addPluginDir( "{$localdir}bundledplugins" );
$reg->addPluginDir( "{$localdir}userplugins" );
$reg->findPlugins();
$argv[0] = "calc";

$runner = new stickleback_Runner( );
$runner->run(  array( 'use' => 'sbcli', 'argv' => $argv )  );
?>

I acquire a reference to the stickleback_PluginRepository object and establish two plugin directories: bundledplugins and userplugins. This is a typical configuration, in that it allows both for core and contributed plugins. Of course, in the real world you would need to decide where these directories should reside. PEAR can help you with your own plugin directory. The contributed plugins directory might be controlled from a configuration file.

Having set up a plugin environment, I need to start my invocation somewhere. An sbcli interface consists of a tree of sbcli_cmd objects. The end user’s command-line input is analysed by the sbcli plugin object which invokes the correct sbcli_cmd objects. I pass the user’s input to the sbcli object via a stickleback_Runner object. This is a simple helper that handles the basics of finding the plugin to run, and passing it the argument list.

Why did I change $argv[0]? sbcli employs the user’s input to locate which sbcli_cmd plugins to run. The first element in a command line call might be calc, but it might equally be ./calc or /my/path/calc.

Now that I’m invoking sbcli, I need to make sure that an sbcli_cmd plugin called calc exists for it to find. There are always two aspects to creating a plugin with stickleback. There’s the class, and there’s the metadata. Let’s begin with the class at ./bundledplugins/calccli/Calc.php.

class calccli_Calc extends sbcli_cmdImpl {

    function doHandleEvent( sbcli_Event $event, array $args ) {
        $event->addOut("welcome to calc\n");
    }
}

Command plugins must implement the sbgui_cmd interface (remember that stickleback uses interfaces to define its extension points). sbcli provides a partial implementation, however, in the sbgui_cmdImpl class which itself implements the interface. Unless you want to do something unusual therefore, the easiest way of creating a command is to extend sbgui_cmdImpl.

As you can see, all you’re required to implement is doHandleEvent(). This is where you do your commanding stuff, whatever that may be. Notice that you’re given an sbcli_Event object. Why call its addOut() method rather than simply printing your output? It’s up to you really - printing works fine, but because there may be a pipeline of commands you may need to support undo functionality, should one of the commands in the pipeline fail (cf the Gang of Four’s discussion of the Command pattern). For the purposes of this keep-it-simple example, though it really doesn’t matter which way you go.

No stickleback pluginset or plugin will be recognised without the metadata to be found in an sbplugin.xml file. This pluginset’s file is to be found at ./bundledplugins/calccli/sbplugin.xml. Here it is:

<?xml version="1.0" standalone="yes"?>
<pluginset
    name="calccli"
    xmlns='http://developer.yahoo.com/xmlns/stickleback/sbconfig'>

    <plugin name="calccli" type="calccli_Calc">
        <param name="cmd-aliases" value="calc" />
        <param name="describe" value="a simple calculator" />
        <offers epoint="sbcli_cmd" />
        <offers epoint="sbcli_flag" />
        <honors plugin="sbcli" epoint="sbcli_cmd" />
    </plugin>

</pluginset>

Let’s break this down. Remember a plugin essentially does two things. It extends (&lt;honors&gt;) host plugins and provides (&lt;offers&gt;) extension points for its own extenders. The eponymous plugin for the calccli pluginset (every pluginset must have an eponymous plugin) extends sbcli. CLI command plugins should either extend one another, or extend sbcli. This one, being the root of our CLI application plugs straight into the source. It offers two extension points: sbcli_cmd and sbcli_flag.

The &lt;param&gt; elements are a way of passing information from the sbplugin.xml file to the instantiated plugin. In this case describe is used in usage messages and cmd-aliases lists the commands that the plugin should recognize from user input. You may want to label your command generate, for example, but also accept gen for short. In this case, the name calc is important, because it’s the hard-coded first argument the executable set at $argv[0].

Let’s run the code:

$ ./calc
welcome to calc

There’s more to see though. What if we cause an error by passing our command unexpected input? Here’s what:

$ ./calc kjlj
ERROR: command 'kjlj' not found

usage:calc
a simple calculator

As you can see sbcli automatically generates usage information for you. As you will see in future articles, such messages can be quite complex.

In part 3, I will extend this example to include a subcommand that can actually do stuff.