Quick Start

We'll start first with a bare bones example to show how little you need to get up and running. There's very little to configure. We've adhered to the convention over configuration philosophy so there are no code generators you need to run and no xml/yaml mapping files to maintain.

<?
// make sure the ActiveRecord project is in your current directory
// or your include_path
require_once 'ActiveRecord/ActiveRecord.php';
 
// assumes a table name of "books" with a primay key named "id"
class Book extends ActiveRecord\Model { }
 
// initialize ActiveRecord
ActiveRecord\Config::initialize(function($cfg)
{
    $cfg->set_model_directory('.');
    $cfg->set_connections(array(
        'development' => 'mysql://username:password@localhost/database_name'));
});
 
print_r(Book::first()->attributes());
?>

That's it! The code for this is located in the examples/simple directory. You can run it with "php examples/simple/simple.php". Make sure to modify the connection string to suit your system and run the simple.sql script appropriately.

Serious Business Time

Now for a not so completely trivial example. This example will simulate a very dumb ordering and payment model. See the examples/orders directory for the source. You can run this sample by executing "php examples/orders/orders.php". Again, make sure you modify the connection string to suit your system and be sure the schema in orders.sql has been created in your test database.

First, let's look at the models:

class Person extends ActiveRecord\Model
{
    // a person can have many orders and payments
    static $has_many = array(
        array('orders'),
        array('payments'));
 
    // must have a name and a state which has a custom error message
    static $validates_presence_of = array(
        array('name'), array('state', 'message' => 'Where do you live then?'));
}

class Order extends ActiveRecord\Model
{
    // order belongs to a person
    static $belongs_to = array(
        array('person'),
        array('order'));
 
    // order can have many payments by many people
    // the conditions is just there as an example as it makes no logical sense
    static $has_many = array(
        array('payments'),
        array('people',
            'through'    => 'payments',
            'select'     => 'people.*, payments.amount',
            'conditions' => 'payments.amount < 200'));
 
    // order must have a price and tax > 0
    static $validates_numericality_of = array(
        array('price', 'greater_than' => 0),
        array('tax',   'greater_than' => 0));
 
    // setup a callback to automatically apply a tax
    static $before_validation_on_create = array('apply_tax');
 
    public function apply_tax()
    {
        if ($this->person->state == 'VA')
            $tax = 0.045;
        elseif ($this->person->state == 'CA')
            $tax = 0.10;
        else
            $tax = 0.02;
 
        $this->tax = $this->price * $tax;
    }
}

class Payment extends ActiveRecord\Model
{
    // payment belongs to a person
    static $belongs_to = array(
        array('person'));
}

And here's the code that does everything:

<?
require_once dirname(__FILE__) . '/../../ActiveRecord.php';
 
// initialize ActiveRecord
ActiveRecord\Config::initialize(function($cfg)
{
    $cfg->set_model_directory(dirname(__FILE__) . '/models');
    $cfg->set_connections(array(
        'development' => 'mysql://test:test@127.0.0.1/orders_test'));
 
    // you can change the default connection with the below
    //$cfg->set_default_connection('production');
});
 
// create some people
$jax = new Person(array('name' => 'Jax', 'state' => 'CA'));
$jax->save();
 
// compact way to create and save a model
$tito = Person::create(array('name' => 'Tito', 'state' => 'VA'));
 
// place orders. tax is automatically applied in a callback
// create_orders will automatically place the created model into $tito->orders
// even if it failed validation
$pokemon = $tito->create_orders(array('item_name' => 'Live Pokemon', 'price' => 6999.99));
$coal    = $tito->create_orders(array('item_name' => 'Lump of Coal', 'price' => 100.00));
$freebie = $tito->create_orders(array('item_name' => 'Freebie', 'price' => -100.99));
 
if (count($freebie->errors) > 0)
{
    echo "[FAILED] saving order $freebie->item_name: " .
        join(',',$freebie->errors->full_messages()) . "\n\n";
}
 
// payments
$pokemon->create_payments(array('amount' => 1.99, 'person_id' => $tito->id));
$pokemon->create_payments(array('amount' => 4999.50, 'person_id' => $tito->id));
$pokemon->create_payments(array('amount' => 2.50, 'person_id' => $jax->id));
 
// reload since we don't want the freebie to show up (because it failed validation)
$tito->reload();
 
echo "$tito->name has " . count($tito->orders) . " orders for: " .
    join(', ',ActiveRecord\collect($tito->orders,'item_name')) . "\n\n";
 
// get all orders placed by Tito
foreach (Order::find_all_by_person_id($tito->id) as $order)
{
    echo "Order #$order->id for $order->item_name " .
        "($$order->price + $$order->tax tax) " .
        "ordered by " . $order->person->name . "\n";
 
    if (count($order->payments) > 0)
    {
        // display each payment for this order
        foreach ($order->payments as $payment)
        {
            echo "  payment #$payment->id of $$payment->amount by " .
            $payment->person->name . "\n";
        }
    }
    else
        echo "  no payments\n";
 
    echo "\n";
}
 
// display summary of all payments made by Tito and Jax
$conditions = array(
    'conditions' => array('id IN(?)',array($tito->id,$jax->id)),
    'order'      => 'name desc');
 
foreach (Person::all($conditions) as $person)
{
    $n = count($person->payments);
    $total = array_sum(ActiveRecord\collect($person->payments,'amount'));
    echo "$person->name made $n payments for a total of $$total\n\n";
}
 
// using order has_many people through payments with options
//
// array('people',
//    'through' => 'payments',
//    'select' => 'people.*, payments.amount',
//    'conditions' => 'payments.amount < 200'));
//
// this means our people in the loop below also has the payment information since
// it is part of an inner join we will only see 2 of the people instead of 3
// because 1 of the payments is greater than 200
$order = Order::find($pokemon->id);
echo "Order #$order->id for $order->item_name ($$order->price + $$order->tax tax)\n";
 
foreach ($order->people as $person)
    echo "  payment of $$person->amount by " . $person->name . "\n";
?>

The orders example should produce the following output:

[FAILED] saving order Freebie: Price must be greater than 0, Tax must be greater than 0

Tito has 2 orders for: Live Pokemon, Lump of Coal

Order #3 for Live Pokemon ($6999.99 + $315 tax) ordered by Tito
  payment #4 of $1.99 by Tito
  payment #5 of $4999.5 by Tito
  payment #6 of $2.5 by Jax

Order #4 for Lump of Coal ($100 + $4.5 tax) ordered by Tito
  no payments

Tito made 2 payments for a total of $5001.49

Jax made 1 payments for a total of $2.5

Order #3 for Live Pokemon ($6999.99 + $315 tax)
  payment of $2.50 by Jax
  payment of $1.99 by Tito

EXTENDED CONFIGURATION

Here's one more example that is basically the same as the first, but the point here is to display some of the ways to extend configuration (while remaining close to convention).

class Book extends ActiveRecord\Model
{
    // explicit table name since our table is not "books"
    static $table_name = 'simple_book';
 
    // explicit pk since our pk is not "id"
    static $primary_key = 'book_id';
 
    // explicit connection name since we always want production with this model
    static $connection = 'production';
 
    // explicit database name will generate sql like so => db.table_name
    static $db = 'test';
}
 
// ActiveRecord allows the use of multiple connections
$connections = array(
    'development' => 'mysql://test:test@127.0.0.1/development',
    'production' => 'mysql://test:test@127.0.0.1/production'
);
 
// initialize ActiveRecord
ActiveRecord\Config::initialize(function($cfg) use ($connections)
{
    $cfg->set_model_directory('.');
    $cfg->set_connections($connections);
});
 
print_r(Book::first()->attributes());
 

Simplicity

From the configuration, all the way to a serious-business example, ActiveRecord takes care of the heavy-lifting and the gritty details for you. This allows you, the developer, to focus more on business logic and complex code instead of composing custom SQL queries or designing ways to handle your data. Because we have embraced a convention over configuration philosophy, using our library is not painful. The conventions are easy to remember which will also contribute to stream-lining your productivity as a developer.

Where can you find ActiveRecord?

Again, the code is hosted on http://github.com/kla/php-activerecord/. We also have a website which will be available in the near future. We hope to have all the necessary content such as: tutorials, screencasts, and documentation.

pnPoly - Used to determine if a coordinate falls inside a polygon.
Centroid - Find the center of a polygon..
Area - Calculate the area of a polygon.
googleGeoCoder - Extracts GIS information from Google Maps from an address.
PolylineEncoder - Takes a set of coordinates and encodes it for Google Maps.

If you ran into the problem I did, which is that a lot of the data is coming in the form of shp/dbf files and needs to be parsed out to something friendlier either KML or CSV, there are a couple of solutions for that. You can parse out the data with shp2text if your source coordinate format is already in lat/lng or if you have different coordinate system and use ArcGIS, you can try the plugin Export to KML 2.5.3 to help with the exporting of data with the ESRI suite of products.

Once your data is in SQL, the following query is an example of distance sorting with SQL. You can grab a copy of the zip_codes database here and play around with it.

SELECT *,
sqrt((69.1 * ("37.6" - latitude)) * (69.1 * ("37.6" - latitude)) +
(53.0 * ("-77.6" - longitude)) * (53.0 * ("-77.6" - longitude)))
AS distance
FROM `zip_codes`
HAVING distance < 10
ORDER BY distance ASC

/*
SQL script to grab the worst performing indexes
in the whole server
*/
SELECT
t.TABLE_SCHEMA AS `db`
, t.TABLE_NAME AS `table`
, s.INDEX_NAME AS `inde name`
, s.COLUMN_NAME AS `field name`
, s.SEQ_IN_INDEX `seq in index`
, s2.max_columns AS `# cols`
, s.CARDINALITY AS `card`
, t.TABLE_ROWS AS `est rows`
, ROUND(((s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) * 100), 2) AS `sel %`
FROM INFORMATION_SCHEMA.STATISTICS s
INNER JOIN INFORMATION_SCHEMA.TABLES t
ON s.TABLE_SCHEMA = t.TABLE_SCHEMA
AND s.TABLE_NAME = t.TABLE_NAME
INNER JOIN (
SELECT
TABLE_SCHEMA
, TABLE_NAME
, INDEX_NAME
, MAX(SEQ_IN_INDEX) AS max_columns
FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA != 'mysql'
GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
) AS s2
ON s.TABLE_SCHEMA = s2.TABLE_SCHEMA
AND s.TABLE_NAME = s2.TABLE_NAME
AND s.INDEX_NAME = s2.INDEX_NAME
WHERE t.TABLE_SCHEMA != 'mysql'                         /* Filter out the mysql system DB */
AND t.TABLE_ROWS > 10                                   /* Only tables with some rows */
AND s.CARDINALITY IS NOT NULL                           /* Need at least one non-NULL value in the field */
AND (s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) < 1.00 /* Selectivity < 1.0 b/c unique indexes are perfect anyway */
ORDER BY `sel %`, s.TABLE_SCHEMA, s.TABLE_NAME          /* Switch to `sel %` DESC for best non-unique indexes */
LIMIT 10;

To audit just one database if you are running on a server with several different databases, just adjust the where clause to WHERE t.TABLE_SCHEMA = 'mytable'. This would have been very useful when working with the cluster to recover memory/space from indexes that aren't being used and to optimize queries to hit indexes that are meaningful.

1. Download the latest version at http://dev.mysql.com/downloads/mysql-proxy/.
2. yum install glib2-devel.x86_64 ncurses-devel.x86_64 libevent-devel.x86_64 mysql-devel.x86_64
3. ./configure -without-lua (for a straight proxy, you will not need LUA and this will save you having to install additional dependencies).
4. make && make install

Easy enough to get it installed in the system, but I noticed that the default parameters and as with most software is rather wide open so be sure to lock down the ip addresses you are listening on.

mysql-proxy --proxy-backend-addresses=192.168.1.101:3306 --proxy-backend-addresses=192.168.1.102:3306 --admin-address=127.0.0.1:4041 --proxy-address=127.0.0.1:4040 --proxy-skip-profiling --daemon

The admin address and proxy address will be accessible by anyone hitting your server unless you lock it down to your local host. Now configure your web servers to use localhost:4040 to connect to MySQL and all should be working fine. You can find additional configuration parameters, admin commands, and LUA scripts at http://forge.mysql.com/wiki/MySQL_Proxy.

• Price- An m1.xlarge instance will run you ~$600 with data transfer costs. Managed hosting solutions run cheaper especially if you plan on purchasing in bulk. The grid is designed for on-demand computation and not as a cost efficient web services.
• Configuration- There are a limited number of options and you will not be able to tailor the hardware to your application. Databases over 10GB of size will have performance issues since that is the memory cap.
• Network storage- The primary disks offers limited storage, additional volumes will need to be attached across the network and at an additional cost.
• Software - No hardware based solutions for load balancing or custom application servers. The model is software driven so all needs must be met with a software solution. In a managed hosting or collocation solution you will at least have the option of adding additional hardware and having a private network. No dedicated switching, routers, firewall, or load balancers.

EC2 might be right for you if:

• Distribution Awareness- Your application was designed to scale horizontally from the get-go and you can take advantage of grid computing.
• Research and Development- EC2 & Rightscale will allow for you to bring up new servers, test configuration, and scale quickly. If you are not sure what your hardware demands will be or the scope of the project it will allow for some flexibility to get this right before committing to rather lengthy contracts with other hosting options.
• Disaster Recovery- If you need an off-site mirror for your site that you can keep dormant and activate as needed.

Over the past several months I have been doing extensive development using Amazon's EC2 as my hardware infrastructure. I was tasked with taking CitySquares.com from a New England area hyper-local search and business directory to a national site in a few months. Due to the memory limitations of EC2 instances, m1.xlarge only providing 15GB the jump from a 15GB database to anything larger becomes very costly. When everything was able to be contained in two servers in a master/slave environment we were able to provide redundancy, performance, and easy management when working with the database. The final estimations of the national roll out would put our core data at 50GB. Far too large for any one EC2 instance. Going to disk was not an option as everything works off of EBS attached storage and that any disk writes means traveling over the network. An additional overhead which degrades performance even more when switching off RAM. Then, there was also the nature of the data, which means that at any page load, any piece of data could be requested.

With OS overhead, index storage, and ndb overhead, each x1.mlarge instance gave about 12GB of usable storage. Include replication, and it comes down to 6GB of storage per node. To store a ~50GB database that I had planned on requires 8 storage nodes, two management servers, and two mysqld api servers. This is where it became important to understand the advantages of vertical scaling versus horizontal scaling. EC2 provided fast horizontal scaling and configuration. Servers can be launched on demand and their configurations scripted. While I appreciated that aspect of cloud computing and being able to bring that many servers up and configure each one relatively quickly I really just needed two decent boxes with 64GB of RAM in each and a master/slave setup. The operations costs for the cluster was $6000/month, a hefty bill considering I could buy all the hardware needed to run the cluster in just a few months of paying for EC2.

Twelve servers later and a working cluster we were able to successfully roll out our MySQL Cluster with minimal performance loss. A lot of it was due to tweaking every query top to bottom. The site runs on a Drupal core, which meant a lot of the queries were not designed with distribution awareness both from code in house and the core. This was another added growing pain since the network overhead of running 12 servers on shared resources, with mediocre latency, and throughput limitations was amplifying flaws in the database design and every poorly designed query and join degraded performance significantly.

To give credit where credit is due, EC2 did allow us to scale the site up rather quickly. We were able to test server configurations, new applications, and have easy management. It would not have been possible for us to push out the data, handle influx of new traffic, and expand as fast as we had without it. Long term however, it made absolutely no sense that once we were finished scaling up, to stay on EC2. It is a great platform for start-ups to be able to configure and launch servers for their service or application and grow rapidly.

As a database platform, until EC2 offers more configuration options in it's hardware and the ability to increase memory, the cap of 15GB will make EC2 problematic for any database that plans on growing past that. It is important to understand your application and database needs before considering EC2. It is no surprise to me that even with the RightScale interface, and easy management of EC2 web sites are reluctant to switch off their own hardware or managed hosting.

Both Sun and Continuent are pursuing MySQL Clustering on cloud computing. As of this post, Continuent for EC2 was still in closed beta and testing and Sun is doing their own research into offering more support for database clusters on compute clouds. Maybe in the future this can be something to revisit, but it would require more ndb configuration options on the network layer to cope with shared bandwidth and additional hardware configurations by Amazon (someday).

The 6.4 release which is in beta now offers new features which would make MySQL Clustering more attractive on cloud computing architecture:

• Ability to add nodes and node groups online. This will allow the database to scale up without taking the cluster down.
• Data node multithreading support. The m1.xlarge instance comes with 4 cores.

If you are setting up a MySQL cluster, the following resources will help get you up and running quickly:

• http://www.severalnines.com/
• http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-configuration.html