Codesta Weblog

Seven iPhone Development Rules of Thumb

I've been an Apple developer since it was called NeXT and found the following rules of thumb helpful when introducing new people to Apple development.

1. Read the introductory documentation.  Build a simple application.  Read the documentation again.

The Apple development environment is very different from other application development environments. There is going to be some ramp up time. This isn't like learning .NET after you learned Java.

Expect to throw away your first couple of applications.

2. Don't buck the system.

This is a general theme in the Apple world. If you do things the Apple way, life is pretty easy. If you try to do things the MFC/.NET/Java way... it will be very hard.

Apple heavily uses the delegate pattern. Don't fight it. Apple's view of MVC is a little odd. Adapt.

3. If it seems hard, you're probably doing it wrong.

The Apple developer forums are full of people tracking down difficult problems that are solved by simply doing it in a slightly different way. Not all of these are cases of RTFM, sometimes it's just a trick that all the experienced developers know and you just need to ask.

If something seems harder than it should, head for Google or Bing sooner rather than later!

4. Respect "Apple doesn't want you to do this".

Apple makes customization of most things easy and customization of things it doesn't really want you mucking with very hard. This has been the case since Mac OS X was NeXTStep and cell phones were the size of a brick.

Apple has stringent rules on iPhone apps. Apple wants you to do things a certain way, so just go with the program. Or develop for Android/Blackberry/Pre.

5. Understand retain, copy, release and autorelease.

Really, really understand them. You need to know at a visceral level when to retain or copy, when to release and when to autorelease.

If you run into a retain/release imbalance, stop and think about it until you understand where you went wrong. Don't just start inserting retains and releases until the problem goes away. If you really think about these every time it won't be too long before you never create them again.

6. Learn the difference between should, will and did methods.

Create a simple delegate example and log when every method is called. Read the documentation before and after you think you understand how this all works.

7. Don't ignore warnings.

Warnings are often telling you that your code doesn't do what you think it is  doing. Or that things don't work the way you think they do. 

Beyond Pathauto

The Pathauto module is the Drupal supplied solution to automatically create URL aliases based on the title of a node. You can adjust this in the Pathauto configuration following some basic patterns, but sometimes this is not enough.

Drupal has another mechanism with which you can create custom URL’s for each node. One just has to edit the node, find the “URL path settings” fieldset, and add an alias to that node. If you have a website that publishes a lot of content, doing this manually will become tedious, error prone, and even impractical.

We had a requirement on a recent project where there were Drupal nodes that had to be accessible from a URL that was not possible to automatically generate with the Pathauto module. Our custom content type stored data for a media review that was one of several types: a video game review, movie review, book review and so on. For SEO purposes, we needed to generate a URL based on the review type and its title, for example a game would have the URL games/game-title, a movie would have movies/movie-title. The problem with the Pathauto module was that it could only categorize a content type with one type of URL pattern so all reviews could be stored in games/game-title or movies/movie-title, but not both. Using a custom Drupal module allows you the flexibility to create different URL alias patterns based on your own rules and data. Here is how:

Step 1: Configure Pathauto

• Go to the Pathauto configuration page at "/admin/settings/pathauto".
• Click on the "Node Path Settings" field set.
• Find the Label "Pattern for all Page", then set the "pattern" for that content type.
• For now we just need to add the "[title]" pattern in the text box. 

Step 2: Create a Module:

We need to create a module that is the same name as the CCK content type you have existing in your system. In our case we would create a module named Review.module. (Please see Drupal documentation for how to add modules).

Step 3: Override the Hook Nodeapi:

What we will do in the module is hijack this pattern and add what we want to it.
In the review.info module you will need to add a dependency with the AutoPath module to ensure that it is turned on for this to work (Please see Drupal documentation for how to add module dependencies).

Next edit the review.module file and add the following code.

// This allows us to use the functions defined in the
// autopath module.
require_once '../modules/contrib/pathauto/pathauto.module';

//Now all we need to do is to overload the node_api hook

function review_nodeapi (&$node, $op, $a3 = NULL, $a4 = NULL){

  $is_inserting = FALSE;
  if ($node->type == 'review') {
if ($op ==  'insert'){
  $is_inserting = TRUE;
}

// Preparing the array for the URL alias insertion   
$placeholders = pathauto_get_placeholders('node', $node);

// Detect what type it is, and modify the title
// the Pathauto module must be enabled for this to work.
// $node->node_type stores the type of review
//  (‘game’,’music’, ‘movie’)
$node_type = $node->node_type;

// pathauto_cleanstring removes all spaces and special
// characters from the title in preparation of url format.
$cleantitle = pathauto_cleanstring($node->title, FALSE);

// The url that will be created: 
$cleantitle = ‘reviews/’ . $node_type . '/'. $cleantitle;

// Place it in the array in the 15, 16 position
// (this indicates it is a alias for a node)
$placeholders['values'][15] = $cleantitle;
$placeholders['values'][16] = $cleantitle;
if ($is_inserting) {
  pathauto_create_alias('node', 'insert', $placeholders,
"node/" .$node->nid,  $node->nid, $node->type);
} else  {
  pathauto_create_alias('node', 'update', $placeholders,
"node/" .$node->nid,  $node->nid, $node->type);
}
  } // end if node->type = ‘review’
} // end node_api hook

Now any time you add a review it will have a url alias that consists of www.yoursite.com/reviews/[review-type]/[title-of-review]’

Drupal: Easing Content Type Proliferation

In this blog post we will show you how to allow content editors to edit similar but different "pages" in Drupal, all based on a shared content type without confusion, by hiding the unnecessary form elements.

There can be times when you wish to create several similar content types that only differ slightly from one another (perhaps by a field or two). Let’s take a website that posts environmental warnings on different types of pollution as an example. The site could report air pollution, water pollution and ground pollution warnings. One option is to create 3 separate content types, one for each type of pollution. Air pollution may have an air quality index field, while ground pollution may have a field that shows how it affects the property value of nearby buildings. The point here is that most of the fields for all 3 content types are the same. Discovery Date, Detail Review, Severity, and so on. An alternative implementation is to use one content type called Pollution and a field in that content type named ‘Pollution_type’. This field in the content type would allow us the flexibility to know what type of pollution we are dealing with.

The problem comes when an environmental investigator wishes to add a new air pollution warning. They will see a Drupal form with a ground pollution area field, land value field and all the other fields which are not related to the air pollution type. This can make the user interface crowded with unrelated fields, and unusable. So how can we have one content type and one form which adapts to the type of pollution a user has selected? 

Simple.

By overriding the node_form.tpl file in the themes directory for the ‘pollution’ node type, we can take full control of the appearance of our form. We can then add some JavaScript, or better yet JQuery, which will hide and show elements based on which review type the user has chosen.

Step 1: Set Up CCK Fields

To set this example up we need a simple CCK content type with the following fields (please see Drupal documentation about how to create CCK fields):

Review_ type : drop down with default fields: ( 1 – air, 2 – ground, 3 – water)
hazard_time (text box)
area_affected (text box)
warning_date (date)
warning_details (text area)

etc ...

Step 2: Override the Node Form for the Review Content Type

To do so you need to create a function in the template.php file (which will be found in your themes directory). The name of the function will need to take on the following form:

phptemplate_[content_type]_node_form() and in our case the function will look like the following:

function phptemplate_pollution_node_form($form) {
    drupal_add_js('sites/all/modules/js/pollution.js');  

    // Add JQuery to act on the form
    // load the review_node_form.tpl.php file to render this 
    //form.   
    return _phptemplate_callback('pollution_node_form',
    array('form' => $form));  
}

The reason why this works is because when Drupal processes the form for a particular node that has been queued up to render, it will search through the template.php file for a function in the form of phtemplate_[content_type]_node_form: If it finds one it will then load and render the form returned by this function.

Step 3: Render a Custom Form

In the themes directory you need to create the form theme function that is returned by the function in the template.php function. In our example this would be pollution_node_form.tpl.php

Now in the pollution_node_form.tpl.php file we create the HTML and attributes we need to render the form. This includes any tags and classes that will help us when we use JQuery to hide and show the form.

<?php>

<legend> Type And Title </legend>
<?php echo drupal_render($form['field_pollution_type']); ?>
    <?php echo drupal_render($form['title']); ?>
<div id=’warning’>
    <?php echo drupal_render($form['field_warning]); ?>
</div>
<div id= 'land_pollute_field’>
    <?php echo drupal_render($form['field_land_area_affected
]); ?>
</div>
<div id= ‘air_pollute_field’>
    <?php echo drupal_render($form['field_direction_of_
movement]); ?>
</div>
// render other field from the CCK content type ..
<?php>
    // Unset form elements that you don’t want rendered
        unset($form['attachments']);
        unset($form['menu']);
    …
     // Render the rest of the form …
        echo drupal_render($form);
?>

Now to get the JQuery to connect with the HTML id’s of each separate div we add the following JQuery file to the JavaScipt file referred to in Step 2. In our example the file should be named warning.js and be in the /themes/yourtheme/js directory.

$(document).ready( function() {
    // Need to initialize on page load which fields are shown on the page.
    $("#land_pollute_field").css("display","block");
    $("#water_pollute_field").css("display","none");
    $("#air_pollute_field").css("display","none");

    var choice = $("#edit-field-pollution-type").val();  
// You will need to find the ‘id’ of the dropdown list 
    switch(choice) {
        case “land":
            $("#land_pollute_field").css("display","block");
            $("#water_pollute_field").css("display","none");
            $("#air_pollute_field").css("display","none");

        break;
        case "air": 
            
            $("#water_pollute_field").css("display","none");
            $("#land_pollute_field").css("display","none");
            $("#air_pollute_field").css("display","block");

        break;
        case "water":
            $("#land_pollute_field").css("display","none");
            $("#water_pollute_field").css("display","block");
            $("#air_pollute_field").css("display","none");
        break;
});

From here its easy to expand the form and manipulate the layout as needed.

Canada 3.0 - Defining Canada's Digital Future

A few months ago at Communitech's Techworking breakfast, I had the opportunity to see Tom Jenkins, Executive Chairman, and CSO of Open Text share a little about Open Text's Enterprise Content Management Solutions, and how they are working with clients to manage their digital assets. 

Jenkins also discussed the Canada 3.0 Forum that will be held in Stratford, Ontario on June 8th & 9th.  My initial reaction was "3.0?  Really?". Many are still trying to figure out 2.0 as others have moved on and begun to argue 3.0... Is 3.0 the semantic web, a mobile web, or digital media? While others argue that technology is moving so quickly right now it is impossible to wrap up in a tidy little box and name it. Regardless of where you stand, Richard McManus' post provides some great insight with additional thoughts on 3.0. 

Independent of which direction 3.0 goes, Canada cannot afford to sit by and wait for its definition. We must find a way to ensure that digital innovation continues, accelerates, and succeeds in our country.  Canada 3.0 is an opportunity to bring the public and private sector together to discuss the future of the web and act now to help shape Canada's digital media strategy.

With speakers from the private sector such as RIM Co-CEO's Mike Lazaridis & Jim Balsillie, Tom Jenkins, Ian Wilson, Konrad W. von Finckenstein of the CRTC, and the Honourable Dalton McGuinty, in additon to stream speakers Austin Hill of Akoha, Anthony Lacavera of Globalive, Mark Relph of Microsoft, Sara Diamond of OCAD, and Alrene Dickinson of Venture Communications and Dragon's Den fame, Canada 3.0 will have a strong cross-representation from government, industry, and education all together.

Codesta is proud to sponsor Canada 3.0 to help foster this discussion and help act on its ideas for the future of the web.

SEO: A Lesson in Website Updates

Codesta recently updated its website and we wanted to share some of our experiences from the process. One of the major reasons we endeavoured to update our website was to increase the breadth of our Search Engine Optimization (SEO) reach. Too often SEO is an after-thought of many projects, and this can prove costly over the short and long term. We've spoken with companies who have undertaken huge and costly website upgrades only to experience significant declines in their online traffic. What's worse, the new site is not "bad" or unfriendly, usually it's quite the opposite. They have done extensive user testing, tweaked their interface to make it that little bit better and wham. Traffic drops. Why? Most often we find it's because they had no previous understanding of their website reach. Their audience is linking to articles deep within their site, and their site visitors are from all over the place. This deep linking drives important search results relevance with Google, the major source of traffic for most websites. If a company upgrades or wholesale changes its site without understanding the impact, it's no small amount of work to fix the damage. To elaborate, I'll use our website and what we did to ensure this didn't happen to us, as an example.

Before upgrading we endeavoured to make sure we didn't "hurt" ourselves as part of the process. A major component of this was understanding how Google currently indexes our site and drives traffic to www.codesta.com. As mentioned in other articles, Codesta is an Agile driven organization, so we included a spike in our first website upgrade sprint, to understand our relationship with Google. What we found surprised us. Our oldest articles on our original website from 2002 still had valid links to it from remote destinations on the Internet. Though a little dusty, these links were still important as linking drives relevancy, and that drives search results traffic. Our first lesson... keep the old articles around.

The second learning through the spike was that Google continues to offer great tools for webmasters to understand their place in the Google Index. A quick visit to Google Webmaster Tools showed us the last time Google had visited our site, all the errors their web crawlers were finding, our index statistics, webpages with the highest Page Rank and much more. It was a full day of leveraging their Diagnostic tools and site linking reports for us to understand what we needed to do. Through everything we found on their site we were able to develop a clear plan to smoothly transition to our new website instance.

Our third learning was the value of a sitemap. Most websites maintain a sitemap somewhere on their site, but the days of end users navigating directly to it are long since over (am I dating myself?). Though sitemaps still have their place in helping with page ranking and end user navigation, they pale in comparison to the value offered by a sitemap.xml file submitted to Google. Google's Googlebot is a much happier camper when provided with an xml file it can digest. Further, the xml sitemap provides a webmaster with the opportunity to explicitly tell Google what pages are a priority to the site operators. Without these values Google assigns each page the same priority which could inadvertently direct traffic to less important areas of your site. We also found that the presence of the sitemap.xml consistently accelerated the Google verification and crawling process. If a sitemap.xml existed we always found we were visited the same day and able to make changes very quickly.

Our last key learning was from our analysis of our Google Analytics (GA) Reporting. We have been GA users for the last 2+ years and have been very happy with the information we've gathered. We've been happy to learn that our blog has been productively generating traffic since we launched it 2 years ago. We also knew that we wanted to lower our Bounce Rate to provide visitors with a better experience. One nifty new tool we were happy to play with was the Google Website Optimizer. It was a critical tool that helped us understand how people were navigating our site. It enables you to develop scenarios, called Experiments, where you can test where you place content and how users respond to it. We found one key learning from this process. We had WAY too much content on our old homepage. Visitors weren't staying to read everything we shared. The new site is borderline sparse, but we like it that way. We were pleasantly surprised to see users frequently navigating to our Contact Us page, a positive reinforcement of our call to action.

Once our Google-centric tinkering was done, we wanted to perform one last validation step. We wanted to make sure that we weren't just Google friendly, but had implemented a web friendly site for all users. There are great resources and examples on the Internet of sites that have spent considerable time and energy making sure their site works for all users. Mobile devices, the visually impaired, or individual preferences on font sizes require consideration and we wanted to make sure we were doing our best. A personal favourite of mine for benchmarking our deployment is the Canadian National Institute for the Blind (cnib.ca). A quick look at their page header reveals they've considered many aspects of their user base. From contrast levels to screen readers they need to carefully consider their users. Codesta's long term goal is to provide the same level of access to our site, and our latest site release is a first step in that direction. To meet this goal we leveraged some well known SEO testing tools to ensure we'd done our job. Some of the tools we used (and liked) include:

1. Electrum's SortSite SEO Testing Tool - SortSite does an excellent job providing a robust evaluation tool which makes you want to upgrade to their full version after just one use! Their tools provide information on coding errors, accessibility issues and compatibility with major search engines. The level of detail they provide is excellent and the interface makes clicking through to detailed data very easy.
2. Aaron Wall's SEOBook and SEO Toolbar for Firefox - Built right into your browser, the SEO Toolbar does a great job of providing data while you review a site.
3. W3C Markup Validation Service - Not to be outdone, the W3C has provided many free tools you can use to test your site. 
4. SEOmoz - Lots of useful info, and plenty of data and tools to digest.
5. Yahoo UI Library - Last, but definitely not least, Yahoo maintains a large UI library for developers to maintain. It's a great resource for all UI developers out there, and contains many practical tools to help develop a great, SEO friendly website.

Launch! Finally...

Away we go... We're relieved and happy with the results of our learning exercise. Much of what I've shared is very well known, documented and free on the Internet. Despite that fact, we felt we really needed to share because there are still so many people doing it wrong. We understand no one is perfect, and we really hope this article helps. With that, one last piece of advice...

Continue using the SEO tools we've described and the Google Webmaster resources to check-in on your site after launch. This is particularly true for Google immediately after launch, and roughly a week or so afterwards. In that timeframe Google will have visited your site, indexed results and started sharing that information with the world. You can consume all of this reporting via the Google Webmaster tools, and help them do a better job of promoting your website. This is your window of opportunity to "limit the damage" and catch errors before they're widely distributed across the Internet. Don't delay, and you'll save yourself one more headache.

Drupal 5 vs Drupal 6 - Head to Head Comparison

    Drupal 6 has been in the wild for about a year now, and Drupal 7 is on the way. This article will take a look at Drupal 6 from the point of view of a Drupal 5 user, especially with regards to whether it makes sense to update your Drupal 5 site to Drupal 6.

    First, we can take a look at the new features that have been included in the Drupal 6 release. One of the major features is a new installer to aid the technically challenged in setting up their Drupal website, which could be a difficult task with Drupal 5. The installer can now check for “Clean Url” support (which the vast majority of website administers will want to take advantage of), and enable this feature if the support is available. The installer also sets the “Files” directory during installation, and multi-language support is available from the outset.

    Another improvement is the drag-and-drop administration that has taken the place of the unwieldy weighting system that was used throughout Drupal 5. This interface is available for menu items, forums, taxonomy terms, uploaded files, input formats, profile fields and more.

    Other improvements include an updated theming system to improve the ease of custom theming, password strength checking, granular permissions and other small changes.

    Drupal 6 also boasts several new core modules, including OpenID, Update Status and Actions plus Triggers. All of these, however, are contributed modules that were available for Drupal 5, so this is not a major improvement.

    One of the features touted in the release notes for Drupal 6 is optimized code that “improves performance for both authenticated and anonymous users”. However, several benchmark tests have concluded that Drupal 5 is consistently faster that Drupal 6, using core modules only. This may be a strong consideration for those considering upgrading to Drupal 6, especially since Drupal 5 is not the most efficient beast itself.

2bits.com Drupal 5 vs. 6 Comparison
Vision Media Benchmark Testing
2bits.com Drupal Getting Slower Article

Upgrade Considerations

    Now that we have taken a look at the new features of Drupal 6, let’s look at whether it makes sense to update your Drupal 5.x website to Drupal 6. This choice will depend entirely upon what sort of website you are running with Drupal 5. If you have a simple website with a small number of contributed modules, then upgrading may be an easy task. If you have a more complex website with a larger number of contributed modules and customizations then upgrading starts to become more daunting.

    Not only will you have to update Drupal core, you will also need to update each of the contributed modules that you are using. This task is made more difficult by the fact that there may not be a Drupal 6 compatible version of each of your web site's modules. Even if the module has a 6.x compatible version, that version may not be as stable as the 5.x version. This necessitates the extra step of checking the issue queue on Drupal.org for the 6.x version of each module to ensure that people aren’t having problems with the module on the 6.x branch.

    Given that updating your Drupal 5 site may prove to be difficult, does the new feature set necessitate an upgrade? From the point of view of a Drupal 5 user (myself), there are no “must-have” features in the Drupal 6 release that would balance out the difficulty of an upgrade. The drag-and-drop interface is the one feature that sticks out as an important, probably due to the fact that the old weighting system was so cumbersome.

Conclusions

    Drupal 6 is an incremental improvement over Drupal 5. Although the new feature set is fairly large, the difficulty of upgrading your Drupal 5 site overwhelms any small improvements that you may get by performing the update. In addition, if you have a Drupal 5 site with a custom theme you may want to take a very close look at the theming changes before even considering an upgrade.

    If you don’t currently have a Drupal 5 website, then you will want to start with Drupal 6, as it does seem to be an improvement over the previous version. The only reason not to use Drupal 6 for a brand new website would be if a module you need exists for Drupal 5 only, and has not been ported to 6. This issue may well occur – there are 75 pages of contributed modules for Drupal 5, and only 46 for Drupal 6.

Drupal Content Types - Part 1

Drupal supports many different ways to create and manage content. In Drupal, all content types derive from something that is called a node, which just provides you with a title and body field. In fact if you do not know what a node is then you shouldn’t read on any further and get more acquainted with Drupal before delving into more complex topics.

As you probably know, Drupal comes with several content types to begin with, namely the story and page types. These are useful if you are building a simple website with a blog and some extraneous pages. Over the past several months at Codesta we have worked on several different projects which required the use of our own custom content types. We discovered that Drupal can be a maze when it comes to selecting a suitable content type which extends the basic Drupal ‘node’ concept. We have dabbled using CCK (Content Construction Kit) to create content types programmatically and would like to share some of our findings. In this Part I will provide a general overview of the variety of content and node types, including some of the high level benefits and drawbacks to each method. In future articles I will flesh out the details of each. You should have a good understanding of the purpose and function of the CCK to get the most out of this overview.

Types of Content Implementations:

a) Create your own.
b) Use the CCK to create your content types.
c) Using a mixture of CCK and your own modules.
d) Programmatically create a field for use in a CCK.

A) Create your own module.

Overview:

This method involves using Drupal’s hook system to tell the framework that you’ve added a new content type. You need to take care of creating any new tables in the database that the content type will use to store data for the content type.

Discussion:

This method gives you the most flexibility, but it also usually requires more time to implement. You have full control over the database schema and the relationships they have over other content types or tables. In light of this you are responsible for the validation of input, creation of the input forms, and the database queries to update the database.

B) Use the CCK to create your content types.

Overview:

Through the CCK module you can create new content types and connect them to views quickly and readily.

Discussion:

Table structures are automatically created so there is no need to worry about database management. Integration with views makes it easy to display lists of content in many different ways. The CCK also takes care of validation of form elements for you.

C) Using a mixture of both CCK and your own modules.

Overview:

Sometimes it may be the best option to integrate a node module with a CCK content type. This gives you the benefit of CCK where you do not need to worry about the forms and database queries while at the same time leverages the flexibility of creating your own module. This involves configuring your content type through CCK and creating a module with the same name as the content type.

Discussion:

We found it necessary when working with client requests for dynamically generated forms in Drupal to use a mixture of CCK and own node module. This required the AHAH module (which provides the framework to create new form elements on the client side while talking to the Drupal back end to keep track of these new elements). The reason we didn’t implement the content type in a custom node module was that only one field needed to be dynamically populated. All remaining fields were good as they were, so we wanted to leverage CCK to rapidly develop the content type.

D) Programmatically creating a new field that can be then used through the CCK module interface.

Overview:

The CCK allows you to create reusable ‘field types’ that can be used with any CCK content type. For example, perhaps you would want a field in your content type that stores the graduation date and major for a student. You would like these items to be grouped together, and you already have a CCK content type built for students. You can use the CCK API to create the new field and add to any it to you student CCK content type.

Discussion:

In our case we needed an enhanced predictive text box field that queried several different tables. We also knew that there would be several slight variations of the same field. We already had a CCK content type and it did not make sense to write a node module for it, so we choose to write an enhanced predictive textbox field to be used with CCK. Now we can use this field in the future in other CCK content types without writing a single line of code.

We have briefly discussed several different methods for extending the content that Drupal natively supports, in the coming weeks we will go into more depth with each method.

Thoughts on the Google App Engine

Google threw their hat into the cloud computing arena in April 2008 with the introduction of the Google App Engine (GAE).  The GAE allows developers to create and deploy applications using Google's infrastructure, leveraging their extensive expertise in the domain of scaling BIG. The GAE is still a beta product, given their track record for extended beta periods (gmail anyone?) it may continue to be for some time. It's free to create and deploy applications supporting approximately 5 million visitors per month. If you're not scared by the beta status, willing to invest some time learning the "Datastore", and looking for a platform to develop your next web app, GAE is worth exploring.

GAE is intended to compete with offerings like Amazon's EC2, FlexiScale, and Microsoft's new Azure. However, GAE differs in one very significant way, it's not a virtualized server hosted in the cloud. Rather, it's a tightly walled platform which forces you to develop using Python and the Datastore, with Django templating bundled in. To build an application using GAE you have to give up virtually all of the flexibility that competing services offer. On its surface that doesn't sound like a great deal, but Google makes it a trade off worth considering. With flexibility comes configuration and management. To pick on EC2 for a second: you'll need to get an OS, web server, web application framework, and database server up and running before you can do any development; You'll have to manage and maintain each of these, worrying about downed services and backups; When it comes time to scale you'll have to ensure that your software architecture is scaleable; Plus you'll need the expertise to scale up your server infrastructure, not a trivial feat.  

Google's GAE exchanges flexibility (and the headaches that come with it) for one click deployment, near zero maintenance and management, and virtually transparent scaling. The promise being that using GAE you'll be able to build highly trafficked web applications without having to worry about designing, managing, and scaling your back-end infrastructure. You'll be able to do this at zero cost for 'development' applications. Once GAE reaches general release, you'll be able to pay for additions to your quotas (more later) as well as 'production' status and support.

The Good:

It really is as easy as one click deployment. If you're familiar with Python and HTML you can go from nothing, no infrastructure and no code, to the foundation for virtually any web application in a matter of hours. In over a decade of building applications for the web I've never had an experience as easy as this.

Google is the best in the business when it comes to building out high availability scaleable infrastructure. They've also done a tremendous job of abstracting the intricacies of developing for multi-node parallel systems away from developers with BigTable and MapReduce. For the first time GAE exposes some of this to external developers. It's a beautiful thing.  

You'll quickly learn that the Datastore is very different from RDMSs you may have worked with in the past. GAE uses what they're calling the Datastore, which is really an implementation of BigTable, Google's proprietary database. BigTable doesn't run on a single computer or cluster of computers, rather it can span tens of thousands of disks on thousands of servers. You create data models as Python classes. Instances of these classes, called entities, are synonymous with database records. Once created these entities are stored and moved around by BigTable to account for hotspots and increasing storage needs.

Because of this distributed architecture, what works in a traditional RDMS systems like MySQL or MS-SQL doesn't necessarily translate well to the Datastore. You generally wouldn't think twice about performing a count, sum, or avg operation on records in a traditional RDMS. Due to the distributed nature of the the Datastore it can be very costly perform the entity fetching necessary for these. Pre-computation is paramount where possible. Joins are also expensive in the context of distributed systems and as such, are not handled in a traditional fashion, although relationships between data entities are possible. A basic rule of thumb is that storage is cheap, so a lot of the rules of normalization that you're used to get thrown out the window - don't feel bad about data duplication.

The Bad (aka the limitations):

There are a number of limits which Google has placed on use of the GAE.  These include:

* No long running processes

* A maximum of 1000 results returned from any query

* Read only access to the file system

* No scheduled activities

* No easy way to perform one off maintenance on your data

* No official support for backing up your data.

* A number quotas including bandwidth, storage, sent email, and CPU utilization 

Given these limitations, there are a number of applications which are just not suitable for GAE. For instance, if you require regular maintenance to occur on your data there is no easy way to achieve this; if you require any activity to occur on a scheduled basis, the only way to make this happen is to 'ping' your application from an external source; and, if you wanted your application to perform some activity like crawling the web or performing server side image manipulation then you'll quickly run up against the long running processes limitation.

My Experience:

I sat down to build a basic blogging application. I had zero familiarity with the platform and only a passing familiarity with Python. It only took a few hours to complete the getting started tutorial and get the blogging app built and deployed. I mentioned earlier how easy I found the process to be, I'll reiterate that again. It was the easiest application construction that I've ever done. By far.

Conclusion:

The platform isn't perfect and there are many limitations which might exclude it from being a candidate for your project. It's also amazingly simple, well documented, and easy to use. My biggest fear if deciding to use GAE for a serious project would be being locked in to the Google platform. You can't just pick up and deploy elsewhere. The Datastore alone limits you to a continued relationship with Google. Then again if you trust their "Don't be evil" slogan they might not be such a bad partner to get into bed with. That being said I wouldn't recommend using the platform for anything besides pet projects until the it's post-beta costs are known. Still, it was the most enjoyable development experience I've had in a long time. I look forward to future encounters with the GAE.

Thoughts on PHP Frameworks - Part 2 - Zend Framework

Over the course of my time here at Codesta I have been exposed to various platforms and technologies. On many occasions I have had to work on a PHP application that already had its own custom framework in place. One project in particular scarred me horribly; the Model-View-Controller (MVC) logic was muddled together and a pain to work with. Overall the framework the application was built on was a total mess.

This article is part two in my quest to see if there are any good PHP Frameworks out there that can compete with established frameworks like J2EE and .Net. In the first part (found here) I reviewed the CakePHP framework. Now I will be documenting my experience in getting a simple Zend application running.

Zend

Zend is one of the oldest Frameworks available for PHP; it is a part of the Zend Platform by Zend Technologies which specializes in PHP development. I attempted to focus in on the Zend Framework itself but at the same time I downloaded and tried out the Zend Studio. It should be noted that a few of their products (like the Zend Studio) need to be purchased after the trial period is up.

Install and Setup

The Zend Studio installation is straightforward and you can start working with it right away. The Zend Framework itself is a library of PHP classes that you include in any PHP project to use. Zend Studio has the ability to create a project that has the proper directory stucture and the Zend Framework code setup for you already.

First Application

With everything installed and ready to go I was set to try building my first application. As with my last blog article I decided the best way to test out the framework would be to follow a tutorial and then see how easily it would be to modify the resulting application.

I downloaded version 1.5.1 of the Zend Framework and quickly realized how new the Framework was. It was very difficult to find any tutorials that were valid for the version that I had. It seems that the Zend Framework had only reached version 1 very recently and there wasn't much community documentation available yet. There were lots of resources and tutorials on Zend in general; just not the version I had.

After praying to the Google overlords for a bit I came across a good webcast here. The tutorial also outlined how to install and run the Zend Framework without any other Zend tools as well.

Some highlights from the webcast were:

• PHP exceptions!
• The Zend Controller and View magic worked as expected out of the box
• Demo on how to send custom data to the Views through the Controllers
• Demo of the Zend Reference Guide (found here)

Mod Rewrite Problems

Although it seemed as though everything on Zend was working fine on my machine it wasn't the case. Most of my work to this point had involved working with changes to the Index View and Controller. When I started adding in other Controllers, I realized I could not get through to them via the URL like I should be able too. I tracked the problem down to a conflict between rewrite rules in my Apache httpd.conf and the .htaccess rewrite rules that come with Zend.

It turns out that a number of people have run into similar conflicts with the rewrite rules; a Google Search reveals how other people have tried to handle the problem.  As an interesting side note, at Codesta we have run into identical problems with rewrite rule conflicts when using Drupal. If you are using Zend and a have complex Apache configuration, expect to have to do some work to get Zend to work properly.

Here is how I tried to work out the problem.

The format for a typical URL in Zend is [base_url]/Controller/View/. To get to the index page you should be able to use [base_url]/ and that should take you to the IndexController with the Index View.

Up to this point I had been using http://localhost:8080/ptp/index.php to test out my changes, but when I tried something like http://localhost:8080/ptp/ I got Apache permission errors. I had setup my own rewrite rules in Apache to point to the Zend Project like this:

RewriteCond %{REQUEST_FILENAME}   ^/ptp/(.*)
RewriteRule ^/ptp/(.*)   "C:/Zend/workspaces/DefaultWorkspace
                          /Zend_Test/html/$1" [L]

Which would forward any requests to my Zend install.

The problem was that the Zend framework has its own .htaccess file to provide setup information to Apache. The rewrite rule I had wrote broke how index.php redirects the browser to the proper Controller and View in Zend.

If you had to integrate a Zend Project as a component to a larger application that relied on Apache you could try this rewrite rule in the http.conf file:

RewriteCond %{REQUEST_FILENAME}   ^/ptp/(.*)
RewriteRule ^/ptp/(.*)   "C:/Zend/workspaces/DefaultWorkspace
                         /Zend_Test/html/index.php" [L]

However when I attempted this configuration change I got the following exception thrown by Zend:

Fatal error: Uncaught exception 'Zend_Controller_Dispatcher_
Exception' with message 'Invalid controller specified (ptp)' 
in C:\Zend\workspaces\DefaultWorkspace\Zend_Test\library
\Zend\Controller\Dispatcher\Standard.php:249 
Stack trace: 
#0 C:\Zend\workspaces\DefaultWorkspace\Zend_Test\library
\Zend\Controller\Front.php(914): 
Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend
_Controller_Request_Http), Object(Zend_Controller_Response
_Http)) 
#1 C:\Zend\workspaces\DefaultWorkspace\Zend_Test\html\
index.php(58): Zend_Controller_Front->dispatch() 
#2 {main} thrown in
C:\Zend\workspaces\DefaultWorkspace\Zend_Test\library\Zend
\Controller\Dispatcher\Standard.php 
on line 249

The problem appears to be that the Zend project is parsing "ptp" as a controller. I stopped my attempts at ramming the Zend project into my Apache configuration and used a fresh configuration file for Zend testing. If you do this, just make sure the DocumentRoot value to point to your project.

If you want to try to solve the rewrite problem, I would look to see if there was a base URL setting anywhere you could set in Zend so that it knows where to look for the Controllers and Views in the URL. The Zend exceptions were very helpful in narrowing down the problems in this case!

Creating CRUD Pages

After finishing up the simple application I noticed that there was no mention of how to do anything on the Model Side of the Zend Framework. Considering that the Model component is a vital part of any application I thought it would be important to research.

After a bit of digging around I realized that it gets much worse. At first I was unable to find any tutorials or guidelines on setting up Models in the version of Zend that I was using. I was able to find a few tutorials for older versions of Zend; but not the exact version I was working with. I ended up thrashing around trying to figure out the differences between versions but I had a lot of trouble just getting the bootstrapping and conventions right.

Finally I stumbled onto http://www.akrabat.com and found tons of resources on Zend. Most importantly there was a tutorial (found here) that outlined an application that would store a CD collection using Zend. I ran into some configuration problems with PDO_MYSQL. The tutorial specifies PDO_MYSQL as the database adapter in the config.ini file. It took a little bit of work before I realized that PDO_MYSQL was not a Zend component but rather an extension of PHP to use MySQL. I found out how to install the extension here and I was back to work.

The rest of the tutorial was straightforward and worked for me on the first try.

Modifications

While following the tutorial, I setup the Album Controller on its own. Rather than having it as the default Index Controller I created a new controller that I tested by using the url: http://localhost:8080/html/album/.

By setting up the Album as its own controller, I had to modify all redirects in the Album Controller to point to the proper location.

For example whenever the tutorial used:

$this->_redirect('/'); 

I had to insert

$this->_redirect('/album/'); 

Then all index actions pointed back to the Album Controller rather than the Index Controller. I wanted to do more modifications to my application as well as try out the authentication tutorial at akrabat.com but time constraints forced me to abandon those tasks.

Zend Studio

At first I used the trial version of Zend Studio to see what it was like. More specifically I wanted to see how the debugger worked. As I mentioned earlier it is possible with Zend Studio to create a Zend Project with the framework and directory structure pre-defined for me. So I started writing my test application with Zend Studio.

However, after a bit of work I started going back to using Eclipse 3.1 on my Zend Application. I was never able to get away from the fact that Zend Studio is based on Eclipse and that I could be just as productive using Eclipse. Due to other projects, my trial license of Zend Studio expired before I really got into my Zend Application and I was never able to try out the debugger.

Overall, I felt that what I could do with Zend Studio, I could also do with Eclipse. Although the Zend debugger is built into Zend Studio; there are other PHP debuggers that work with Eclipse like PHP DBG.

Conclusions

In comparison with CakePHP, the Zend Framework is more mature and complete. There is more of a community and the framework is more complete. I felt that I was hampered a bit because I was using a newer version of Zend and did not have access to more tutorials and guides online. Also, I was not convinced that the Zend tools are worth the investment. Like CakePHP I was impressed at the level of work that had been invested into Zend and it is a viable, quick method to get a PHP application started.

Knit 1, Perl 2...

I Know As Much About Perl Programming As I Do Knitting And That Isn't a Bad Thing!

I have been doing QA for 7 years now and believe I have the essential skills that make for a solid QA analyst, however I do not possess the skills that would make me a basic developer. I feel that if I gain some knowledge of the technology our developers use I can better represent the customer in the development process. Codesta is very supportive of this in the way that the company grants us personal time to learn new technology and skills.  I have made several attempts to go outside my skill set and role to bring something new to my QA tool shed. While I haven’t brought back the tool I intentionally set out to learn, I have brought back several other tools that have benefited both my testing skills and my knowledge of technologies our development team uses.

My journey into the realm outside my QA skill set started when I was in-between projects and I wanted to learn anything about a platform our development team uses. This brought me to PHP (www.php.net). A colleague tells me “PHP is easy. It shouldn't be a problem for you.”. Ha! I gave it a valiant effort but quickly realized how deep I was when I didn't understand some of the basics of HTML and was thrashing about in PHP land. What I learned here was that I needed to learn something basic about coding and that PHP needed more of a knowledge base and comprehension than I possessed.

PHP brought me to HTML. Now this was something that I thought I was grasping. I found http://www.w3schools.com/html/html_intro.asp and I was off. I was learning about basic tags, links, images, and tables to name a few. I definitely felt like I was getting the hang of it even going so far as to use my new found knowledge of HTML tables to help with the creation of automated test suites in Selenium. More on the Selenium side of the story in a bit. So there I was making HTML tables with all my Selenium test cases in them to create a giant Selenium test suite. I was on fire. I took the HTML quiz at http://www.w3schools.com/quiztest/quiztest.asp?qtest=HTML and got a 100%! I then had to put my HTML mastery on hold as a project came up that needed my attention. For the purpose of this blog I went back and took the HTML quiz again. I got a 65%. The lesson I learned is "use it or loose it".  I need to use my newly acquired skill often or risk forgetting key points.

As mentioned above Selenium (http://selenium.openqa.org/) is an automated web testing tool that has become an invaluable part of my QA testing tool shed. You can use basic HTML to write Selenium tests or for the non-coding inclined Selenium has a plug-in for FireFox called Selenium IDE (http://selenium-ide.openqa.org/). This plug-in basically lets you click a button to start recording user navigation around a website or application and “Play” those steps back letting you know if a test passes or fails. I have been using this testing format extensively for acceptance and bug regression tests. Acceptance tests on a website, for example, would be running through all pages verifying site navigation and external links are functioning correctly. This cuts down the time it takes me to do this manually and frees me up to look deeper into the product to find possible issues. As for bug regression, I find a bug and log it along with a Selenium test case. I do this so the developer can run the test which contains the specific steps I used to find the bug and they can not say a bug is fixed until the Selenium test passes. I learned that a more “user friendly” technology can help you learn a more difficult technology.

In my latest attempt to bring something new to my tool shed, I was tasked with learning how to use Perl (www.perl.org) to parse Apache (www.apache.org) access_logs to create reports of website load times. I bought a book to help me learn Perl and give me some insight. I proceeded to Google my fingers off looking for someone who had done it before. What I found was many people had done it before but for some reason their Perl scripts would not work for me. I started looking for help, asking my peers for assistance and getting some answers that I just couldn't comprehend. I realized I was back in the same place as I was with PHP. I felt like I should really be able to understand it but I just couldn't. In the process, however, I became comfortable using the command line and I have a better understanding of what code is supposed to do. Despite my struggles, I believe that these are steps in the right direction for me.

My QA tool shed has a lot of room for all kinds of tools. Some I am not ready for and some may never make it into the shed. This won’t stop me from trying Perl or PHP again. Maybe something else I embark on will tie into either of them. In the case of Selenium IDE, it brought me back to HTML and the little bit I was able to learn there has helped me tremendously. All in all, I learned that even if you set out to learn something that is outside of your role and fail you probably picked up something useful along the way.