Wednesday, August 10, 2011

Javascript Unit Testing Environment

Ready to kick your Javascript Unit Testing up to the next level? Or even to the very first level?

Javascript Unit Testing suffers from lack of standardization. There is no universal 'JUnit' or 'PHPUnit' to 'rule them all'. There are lots of very good Javascript testing frameworks available, especially YUI3's test framework.

YUI3's framework provides the foundation for writing unit tests, collecting them into test cases and suites, and even mocking objects. And that's great. But we can do more, and do it easier!

The Javascript Unit Test Environment (JUTE) combines the YUI3 testing framework, automatic dynamic code coverage, a web UI, a command line interface, seamless client- and server-side Javascript testing and 3 testing back ends (Capture, Selenium, and V8) in one easy to setup and use package. This is all provided with very little setup by you - just like we all like it!

Anatomy of a client-side Javascript Unit Test

First let's look at what a simple client-side Javascript Unit Test looks like. Here's some simple code we want to test (not JUTE does NOT require that code you want to test be written using YUI3!).

In 'mySum.js':

function mySum() {
var result = 0, i = 0;

if (typeof arguments[0] == 'object' && arguments[0].length) {
for (;i < arguments[0].length; i++) {
result += arguments[0][i];
}
} else {
for (;i < arguments.length; i++) {
result += arguments[i];
}
}
return result;
}

A slightly bizarre 'sum' function - does it work? Let's write a test for it using JUTE! JUTE builds on top of the YUI3 test module, so we need to use that to write our tests. Again, the code you are testing does NOT need to use or have anything to do with YUI3 (although it can!) - only the TESTING code needs to utilize the YUI3 test stuff - which is good test stuff to use regardless!

In 'testMySum.js':

YUI({

logInclude: { TestRunner: true },
gallery: 'gallery-2011.06.22-20-13'
}).use('gallery-jute', function(Y) {

var suite = new Y.Test.Suite('mySum');
suite.add(new Y.Test.Case({
name:'simple sums',
testTwoNumbers: function() {
Y.Assert.areEqual(mySum(5, 5), 10);
},

testArray: function() {
Y.Assert.areEqual(mySum([5, 5]), 10);
}

}));
Y.Test.Runner.add(suite);
Y.UnitTest.go();
});

So those are two reasonable tests for our 'mySum' function. Now we need an HTML file to pull it all together.

In 'testMySum.html':

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
</head>
<body class="yui3-skin-sam">
<div id="log" />
<script src="http://yui.yahooapis.com/3.3.0/build/yui/yui-min.js"></script>
<script src="mySum.js?coverage=1"></script>
<script src="testMySum.js"></script>
</body>
</html>

OK so we've got all we need for a basic client-side Javascript Unit Test. We have what we want to test (mySum.js), our test code (testMySum.js) and finally an HTML file to pull it all together (testMySum.html). We could be done here, run the tests, hope it all passes and move on.

The only things different here than a vanilla unit test are the inclusion of the 'gallery-jute' module (which also pulls in YUI3's 'test' module for you) in testMySum.js AND node the '?do_coverage=1' query string I snuck in testMySum.html on the mySum.js file! This will transparently tell JUTE that IF you want to run these tests with code coverage THIS is the file we want code coverage for - which, surprise surprise, is the file we're testing.

Anatomy of a server-side Javascript Unit Test

Our 'mySum' function can clearly just as easily be run on the server side using NodeJS. What would a unit test for server-side Javascript look like?

First off our mySum.js file will change slightly because we want to utilize the NodeJS 'require' feature.

myServerSum.js:

module.exports = {
mySum: function() {
var result = 0, i = 0;

if (typeof arguments[0] == 'object' && arguments[0].length) {
for (;i < arguments[0].length; i++) {
result += arguments[0][i];
}
} else {
for (;i < arguments.length; i++) {
result += arguments[i];
}
}
return result;
}
};

Here we just wrapped our 'mySum' function into a module.exports declaration. Now our tests will change slightly to use 'require' instead of relying on an HTML file to link everything together.

In 'testServerMySum.js:

YUI({
logInclude: { TestRunner: true },

}).use('jute', function(Y) {

var suite = new Y.Test.Suite('mySum'),
mySum = require('myServerSum', true).mySum;
suite.add(new Y.Test.Case({
name:'simple sums',
testTwoNumbers: function() {
Y.Assert.areEqual(mySum(5, 5), 10);
},

testArray: function() {
Y.Assert.areEqual(mySum([5, 5]), 10);
}

}));
Y.Test.Runner.add(suite);
Y.UnitTest.go();
});

No not a lot of (visible!) magic here - we just 'require' the myServerSum javascript & pull out the 'mySum' function. Besides there being NO HTML file I've snuck something else in - note:

mySum = require('myServerSum', true).mySum;

In 'testServerMySum.js'! Similarly to '?do_coverage=1' in the client-side example, this tells JUTE that IF we run unit tests wanting code coverage THIS is the file to apply code coverage to. And, surprise surprise, this is the file we are testing.

Ready To Rumble

At this point JUTE now gives you the flexibility to run your tests via a web UI or a command-line interface. JUTE will persist not only your unit test output in JUnit XML format, but will also automatically generate pretty HTML for code coverage AND give you the option of running these tests in either Captured browsers, on a Selenium RC or Grid host, or directly thru V8. Of course the NodeJS version of mySum can ONLY be run thru the V8 backend. However in this case the client-side version CAN be run thru any of the 3 backends.

So you've actually done all the hard parts already: for client-side Javascript testing you have the file you want to test, the file with your tests in it, and an HTML file tying them all together. For server-side Javascript you've got the file you want to test and file containing all of your test code.

Now lets sit back and run it all thru JUTE and start living the good life!

Installing JUTE

First you need to install the thing - JUTE requires only the current versions of NodeJS (0.4.10) and npm (1.x). Simply:

% npm install jute -g

Starting, stopping, and restarting JUTE is equally easy:

% npm start jute
% npm stop jute
% npm restart jute

Basic Configuration

JUTE is a standalone webserver so before you start it you need to tell it where your document root is, where your test files are, and where it should put its output. You can also give JUTE another port to run on and some other configuration. Regardless all configuration is accessible via npm config variables:

% npm config set jute:docRoot /var/htmlroot
% npm config set jute:testDir test
% npm config set jute:outputDir output
% npm config set jute:port 8080

SO JUTE will look for all of your *.html 'glue' test file in docRoot + testDir (/var/htmlroot/test) in this example and output test results and code coverage information into docRoot + outputDir (/var/htmlroot/output) in this case. AND JUTE will listen on port 8080.

If you make ANY changes to JUTE variables you MUST restart JUTE for your changes to take effect:

% npm restart jute

Now that JUTE is configured you are ready to run some tests the JUTE way!

Backends

Before we run our tests let's quickly divert and discuss the testing 'backends' available with JUTE.

Capture

Capture mode runs tests on all 'captured' browsers. A 'captured' browser is simply one that is currently pointing to http://<JUTE_SERVER>:<JUTE_PORT>/ All such browsers (desktop, mobile, whatever) are 'captured' by JUTE and any submitted 'capture' unit tests will run on ALL currently captured browsers in parallel.

Selenium

JUTE plays nicely with Selenium RC and Grid. The Selenium back end will ship all requested tests to a Selenium RC or Selenium Grid instance. Just specify the Selenium browser specification (e.g. '*firefox') and off your tests go.
The Selenium back end is only available via JUTE's command-line interface.

V8

Some client-side Javascript and of course and all server-side Javascript will run through the V8 back end. The V8 back end is only available via JUTE's command-line interface.

Running Your Tests

Hey we are just about ready to run our awesome unit tests the JUTE way! You have two choices to kick off your unit tests, via the WebUI or the command-line interface. Note ONLY Capture tests can be run via the web UI - Selenium and V8 back ends are only accessible via JUTE's command-line interface. See below for details!

WebUI

JUTE comes with a snazzy webui available here:

http://<JUTE_SERVER>:<JUTE_PORT>/

This will bring up a lot of information - the most useful being the list of unit tests available to be run assuming you set up docRoot and testDir correctly!
From the WebUI you are able to run any/all tests in Capture mode. You can also see output from ALL modes in the 'results' pane on the right.

Command-Line Interface

This is where things get especially interesting. From the command-line interface you can run tests in any of the 3 back ends with code coverage or not. Let's look at some examples! NOTE!!!!!! All testfiles are specified RELATIVE to <docRoot>/<testDir>!!

% jute_submit_test --test myTestSum.html

This is as simple as it gets - submits that 1 HTML file to be unit tested in capture mode.

% jute_submit_test --test myTestSum.html --v8

Same thing BUT use the V8 backend.

% jute_submit_test --test myTestSum.html --sel_host our.selenium.rc.com -sel_browser '*firefox'

How about multiple tests?

% jute_submit_test --test myTestSum.html --test myOtherTest.html

Again added the '--v8' or '--sel_host' flags for those back ends.

How about code coverage?

% jute_submit_test --test myTestSum.html?do_coverage=1 --v8

Simply stick the '?do_coverage=1' querystring at the end any test to enable code coverage.

Now JUTE will go off and run you test(s) using the specified back end and will dump output into <docRoot>/<outputDir>. JUTE will also generate pretty HTML for code coverage if requested - all available for viewing via the webui - EVEN Selenium and V8 tests!

More Information

I hope you've seen writing Javascript unit tests CAN be fun AND profitable! More examples are available at the JUTE homepage - so check it out!

Don't let your client- or server-side Javascript get out of control! JUTE makes it easy and seamless to get started or kick up to the next level your Javascript Unit Testing today!

% npm install jute -g

FTW!

Monday, July 11, 2011

Adding Facebook Chat To Your NodeJS Site

Had a fun time adding Facebook chat to a NodeJS-backed site - actual real Facebook chat. That means people on your very own website can chat with their Facebook friends while on your site.

Our good friends at Facebook were kind enough to provide API hooks for authenticating into Facebook Chat using OAuth - SO your Facebook users just need to authorize your app and they can chat thru Facebook on your site. Unfortunately no word yet about opening up Video Chat - hopefully that'll come soon.

The basic idea is this:

FIRST Users approve your app/website with chat permissions with Facebook - your user just needs to do this once. Then in a loop:

User visits your webpage
Your page includes a Javascript XMPP client which can handle Facebook OAuth authentication (and the XMPP protocol in general)
When your user is ready to chat fire up the XMPP client - which is configured to talk to back to your app which will proxy that request to your locally running BOSH server which then proxies the requests off to Facebook
Profit!

What He Say?

If yer new to this stuff as I was here are some basic terms:

XMPP - a chat protocol that is the standardized version of the 'Jabber' protocol. XMPP is your friend.

BOSH - Bidirectional-streams Over Synchronous HTTP (BOSH) Good god don't actually read that - it's yer standard 'how the monkey do we do bi-directional async message passing over HTTP?' - the gory details do not concern us - it works. Like socket.io for web-based XMPP clients.

It is expected your know your way around authenticating your webapp (or desktop app) via Facebook. Here is where to get started.

User Allows your website to host FB Chat

This is your standard FB authentication as documented here - just make sure to request the 'xmpp_login' permission. I would also recommend getting the 'offline_access' permission while yer at it.

Install a BOSH server

I'm going to use 'node-xmpp-bosh' in keeping with the NodeJS theme BUT any old BOSH server will do (here's one written in Java if that's how you roll). This guy will route XMPP messages between your Javascript XMPP client back and forth to Facebook (or any other XMPP server). This thing can be treated just as a service that needs to be started once & forgotten.

% npm install node-xmpp-bosh

You do NOT need to edit its configuration file in /etc/bosh.conf.js - in fact you probably should not change anything in there  Frankly to us this thing is just a black box.  Here are the most important configuration bits:

port: 5280,
  host: '0.0.0.0',
  path: /^\/http-bind\/$/

So it's going to listen on localhost on port 5280 and expects requests of the form /http-bind/* - basically nothing you need to worry about.

You can then either run it in 'standalone' mode - you can start it when your webserver boots just like any other service or start it from the comand line or from within your NodeJS code.

Note is does NOT need to run as root - it's basically a webserver that listens on port 5280 and that's about it.

For grins here's how to start it from within your NodeJS code:

var nxb     = require('node-xmpp-bosh'),
    bosh_server = nxb.start({
        port: 5280,
        no_tls_domains: [ 'chat.facebook.com' ]
    });

Note Facebook's chat servers do NOT support TLS - so if you are running any other BOSH server ensure where ever the config option is to turn TLS off it's turned off for 'chat.facebook.com' .

Client-side XMPP Library

We need a JS library that supports Facebook's OAuth XMPP authentication - Hey! Here's one Now! JSJaC - besides having an insane name, has Facebook OAuth Authentication baked right in! It also has a Facebook Authentication example in its Facebook example.

JSJaC Configuration

First you need to include the JSJaC Javascript on your page - remember the same origin policy - have your NodeJS webapp serve this file and we'll deal with proxying its messages below:

Or wherever you wanna put it. When you're ready to open up a chat connection to Facebook first we create a connection object telling JSJaC where to talk to:

//Setting the XMPP connection configurations
    var oArgs = {
        //Set the http-binding address (typically /http-bind/ but changing it for grins
        httpbase: '/fbchat',
        //Set the Timerval
        timerval: 2000
    };

    //Creation of the XMPP connection object
    var con = new JSJaCHttpBindingConnection(oArgs);

Once we're got a connection object we can attach event listeners for all interesting XMPP events - like new messages and people coming and going. See the JSJaC documentation for more details.

SO we've created our XMPP connection object - attached event handles for interesting events - now we can connect to Facebook's XMPP server:

/* Setting the connection authType and the Facebook app configuration
 * that will be used to allow the user login inside the appication
 * and start to chat with his friends */
var conArgs = {
  //Setting the authorization type to "x-facebook-platform" - VERY IMPORTANT!!!!
  authtype: 'x-facebook-platform',
  //Setting the Facebook Application configuration
  facebookApp: FB_APP_OBJECT
};
//Let's connect to Facebook server
con.connect(conArgs);

First do NOT forget to set the 'authtype' key correctly!! If you don't JSJaC will try to use standard XMPP authentication protocols and die an ugly death.

Second - what the heck is FB_APP_OBJECT???

Ok JSJaC (or ANY FB-compliant XMPP client) needs 3 things to authenticate to Facebook's XMPP server using OAuth:

Yer Facebook API key
Yer Facebook Application's Secret key
The Facebook Session key

These are supplied to JSJaC by an object having 3 corresponding functions which return those values:

getApiKey()
getApiSecret()
getSession()

So those first 2 values are standard values you can get from your Facebook Application's page.

The Session key is given by Facebook when your user logs into Facebook (& has granted your app the 'xmpp_login' permission. Now I myself also request the 'offline_access' permission from Facebook so you only have to get this value once & then save it off somewhere for just this very moment. I'm a Redis man myself.)

Your Facebook API key can be hard-coded into your Javascript safely HOWEVER your Application Secret parameter is supposed to be a SECRET so I would not recommend hard-coding that value into your Javascript.

So you should max an Ajax call (ideally over SSL) to get your Application secret and you can grab the user's Session key while you're at it.

ANYWAYS back to FB_APP_OBJECT. FB_APP_OBJECT provides those 3 function mentioned above that provide those values.

Here's the easiest one possible (note all values are totally made up and will not work 'for reals'):

FB_APP_OBJECT = {
    getApiKey: function( ) { return 'j43983489fj3f498'; },
    getApiSecret: function() { return 'e09ef09kef09wefkefw'; },
    getSession(): function() { return '38328923dj239d8j2938dj'; }
};

Your FB_APP_OBJECT hopefully will be smarter than that!!! So the client side is all done - now we need the server to deal with this...

NodeJS Proxy to BOSH Server

Our JS XMPP client (JSJaC) can only talk to the server/port that served it - ye olde same origin policy. However our NodeJS server in this case is serving this file SO we need to proxy JSJaC's HTTP requests over to our local XMPP BOSH server running on port 5280. Of course you can proxy these requests to a BOSH server running anywhere - ours happens to be running locally.

Here's what our Connect-based NodeJS proxy code looks like:

if (url.pathname == '/fbchat') {

// proxy these connections to localhost:5280
            req.on('data', function(chunk) {
                var options = {
                    host: 'localhost',
                    port: 5280,
                    path: '/http-bind/',
                    method: 'POST'
                },
                fb_req = http.request(options, function(fb_res) {
                    fb_res.on('data', function(fb_data) {
                        //console.log('xmpp response: ' + fb_data.toString());
                        res.end(fb_data.toString());
                    });
                });

                //console.log('xmpp request: ' + chunk.toString());
                fb_req.end(chunk.toString());
            });

So some interesting things here - the XMPP BOSH requests are all POSTs - so we only listen for those. Also we've configured JSJaC to POST back to us a '/fbchat' - could be anything - like '/http-bind' is typical. So as soon as we get POST data from JSJaC we send it to our BOSH server on port 5280 at '/http-bind/' and any data we get back from the BOSH server we pass back to JSJaC.

Fire It All Up

SO fire up our NodeJS server - that will start our BOSH server listening on port 5280. It will serve the JSJaC.js seed file. It should also give JSJaC our Facebook Application secret and the user session for when our user is ready to chat via some Ajax happiness. Obviously anyone can still see these values so SSL for them would be nice any another smart way to get our value out to JSJaC.

User visits our site, gives us permissions to use Facebook chat, which also gives us our Session key that JSJaC needs. Our NodeJS app proxyies JSJaC BOSH POSTs over to our BOSH server and proxies any replies back. Now can then connect to Facebook's chat server and respond to events and send messages all via JSJaC's APIs. And life is good!

Wrapping Up

Now you can write a sweet chat UI (or not!) for your site and users can chat with all their Facebook friends without ever having to leave your wonderful site!!

Since JSJaC is an all-purpose XMPP client your users can also connect to any other XMPP server/chat.

Have fun and I hope you enjoyed my VERY FIRST BLOG POST!!! FTW!!