Eric Wendelin's complete blog can be found at: http://eriwen.com

Items:   1 to 5 of 10   Next »

Thursday, December 1, 2011

I wrote previously about Continuous Integration for JavaScript where I explained a build with Jenkins and Gradle. I’ve learned a lot since that article and thought it’s now significant enough to write more on the topic.

Documentation

When you’re writing code that other developers have to use or maintain, you ought to provide some amount of documentation. Your code is simply not self-documenting enough.

My favorite doc tool right now is jsduck developed by Sencha Labs for their Ext JS 4 docs. It basically consumes JSDoc-style comments (with some extras for namespaces, etc.) and generates beautiful documentation. Super easy to install and use:

gem install jsduck
jsduck src/js --output target/docs

Use the Gradle wrapper

Using the Gradle wrapper allows users to build your project without installing Gradle, which is beautiful because it removes a very significant dependency for building and running your project.

Here’s how to use it. Add this to your build.gradle file:

task wrapper(type: Wrapper) {
    // version of Gradle you want
    gradleVersion = '1.0-milestone-6'
}

Then you just have to run in your shell:

gradle wrapper

and Gradle will add a few files to the project, most importantly gradlew and gradlew.bat (for Windows). These are binaries that your can run in lieu of gradle, and it wraps (obviously) the execution of Gradle such that it will download the correct version of Gradle and execute it locally.

I recommend that you switch to the Gradle wrapper when you have consumers of your project that may not have Gradle. I’m going to be doing this for all of my OSS projects from now on.

Introducing the Gradle Web Suite

Gradle really has building JVM-centric source down (for most languages), but I felt like it was cumbersome to operate on my web files with it. So I’ve written a collection of plugins that focus on making tasks with JavaScript, CSS, and other client-side tech dead simple.

buildscript {
	repositories {
		mavenCentral()
	}
	dependencies {
		classpath 'com.eriwen:gradle-css-plugin:0.2'
		classpath 'com.eriwen:gradle-js-plugin:0.2'
	}
}

apply plugin: 'css'
apply plugin: 'js'

And now we can use the tasks provided by these plugins:

// Combine, minify and GZip CSS to teenytiny.css
css {
    input = fileTree(dir: "${projectDir}/css", include: ["file1.css", "file2.css"])
    output = file("${buildDir}/teenytiny.css")
}

// Same thing here, and look, file-globs :)
js {
    input = fileTree(dir: "${projectDir}/js", include: "**/*.js")
    output = file("${buildDir}/combinedMinifiedAndGzipped.js")
}

// ... and there's support for other tools like CSS Lint!
csslint {
	inputs.files fileTree(dir: "${projectDir}/css", include: "**/*.css")
	outputs.file file("${buildDir}/csslint.xml")
	options = ["--rules=adjoining-classes,box-model", '--format=lint-xml']
}

Gradle CSS Plugin | Gradle JS Plugin

JsTestDriver

One tool I hadn’t paid enough attention to until recently was JsTestDriver. The awesome thing about it is that it has adapters for other JS testing frameworks like Jasmine. This is great because I don’t have to convert my tests (mostly) to JsTestDriver’s test API and I get crazy speed and reporting I wouldn’t otherwise get.

In a nutshell, JSTD is a testing tool developed by Google whereby you start as server and then “capture” browsers so that JSTD uploads tests to them, the browser runs tests and then sends back test results. This is awesome because it’s super fast and you can have it run every major browser simultaneously!

Here’s how you can use it. First, we need to download JsTestDriver and the coverage plugin JARs. Then we need a configuration file for JsTestDriver:

server: http://localhost:4224

load:
  - lib/jasmine.js
  - lib/sinon-1.2.0.js
  - lib/jasmine-sinon.js
  - lib/JasmineAdapter.js
  - ../js/main.js

test:
  - spec/*.js

plugin:
  - name: "coverage"
    jar: "lib/plugins/coverage.jar"
    module: "com.google.jstestdriver.coverage.CoverageModule"
    args: useCoberturaFormat

timeout: 30

then we need to start the JSTD server:

java -jar path/to/JsTestDriver.jar --port 4224

Now we want to capture a browser by navigating it to http://localhost:4224/capture. Finally, run our tests with all captured browsers:

java -jar path/to/JsTestDriver.jar --tests all

and you should see something like this:

Chrome: Reset
.................
Total 17 tests (Passed: 17; Fails: 0; Errors: 0) (13.00 ms)
  Chrome 17.0.942.0 Mac OS: Run 17 tests (Passed: 17; Fails: 0; Errors 0) (13.00 ms)
/Users/eric/src/eriwen.com/test/spec/PageSpec.js: 95.789474% covered
/Users/eric/src/eriwen.com/test/lib/jasmine.js: 51.640926% covered
/Users/eric/src/eriwen.com/test/lib/sinon-1.2.0.js: 22.916668% covered
/Users/eric/src/eriwen.com/test/lib/jasmine-sinon.js: 73.333336% covered
/Users/eric/src/eriwen.com/test/lib/JasmineAdapter.js: 64.83517% covered
/Users/eric/src/eriwen.com/js/main.js: 60.15037% covered

And here’s how you can run it with Gradle:

task jstd(type: Exec, dependsOn: 'init', description: 'runs JS tests through JsTestDriver') {
    // Default to MacOS and check for other environments
    def firefoxPath = '/Applications/Firefox.app/Contents/MacOS/firefox'
    if ("uname".execute().text.trim() != 'Darwin') {
        firefoxPath = "which firefox".execute().text
    }

    commandLine = ['/usr/bin/env', 'DISPLAY=:1', 'java', '-jar', "${projectDir}/path/to/JsTestDriver.jar", '--config', "${projectDir}/path/to/jsTestDriver.conf", '--port', '4224', '--browser', firefoxPath, '--tests', 'all', '--testOutput', buildDir]
}

Notice the coverage numbers? JSTD has generated a report in lcov format, but this isn’t very readable to humans.

Code Coverage!

I couldn’t find any working scripts on the interwebs to convert the coverage file to Cobertura XML for reporting, so I wrote an lcov-to-cobertura-xml converter so we can report code coverage with Jenkins!

And again with Gradle:

task jsCoverage(type: Exec, dependsOn: 'jstd', description: 'JS code coverage with cobertura') {
	commandLine = ['python', "${projectDir}/path/to/lcov-to-cobertura-xml.py", '-b', "${projectDir}/", '-e', 'test.spec', '-e', 'test.lib', '-o', "${buildDir}/coverage.xml", "${buildDir}/jsTestDriver.conf-coverage.dat"]
}

I talked about all of this at the Rich Web Experience and I have some slides from my talk that might be helpful if you want more detail about this stuff.

What about PhantomJS?

PhantomJS is still very necessary because JsTestDriver does NOT allow page or DOM interaction like Phantom does. I’m really excited about CasperJS as it looks like a really nice way to do more functional testing with Phantom. I recommend you check it out.

Conclusion

You can see all of this in action in the build for this very site.

So that’s all I have for now. Let me know your experiences with these things or if there is something I missed, comment it up! Cheers!

Related posts:

  1. Why every programmer should have a Tiddlywiki
  2. Continuous Integration for Javascript
  3. Guest Post: Save Text Size Preference Using MooTools and PHP


Wednesday, August 31, 2011

I have posted more thoughts and tools on how to improve this here.

Jenkins is a CI tool that is often used for Running tests and code analysis for Java and .NET projects. There are a lot of benefits that we as a community are not taking advantage of for our web (CSS, JS, etc) code. In this article I’m going to walk you through setting up automated building and testing for a JavaScript project.

NOTE: The steps outlined are generally Linux/Mac centric, I don’t go into depth on Windows setup, but it shouldn’t be much different using Cygwin.

Why use CI?

Aside from the traditional benefits you see from your compiled code, there are some very compelling reasons:

  1. Automate versioning, combining, minifying, and gzipping files
  2. Run automated tests and get reports, keeping the codebase maintainable
  3. Run static analysis tools like the closure compiler or jshint
  4. Auto-deploy files (to S3, say) if our build passes
  5. Tag and other special stuff for release builds
  6. And that’s not all!TM We can also hook in Selenium tests, CSS Lint, and more

Not convinced? Tell me why in the comments.

Jenkins setup

Downloading and running Jenkins is incredibly easy. Just download Jenkins and run:

wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war
java -jar jenkins.war

There are also native installers for most environments.

Now point your browser to http://localhost:8080 to see it running.

Nailed it! Now we have a CI server!

Prerequisites

If we’re going to be using git and gradle, we’ll need to install them (Windows Git installer).

sudo yum install git-core  # CentOS/RedHat/etc
sudo apt-get install git   # Debian/Ubuntu/etc
sudo port install git-core # MacPorts
sudo brew install git      # Homebrew
wget http://edub.me/ngMAeR # Gradle 1.0m3 ZIP
sudo mv gradle-1.0* /usr/local
sudo unzip /usr/local/gradle-1.0-milestone-3-bin.zip
sudo ln -s /usr/local/gradle-1.0-milestone-3 /usr/local/gradle

To tell Jenkins where to find Gradle, we go to Manage Jenkins > Configure System from the top page. Add a name and enter your GRADLE_HOME (/usr/local/gradle if you followed the instructions above). It should look like this:

Jenkins Gradle Setup

Setup a CI job

As an example, I’m going to use my stacktrace.js project on GitHub. Even though it’s a small JS library, almost all of the setup can be used for any type of project.

First, we want to install a few plugins that will help us out. We’ll need Git to pull down the code and Gradle to build it. You’ll want the Git, Gradle, and Violations plugins installed. It’s pretty easy to figure out, but hit up the Jenkins wiki if you get stuck.

We want to create a new job that runs analysis on our JS and runs our tests. Click New Job on the Jenkins sidebar and you’ll see the setup form.

Checking out and building a repo

Under Source Code Management choose Git and enter git://github.com/eriwen/javascript-stacktrace.git for the repo location and master for the branch. It’ll look something like this:
Jenkins Git Setup

For the Build section of the setup, we want to run the minify build target from Gradle. You’ll also want to enter the location of the build file as build/build.gradle. Don’t worry about the contents of our build script right now, I have a slew of blog posts in-progress that explain it. Stay tuned.
Jenkins Gradle Minify

Now would be a good time to click Save at the bottom and then click Build Now on the sidebar for the job. You should see Jenkins checkout, pull down the latest Closure Compiler, and use it to minify stacktrace.js.

Running JS unit tests with PhantomJS

The days are past when you have to open a browser to see if your JS is generally working. PhantomJS is a headless WebKit browser that lets us interact with web pages (click links, pull from localStorage, etc.) without the browser window! This will let Jenkins run a browser with our tests and report the result. To follow along with the example, download and install PhantomJS in /usr/local/bin (on Mac OS I just sudo ln -s /Applications/phantomjs.app/Contents/MacOS/phantomjs /usr/local/bin/phantomjs). Note that if you’re running Jenkins on a headless server, you’ll want to have xvfb.

We also need to update the Build part of the configuration by telling gradle to run the test target.
Jenkins Gradle Setup

Gradle is now setup to run the QUnit tests in the project, but we need to tell Jenkins where to find the test reports. Easy! I encourage you to check out the source of build.gradle if you want to know how this is setup. It’s really quite interesting.
Jenkins JUnit Setup

Save and try that build again to bask in automated JavaScript testing awesomeness.

Finding potential bugs with JSHint

I’m a big fan of JSHint and I think it’s worthwhile to have it run on every git push. Here’s how to add that. To follow the example, you’ll have to install NodeJS and the jshint module from npm:

sudo brew install node      # Homebrew
sudo port install node      # MacPorts
sudo apt-get install nodejs # Debian-based

curl http://npmjs.org/install.sh | sudo sh # Install NPM
sudo npm install -g jshint

Now we just need to tell Jenkins to run jshint as well as our tests and where to find the JSHint report. The gradle build puts it in target/jshint.xml. It should look like this:
Jenkins Violations Setup
Jenkins JSHint Setup

One more Build Now and we have a fully functioning JavaScript build. You’ll also see graphs with test and jshint results on the job page and can drill down into details.

I have really loved my CI setup for my web projects as well as my JVM-based ones. Now you have the power to go build a cool CI system for your projects. You can read more of my thoughts on this from my notes from my pursuit of the perfect front-end build. Enjoy!

Related posts:

  1. Javascript Stacktrace update
  2. How to highlight search results with JavaScript and CSS
  3. Javascript: Measure those “em”s for your layout


Wednesday, February 23, 2011

The Algorithm Design ManualI took a fair amount of time looking at data structures and algorithms while I was studying for my interviews with Google, and based on informed suggestions from Steve Yegge’s infamous post, I decided to buy The Algorithm Design Manual by Steven S. Skiena.

If you don’t care to read my ramblings about this book, here’s a summary: Buy this book if you do ANY serious programming.

What makes The Algorithm Design Manual

2 main reasons I make this blatantly positive assessment:

  1. The first several chapters are dedicated to the basics of data structures and common problems involving algorithms. This is obviously not a unique feature, but what is unique are the “war stories” from actual field work. The stories include discussion about the failure cases and how Skiena went about solving problems he encountered. This alone is enough to make this book worthwhile.
  2. Chapters 11-18 are a giant catalogue of algorithmic problems. Again, not a unique trait. However, not only does Skiena describe the basic approaches to solving each type of problem, he includes links to different implementations of in-the-field optimized solutions. He also brings up questions you should ask yourself when choosing an implementation.

The only caveat here is that most of the examples are written in C, which can be troublesome if you don’t know or have forgotten about pointers.

You should buy this book if…

You have at least 1 year of computer science training under your belt. If you are just writing one website in PHP for your cousin’s lemonade stand; then I don’t know why have you read this far. In that case you won’t be interested in this book.

Students and professionals alike will find The Algorithm Design Manual useful. In addition to the standard problem sets, I’ve also found the accompanying interview questions to be very interesting.

The only programming book that tops this is The Pragmatic Programmer by Andy Hunt and Dave Thomas.

What are some of the best Data Structures/Algorithms books you’ve found?

Related posts:

  1. How not to pass the SCJP exam
  2. Lessons learned from the SCWCD
  3. Microformats: Add hCard to your blogroll in 2 minutes flat


Thursday, February 3, 2011

I’ve come to love the Python language for its elegant syntax combined with powerful constructs like comprehensions. Jython allows me to take Python to the next level by allowing it to interact with my existing JVM-compatible code. Now I want to extend that even further and allow myself (and you, of course) to integrate Jython with Griffon, a framework for building desktop applications in Groovy.

Introducing the Jython plugin for Griffon

The Jython plugin enables compiling and running Jython code on your Griffon application. You don’t even need to install Jython manually. A Jython REPL is available with access to your Groovy/Java classes if you use:

griffon jython-repl

You can even load Jython scripts on-the-fly by putting them in MyGriffonApp/griffon-app/resources/jython!

In the rest of the article, we are going to create a simple application that mixes Griffon and Jython using the Griffon-Jython plugin.

Jython Sample Application

Getting started with griffon-jython

It’s easy to setup Griffon, and you’ll find instructions here and downloads here. Once you’re done, you can create a Griffon app and install the jython plugin by typing:

griffon create-app MyGriffonApp
# Go into MyGriffonApp directory
cd !$
# Download and install the latest Jython plugin!
griffon install-plugin jython

Now that we’ve installed the Jython plugin, you can create a Jython class on your command-line:

griffon create-jython-class com.mypkg.MyJythonClass

Let’s create a Jython class that greets the user when a button is clicked.

from com.mypkg import IGreeter

class MyJythonClass(IGreeter):
  def __init__(self):
    pass

  def greet(self, who, model):
    greeting = 'Hello %s from Jython!' % str(who)
    model.setOutput(greeting)

Jython classes are exposed to Griffon using an Object Factory Pattern as suggested in the Definitive Guide To Jython, Chapter 10. Therefore, we create a Java (well, Groovy) interface that MyJythonClass will extend to allow Griffon to get at the proper method implementations. It sounds rather complicated, but it really is quite simple. Let’s create the IGreeter interface to show you what I’m talking about:

package com.mypkg

public interface IGreeter {
  // Same signature as our Jython method
  public void greet(String greetee, def model)
}

Our greet() method accepts a String (the textField input) and a model of unspecified type.

Griffon auto-generates MVC classes for you, so we can use those. You’ll find them in the MyGriffonApp/griffon-app/{models,views,controllers} directories. We are going to change those for our app! Here is the aforementioned Model:

package com.mypkg
import groovy.beans.Bindable

class MyGriffonAppModel {
    @Bindable String input = ''
    @Bindable String output = ''
}

Groovy implicitly creates get/setInput() methods and allows them to be bound (to the View textField value, e.g.). Here is that MyGriffonAppView:

package com.mypkg

application(title:'Griffon Jython Sample', pack:true, locationByPlatform:true) {
  gridLayout(cols: 1, rows: 2)
  //Note the ID on this textField
  textField(id: "input", columns: 20)
  button("Click me!", actionPerformed: controller.handleClick)
  bean(model, input: bind {input.text})
}

Finally, the most critical part of our application that will interact with our Jython class, the MyGriffonAppController:

package com.mypkg

import java.beans.PropertyChangeListener
import griffon.jython.JythonObjectFactory
import javax.swing.JOptionPane

class MyGriffonAppController {
  // Values auto-injected
  def model
  def view
  def greeter

  def mvcGroupInit(Map args) {
     // When our model output has changed, show a dialog
     model.addPropertyChangeListener("output", { evt ->
       if(!evt.newValue) return
       doLater {
         JOptionPane.showMessageDialog(app.windowManager.windows[0],
           evt.newValue, "Message from Jython", JOptionPane.INFORMATION_MESSAGE)
       }
     } as PropertyChangeListener)

     // Instantiate MyJythonClass
     JythonObjectFactory factory =
         new JythonObjectFactory(IGreeter.class, 'MyJythonClass', 'MyJythonClass')
     greeter = (IGreeter) factory.createObject()
   }

  def handleClick = { evt = null ->
    if(!model.input) return
    model.output = ""
    // invoke Jython class (outside EDT)
    doOutside {
      greeter.greet(model.input, model)
    }
  }
}

There you have it! Now we can utilize such Jython greatness as generators and list comprehensions!

Further reading

The Griffon Guide is the best place to go for Griffon documentation. You can find more comprehensive documentation for the Jython plugin on the official plugin page. As always, this stuff is completely open-source, and you can find all of the code and submit suggestions and issues at the GitHub repository.

Related posts:

  1. Interview with Andres Almiray
  2. Python first impressions
  3. Using Python to update your FeedBurner stats


Tuesday, November 2, 2010

I’ve been spending a lot of time recently tinkering with different constructs and methodologies in Javascript, and one of the most fascinating things I’ve come across is Spencer Tipping‘s use of continuation-passing style.

It’s ok if you aren’t familiar with CPS, but I think anyone hoping to make the cognitive leap to functional programming should study it. As a bare miniumum, you need to know that a continuation is:

[a reification of] an instance of a computational process at a given point in the process’s execution

Therefore, continuation-passing style is just:

a style of programming in which control is passed explicitly in the form of a continuation

NOTE: I have replaced my crappy examples just below with Outis’s much more reasonable (and correct) ones. I take no credit for the simple CPS code below. Thank you Outis and David for the suggestions.

Consider the naive implementation of the recursive fibonacci function:

function fib(n) {
    if (n < 1) {
        return 1;
    } else {
        return fib(n-2) + fib(n-1);
    }
}

Then, identify function calls and returns. For each such point, create a continuation and pass it to the function call or return it. For example, the continuation for fibr(n-1) + fibr(n-2) is:

function(x) {
    return x + fib(n-2);
}

and you get:

function cpsFib(n, _return) {
    if (n <= 1) {
        return _return(1);
    } else {
        return cpsFib(n-2, function(a) {
            return cpsFib(n-1, function(b) {
                return _return(a+b);
            });
        });
    }
}

Apply this process to the linear recursive fibonacci function and you get:

function cpsFib(n, prev, cur, _return) {
    // Key component: call escape function when done
    if (n < 2) {
        return _return(cur);
    }
    return cpsFib(--n, cur, cur + prev, _return);
}
// Note the use of an identity function here
cpsFib(1000, 0, 1, function(x) {return x});

The only problem with this, however, is that Javascript engines to not optimize tail recursion, so calling: fibonacci_cps(10000) would cause a StackOverflowError in some browsers.

function() {
	return [1000, 0, 1,
		function() { return [999, 1, 1,
			function() { return [998, 1, 2,
				... ]]
			}
		}
	];
}

However, by adding to Function.prototype and changing the way we pass continuations, we can fix this problem.

// Function prototypes heavily inspired by Spencer Tipping's js-in-ten-minutes:
//   http://github.com/spencertipping/js-in-ten-minutes
Function.prototype.tail = function() {
	return [this, arguments];
}
// Tail-call optimization
Function.prototype.tco = function() {
	var continuation = [this, arguments];
	var escapeFn = arguments[arguments.length - 1];
	while (continuation[0] !== escapeFn) {
		continuation = continuation[0].apply(this, continuation[1]);
	}
	return escapeFn.apply(this, continuation[1]);
}

// Note the use of "Function.prototype.tail()"
function cpsFib(n, prev, cur, escapeFn) {
	if (n < 2) {
		return escapeFn.tail(cur);
	}
	return cpsFib.tail(--n, cur, cur + prev, escapeFn);
}

function identity(x) {return x}

// We pass the escape function instead of a reference to cpsFib
cpsFib.tco(1000, 0, 1, identity);
// within while (continuation[0] !== escapeFn) {
[cpsFib, [1000, 0, 1, identity]]
[cpsFib, [999, 1, 1, identity]]
[cpsFib, [998, 1, 2, identity]]
[cpsFib, [3, 2.686e+208, 4.3467e+208, identity]]

// return escapeFn.apply(this, continuation[1]);
[identity, 4.34673e+208]

How CPS and TCO can be practical

All this talk about styles and optimizations can sound rather like “over-engineering” to those of us who aren’t familiar with these concepts. While it’s a risk, good engineers will know when it is effective. Here’s how I would recommend you use these concepts:

  1. Processing large structures in a recursive way, perhaps recursing through an Object or HTML tree of indeterminate depth
  2. You’re adding functional methods to iterable objects (adding Array.prototype.map())
  3. Practicing uses of functional programming with Javascript

Conclusion

Knowing how to use functional techniques in Javascript will be more valuable as it moves to the server-side (due to the rising popularity of node.js) because doing more actual computation will be acceptable instead of blocking a UI.

I learned a lot from Spencer’s write-up Javascript in Ten Minutes, and would recommend you check it out. I’ve also found reading his other Javascript code and comments very insightful.

Please share other uses of functional Javascript you’ve come across.

Related posts:

  1. A Javascript stacktrace in any browser
  2. Javascript: Measure those “em”s for your layout
  3. How to highlight search results with JavaScript and CSS


Items:   1 to 5 of 10   Next »