Over my head

A guide to running nodejs on Windows (almost)

2010-03-29T14:47:00.016+01:00

nodejs doesnt work with windows.

So imo the best thing we can do if we want to join the community is run a virtual machine.

Here's how:

1) Install virtual box

http://www.virtualbox.org/wiki/Downloads

2) Get an unbuntu iso

http://www.ubuntu.com/GetUbuntu/download

3) Install the ubuntu iso. For detailed instructions visit

http://psychocats.net/ubuntu/virtualbox

3) Install vbox guest additions. For detailed instructions visit

http://helpdeskgeek.com/linux-tips/install-virtualbox-guest-additions-in-ubuntu/

4) Open a terminal on your ubuntu virtual machine.

5) Type in the following into your terminal to grab git and the build-essential packages

sudo apt-get install build-essential git-core

6) Type in the following into your terminal to grab node.js from git

git clone git://github.com/ry/node.git

7) Configure, make and install... type the following into your terminal:

./configure

make

make install

node.js is now installed on your machine.

DOJO and the elusive GridTree

2008-12-11T09:14:00.003Z

If you look in the current release of dojo you wont find a GridTree.

However, if you are willing to pull down the full dojo source code (including dojox and the utils etc) from the svn repository and apply the patch associated with ticket #7402, when you build the source, you'll find yourself the proud owner of an unsupported dojox.grid.TreeGrid.

It works well and if you look in the dojox grid tests directory you'll find the file test_tree_grid.html which is a good example of how to use it.

The Physicist and the dairy farmer

2008-11-25T09:04:00.000Z

A bunch of dairy farmers want to increase milk yield so they hire a physicist (who they are told understands all things and can model anything) who spends a year studying cows, their diet, yields, and slowly understanding everything there is to know about dairy farming.

After a year he gathers the farmers together to present his findings and starts his lecture by drawing a circle on a blackboard and saying "Imagine a perfectly spherical cow..."

Hands up who..

2008-11-24T10:43:00.002Z

I wonder how many companies are today scrambling to fix hard coded VAT values in their ecommerce apps.

Hands up.

Gordon Brown doesnt know what he's done :)

Building an application to make my brothers life easier

2008-11-23T17:42:00.031Z

I'm looking at building a couple of apps (outside of my day job) that will help my brother with his job. The first application will need a rich desktop like interface and so I'm investigating my options.

Build upon the application framework I've built - this is MVP based which means its highly testable and has good seperation of concerns. It supports templatable views, binding, observers, code generation (via ruby) and is based upon dojo so has access to all the lovely widgets that dojo provides.
Use an existing framework.

So before I make a decision, I need to research the existing frameworks and define my requirements.

I want an application framework. A framework that lets me build applications rather than add nice effects to my web pages.
The framework should abstract away the concepts of html, css and the browser.
I'm not designing a web page, so for me its perfectly acceptable for my application to look like a windows app or a OS X app. In fact this is desirable since it should mean that my brother will instantly know how to use the application.
The framework should not be closed source or depend on a browser pluggin.

So I'm not using:

Flex (4),
sliverlight (4),
extjs (4),
scriptaculous (1),
moo tools (1)
dojo(1)

Note that I have marked each technlogy with a number which refers to my requirements defined above. So, this leaves me with:

Sproutcore
Cappucino
Open laszloa
1 million other RIA framworks I'm unaware of.

So now I've reduced my area of research down to 3 frameworks. This post will become part of a series which I will investigate these 3 frameworks. The series will serve mainly as a scratch pad, but hopefully may help others.

This remainder of this blog post will discuss my initial thoughts on Sproutcore based upon a few hours reading around and mucking around with code.

Sproutcore Pros.

You write your application prodiminantly in JavaScript.

I dont have to learn an new language!

The framework is based upon MVC.

I like this pattern, I've used it a few times and developed frameworks based upon it. Its simple and usually only the views are difficult to test. I'll talk about views, controllers and models in more detail later. For a good definition of MVC, read this wikipedia article

Its backed by a big boy (Apple)

This means its probably going to be around for a while and should have some money pumped into its development. Apple have in the past branched existing open source projects, invested and changed the licence on the branch. I dont think this will happen here.

Gem

Its installed as a gem and I can generate my application stub, controllers etc at the command line.

Models - Binding and Observing

Sproutcore uses a similar approach to binding model properties to controllers, models and UI as Cocoa bindings.
Its easy to "mark" a function as an observer of a property on any given object.
In order to notifiy an observer of a change to any of your models properties, you are required to get and set property values on the model via two inherited methods, get and set.
To get a property "title" on a model called myModel, I code: myModel.get("title").
To set a property value I call myModel.set("title", "new value here").
This may seem annoying at first since, quite rightly, you will realise that you could just code myModel.title="new value here".
However, the set method is not just setting the properties new value, its also informing all observers that the value has changed.

This is incredibly powerful since, for example, views can be kept synchronised with any model they are observing.

Views

View templates are written as rhtml files using templating language called Erubis (ROR uses this, so I'm familiar with the syntax).

Controllers

These appear to be standard MVC controllers.

Sproutcore Cons

You write your application prodiminantly in JavaScript.

While the amount of css and html I've written so far within sproutcore is small, I'm not really abstracted away from the underlying technologies that the browser interprets.

Look

The widgets are UGLY in comparison to those generated by extjs and dojo! I expect this to change!

Documentation is scarce.

This isnt a massive deal, since I can just look at the sproutcore source, although a few more best practice tutorials would be helpful. This may be a problem for developers who are used to the quality of documenation usually associated with commercial frameworks or more establish opensource libraries such as dojo.

Real World apps

The only application I have found that it is available to the public thats built with Sproutcore is the demo photo viewer. This is quite slow and clunky and I hope it isnt a good representation of what can be achieved with Sproutcore. Its a poor show in comparison to 280 slides, a world real world application built with 280North's Cappucino

And so..

Now I'm going to actually start developing a simple application using sproutcore. I'll write a follow up to this post with opinions based upon 12 or more hours development time. Should provide me with a more informed opinion and I expect the cons list to fill up a little

nil and objective-j

2008-11-17T19:55:00.007Z

Suppose I have an object instance called person, which has an instance method called yelp.

To call this method, in objective-j, I code:

[person yelp];

In most languages if person is null (or in this case nil) calling yelp would result in an exception being thrown.

Not in objective-j. The method* yelp isnt called.

Thats just wierd.

*Note that method calls are referred to as messages in objective-j.

The problem is...

2008-11-16T21:52:00.005Z

My client has charged me with the task of building an application which solves a problem.

A warehouse management application perhaps.

So, I'm going to deliver the applications interface over the web because my client doesnt want to manage the installation and management of software across their 3,000,0000 warehouses.

Further they wont allow me to install 3rd party browser plugins - all I can rely on is HTML, CSS, JavaScript and HTTP.

So I cant use Flash or Silverlight.

Also, the .NET framework isnt available on the client machines. Java isnt either.

There goes ClickOnce and Java Web Start etc etc.

So what do I do?

Shit.

Recently I wrote a little MVP application framework on top of Dojo. It worked well. Nice seperation of concerns, testable etc etc.

Before that I wrote an MVC framework on top of the clientside Microsoft AJAX framework.
That worked okay too. The codes horrid.

I worked with extJS, but then they started to play with their licence...

Before that I wrote a.....

Infact for almost every major web application I've had to write an application framework on top of some other framework/library.

The underlying framework provided code that attempted to hide the fact that the different browsers were... different.

My framework would provide common services to the developer and enforce a development style for the team to work within. Literally an application framework.

The fact that I felt like I had to write an application framework was annoying in itself.

Why hasnt someone whose cleverer* than me done it? If they have done it, why dont I know about it?

Further, the underlying frameworks didnt completely abstract the differences between the browsers and they seemed focused on enabling developers to build small web 2.0 esq additions to their web pages rather than enabling me to build fully formed applications.

In each case, the project started well, however later down the line, browser related issues ALWAYS cropped up.

For example, IE6's piss poor JavaScript engine prevented me from building large DOMS.

The application in question used tabs, probably like the browser you're using. As more tabs were loaded, the app slowed down.

I solved this problem by disconnecting unfocused tabs from the DOM and re-attaching them when the tab was focussed. At this point my layout manager was told to redraw the layout to ensure that scaling had occured correctly.

So in each case I had to solve problems that werent in the business domain. In most cases these issues took up the majority of the time.

Desktop application development isnt generally like this. Sure ,there are still issues, but they often arent as frequent and as complex to solve.

I should be able to develop a web based application where the biggest problems I have to solve are in the business domain.

So, it is possible to write a large scale web base application, google and friends have proven this.

BUT..... its NOT trivial and the majority of the problems you have to solve are down to the platform (browser).

Also, if I'm building small web apps such as RSS feed readers, then I can do so quickly and easily. Dojo and ExtJs are all good enough to do this.

But as soon as you're handed a enterprise scale app your development cycle will probably begin to unravel.

Of course, you guys may be all geniuses and may solve these problems easily and quickly. If so, I havent met any of you :)

At the moment I'm investigating technologies in preperation for a technology referesh of a applications user interface.

Cappucino I'm told, is the solution to my problems.

* Most people are cleverer than me.

Re-applying CSS to the DOM with IE6

2008-11-14T10:06:00.009Z

I've been involved in developing a AJAX framework which sits on top of the Microsoft AJAX libraries.

This contains a layout manager which manages the DOM, so for example if the user re sizes the browser window, the layout manager will manipulate the DOM to ensure the correct layout.

In most cases it merely sets the height and width styles on DOM elements and applies CSS classes.

When testing with IE6 my colleague noticed that any images managed by the layout manager (i.e. tab headers) were flickering and that resizing was generally slow.

I assumed that the layout manger was to blame. It wasn't directly.

All images associated with the layout manager are referenced in CSS classes to allow for "themes".

The layout manager will perform the following sequences of actions:

Apply CSS to a DOM element - if the CSS references an image, your browser will request the file from the server
Next, remove the CSS class from the DOM element...
Then, if re-apply the same CSS class again to the previously mentioned DOM element - you would not expect the image to be requested again.

This is not the case for IE6.

Every time the layout manager was re-applying the same CSS class to the DOM, the associated resources were being re-requested.

We used fiddler to confirm this.

We added a expiry header to all the referenced images (IIS configuration) and now the layout manager runs *almost* as quickly in IE6 as it does in IE7.

Woot.

ASP.NET ViewState

2008-11-13T11:23:00.010Z

Background.
ViewState is a collection of data which enables a control/page etc to persist properties across postbacks.

Like a HashTable, the ViewState contains key value pairs.

The ViewState is represented across posts by a hidden form field.

The data stored within this hidden field is referred to as a StateBag.

TrackViewState

The ViewState object exposes a method called TrackViewState.

Only after this method is called upon the ViewState, are items stored within the ViewState marked as dirty when a keys value is changed.

New key value pairs will also be marked as dirty from this point.

TrackViewState and the page/control life cycle

TrackViewState is called during OnInit.

After this point, data in the StateBag is applied to the ViewState.

This means that if you modify the ViewState after OnInit, the data will be persisted in the StateBag and will be available after a subsequent postback.

However if you modify the ViewState before OnInit, or during OnInit your changes will NOT be stored in the StateBag.

Embracing the new narcissism

2008-07-09T13:40:00.004+01:00

Now that we've finally decided the technology we're going to use to build the new version of ListenTo, we're starting to think in terms of new functionality and improving the user experience.

ListenTo needs bands and needs the bands to interact with the site in order to generate some of our content.

We've never had problems getting bands to sign up, but have had problems with getting to come back to the site to interact.

I'm in a band called Seizures and one things I've realised from talking to the guys in the band and other members of the local music community is that people in bands generally have an element of narcissism to their personalities.

After reading this article we realised that allowing people to add the photos they've taken at gigs and tagging the bands in the photos may draw bands back to the site.

For example, one of our photographers or users takes a photo of Seizures during a gig, uploads it to ListenTo and tags us. The band members get an email telling them they've been tagged, so they visit the site and look at the new photos and perhaps add content themselves.

This is not a new idea facebook and myspace all do this, but I think it is new to our style of website. We arent a social networking site, really we're about promoting the gigs of local bands, but taking some of the ideas that these sites use so successfully may work.

We'll have a few original ideas in the next release too - I'm not telling though.

ListenTo and s#arp.

2008-07-07T16:07:00.006+01:00

I've just started re-developing ListenToManchester.

I seem to do this every 6 months. It goes something like this...

A new technology comes along.

I get excited.

I begin redeveloping the site with this new technology (ie Ruby on Rails, s#arp).

I discover that the flavour of the month isnt quite as good as I expected and that my productivity isnt quite what I was promised (I usually want to have it completed in a few days).

Most recently I looked at S#arp which is a framework based on nHibernate (yay), Spring.NET (yayer) and the new MVC preview from Microsoft.

I love MVC.
I love nHibernate.
I love Spring.NET.

I dont like s#arp based upon what I've seen of the "best practice" NorthwindMVC project shipped with the project.

The NorthwindMVC.Core project contains the business logic of the application.

It also contains nHibernate mapping files.

Why would they do this??

I dont want my business tier to have any dependency upon the implementation of my data access tier!!!

Ideally I'd have a seperate project that defines interfaces i.e. ISupplierDAO (lets call it NorthwindMVC.Shared).

Using spring I'd inject an implementation of this interface into my business tier.

I dont want to care about the details of the DAO implementation in my Business layer!

This way my business and data tiers only depend upon NorthwindMVC.Shared!!

Is this really how the devs of this platform want us to use it?

Of course, I'm probably being a bit whiney. This is probably just an implementation detail.

Thoughts??

Having said all this, this framework is leaps and bounds better than what ListenTo is currently running off CSLA.NET (shudders).

Your companies product....

2008-03-11T11:01:00.002Z

Via fffound

.Net developers and why we're crap

2008-03-10T13:58:00.015Z

ListenToManchester is a website designed to help promote bands in Manchester (UK).

It is now almost 3 years old and recently I decided to have a look at how its being used by our users.

Our stats have made it clear that key areas of the application just aren't being used. People weren't reviewing music, writing articles and rating music. They are however listening to the music and searching for gigs. Interestingly, a large percentage of our users come from the US.

When we created the site we had lots of ideas and we tried to use all of them. As a result the many sections of the site aren't particularly usable or functional.

Like many websites, it feels like it was designed by a developer - it doesn't really solve the problem it was designed to.

Initially I was considering rebuilding the site using Ruby on Rails, but after much thought, we've decided that we don't particularly want too invest to much time in the site, but we do want to fix some of its many problems.

So, 1 hour before me and my co-developer (Tom Adams) were due to play our weekly Power League 5-aside football match, we grabbed a mac laptop opened the website I decided it was time to cull functionality in order to focus the site on the popular content.

We decided that since reviews and articles weren't used often, and when they were people didn't understand the difference between them. More specifically, they didn't understand our definition of articles and reviews.

So, we decided that these two sections of the site should be combined and renamed to news.

On the homepage we decided to drop our music chart, since users weren't rating tracks, which meant that the chart hadn't really changed in over a year.

With these decisions made, we fired up Text Wrangler and opened up some of the usercontrols and web forms (the site was written with ASP.NET).

Our aim was to make as many of the changes without having to recompile all the C# since we were working on a mac.

I was appalled by the code in front of us.

Almost all the changes we needed to make required re-compilation, when really it should have just been configuration and modification of the HTML and CSS! Almost zero thought had been given to the maintainability of the code!

No wonder Tom, who is responsible for the CSS and HTML, had a dislike for ASP.NET during the development of ListenTo!

Worryingly, many designers who have worked alongside developers who use .NET have faced similar problems.

.Net isn't to blame though. In the case of ListenTo, the fault was my own.

The biggest problem with .NET isnt with the framework or the languages it supports. The problem is one of education.

Most "professional .Net developers" when compared with Java developers are little more than hobbyists (from a technical perspective) who happen to get paid for their work. They don't architect solutions. They aren't software engineers. Many seem to be adept at ctrl-c followed by ctrl-v.

I've recently been interviewing people for a development role within the company I work for and it scared me how little people seem to know about basic concepts.

Am I a better developer than they are?

For the last few years a colleague of mine ( a Java Dev) has pushed me deeper and deeper into the world of open source .NET projects such as Spring.NET and introduced me to concepts as inversion of control, loose coupling, designing immutably, design patterns etc etc. Using different frameworks such as ROR has introduced me to entirely new approaches to building web applications (MVC).

It's been quite an eye opener and, as a result, I'm now a much better developer.

Are you a .Net developer? Stop thinking of yourself as a .Net developer! Start developing with other frameworks and languages alongside .Net!

Our next project is being brought to you using Ruby on Rails (mmm bandwagon goodness).

Regular expressions, JavaScript and cross browser hell

2008-03-05T17:27:00.007Z

Using regular expressions with JavaScript can be frustrating.

Specifically, trying to use regular expressions consistently with the different web browsers (Explorer and Firefox) is frustrating.

For example, using the match function upon a string yields different objects.

Firefox returns an Array containing the matches and Explorer returns an Array with the additional properties :

The original string

The matches (attached to properties 0 to n-1 ( where n = number of matches)

A bunch of indexes.

var myMatches = mystring.match(myRegEx);
for(var match in matches ) {
alert(matches[match]);
}

So, the code above will alert out just the matches in Firefox.
Explorer will alert out the matches and the values of the additional properties too.

Which is correct? Time to read the specs (http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf).

Section 15.10.6.2 is pretty explicit. Explorer implements the functionality described in the specs, but the additional properties we mentioned above, are bespoke.

Damn you Microsoft.

Coding in functional manner

2008-03-04T15:39:00.002Z

Functional programming languages, such as Lisp (a mathematical notation for computer programs), are very different from procedural languages such as Java and early versions of C#.

Functional programming is a style of programming that emphasizes the evaluation of expressions, rather than execution of commands. The expressions in these language are formed by using functions to combine basic values. A functional language is a language that supports and encourages programming in a functional style.

What is this "functional" style?

Consider the following quadratic equation:

x^2 + 5 = y

Given a specific value for x, we always expect the same value for y.

So, if x= 2, y= 9.

This is always true.

In many cases, particulary in a multi-threaded environment, it is helpful to develop functions that, for a given set of arguments, will always return the same result.

Why? It helps us develop stateless imutable functions that are thread safe.

NOTE: Something always has to be mutable otherwise our Lisp application couldnt do" anything!

Long running processes and JavaScript

2008-03-04T15:24:00.006Z

Events like mouse clicks and keystrokes are handled in a single dedicated thread.

If the handling JavaScript, which also runs in its own thread, is unresponsive, the browser remains responsive, but eventually willl inform the user of the problem and give them the option of cancelling the handler!

This means that long running client side code cannot be executed without implementing a continuation style approach.

Example 1
could be replaced with:
Example 2

This is horrible. However, the javascript library Narrative enables blocking capabilities for asynchronous event callbacks. This makes asynchronous code more readable.

So, it will convert the following code into Example 2:

Fortunately, it is rare that the client code should ever have to execute long processes.

SetTimeout

2008-03-03T17:32:00.001Z

SetTimeout can appear to provide concurrent code execution. While SetTimeout is not part of the ECMA-262 language specification, it is still implemented within a single threaded environment.

Here is a test I performed to try and understand whats going on.

function() test1{
window.setTimeout( function(){alert('TimeOut function');},100);
for(var i=0; i<=10000000;i++) { if(i==10000000) {alert('long running process');} }} window.onload = Test1;

In this example, the alert 'long running process' always executes before 'TimeOut function'.

From this test we can conclude that the anonymous function is not guaranteed to be executed when the time period passed to the setTimeout method has elapsed.

So whats happening?

As I pointed out previously, JavaScript is single threaded and as such can only ever execute one piece of code. This means that if an asynchronous event such as a mouse click or a setTimeout call to anonymous function, occurs during the excution of a code block (not necessarily an atomic call), the event is queued.

At the next available point during execution the queue will be queried and the next atomic code block executed.

However, if we remove the "if" statement from the loop in the test we get a different result.

function() test2 {
window.setTimeout( function(){alert('TimeOut function');},100);
for(var i=0; i<1000000;i++){alert(i);}}>

window.onload = test2;

The "TimeOut function" can be alerted before, during and after the alerts within the loop, for example:

0
1
2
3
4
5
setTimeOutCalled
6
7
8
9
...

Does this contradict our previous conclusions? No.

The javascript "alert" method is not responsible for creating the popup box that we see. The browsers UI thread does this. Which means that we have two threads, the javascript (containing our async queue) and the browsers thread for rendering UI. So, we cant guarantee the order we see the alerts, is the order that the javascript executes the alert statement

So, to truely understand whats going on, we need to try to remove interactions with the UI thread.

We will place our "alerts" into an array and render (note that I have used attached an add method to the Array constructor to help with manipulating arrays).

var var messages = [];
function test2_Revisited()  {
window.setTimeout( function(){Array.add(messages,"setTimeOutCalled");},100);
for(var i=0; i<1000000;i++) {Array.add(messages,i);}} 
window.onload = test2_Revisited;

0
1
2
3
4
5
6
7
8
...
999999
setTimeOutCalled

Conslusions

We cannot guarantee that the delay requested in a setTimeout will be honoured.
The seperate UI thread means that alerts and console.logs calls from javascript are not synchronised with the user experience.

For more information  see this excellent article written by Jphn Resig

AJAX, Javascript and Threading

2008-03-03T14:57:00.001Z

Javascript and Microsoft's JScript are compliant with the ECMA-262 language specification.

I looked at the specification available at http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf and it doesnt mention threading.

There are no keywords or intrinsic ECMAScript objects available to manage threads, or to provide for thread synchronization.

So, if threading is available in some or all browsers, the implementation is notcontrolled by this spec.

Javascript Engines

I couldnt find any details of the implementation of IE 7s Javascript engine, however, SpiderMonkey, the engine used by Firefox is documented at http://www.mozilla.org/js/spidermonkey/

The SpiderMonkey Javascript engine supports native threading!!

However, it appears that firefox only uses one JSContext for Javascript.

Without Threading, how does AJAX work?

It seems that many people blogging about AJAX mistake asynchronous requests for threading.

This is not the case. Consider the following:

I post an aysnchronous request to a web service. This request is associated with a callback function.

Since this request is asynchronous and is now being handled by the browser rather than the JavaScript engine, I can execute JavaScript code while this request is being processed by the server.

The code I choose to execute is a long running process and server will respond before the process is complete.

Does the asynchronous request and callback function execute in their own thread?

No. The callback function is not called until the long running process is complete even though the server responded quicker. Javascript queues asynchronous code when it is blocked.

Implementing our own threading model

There are libraries, written in Javascript, that imitate the programming conventions associated with threading, but these "threads" still cannot execute code concurrently.

http://www.neilmix.com/2007/02/07/threading-in-javascript-17/ uses "python style" generators introduced in Javascript 1.7

Conclusions:

Microsoft need to start documenting the inner workings of their code. No more blackboxes!!!
JavaScript does not support native threading.

Since we cannot guarantee the order of asynchronous events, coding 'functionally' and 'immutabily' is still useful even in this single threaded environment.

We avoid the use of global variables since asynchronous code mutating the same variable can lead to unwanted "side effects".

We should always have a good reason to mutate!