Over my head

Via fffound

ListenToManchester is a website designed to help promote bands in Manchester (UK).

It is now almost 3 years old and recently I decided to have a look at how its being used by our users.

Our stats have made it clear that key areas of the application just aren't being used. People weren't reviewing music, writing articles and rating music. They are however listening to the music and searching for gigs. Interestingly, a large percentage of our users come from the US.

When we created the site we had lots of ideas and we tried to use all of them. As a result the many sections of the site aren't particularly usable or functional.

Like many websites, it feels like it was designed by a developer - it doesn't really solve the problem it was designed to.

Initially I was considering rebuilding the site using Ruby on Rails, but after much thought, we've decided that we don't particularly want too invest to much time in the site, but we do want to fix some of its many problems.

So, 1 hour before me and my co-developer (Tom Adams) were due to play our weekly Power League 5-aside football match, we grabbed a mac laptop opened the website I decided it was time to cull functionality in order to focus the site on the popular content.

We decided that since reviews and articles weren't used often, and when they were people didn't understand the difference between them. More specifically, they didn't understand our definition of articles and reviews.

So, we decided that these two sections of the site should be combined and renamed to news.

On the homepage we decided to drop our music chart, since users weren't rating tracks, which meant that the chart hadn't really changed in over a year.

With these decisions made, we fired up Text Wrangler and opened up some of the usercontrols and web forms (the site was written with ASP.NET).

Our aim was to make as many of the changes without having to recompile all the C# since we were working on a mac.

I was appalled by the code in front of us.

Almost all the changes we needed to make required re-compilation, when really it should have just been configuration and modification of the HTML and CSS! Almost zero thought had been given to the maintainability of the code!

No wonder Tom, who is responsible for the CSS and HTML, had a dislike for ASP.NET during the development of ListenTo!

Worryingly, many designers who have worked alongside developers who use .NET have faced similar problems.

.Net isn't to blame though. In the case of ListenTo, the fault was my own.

The biggest problem with .NET isnt with the framework or the languages it supports. The problem is one of education.

Most "professional .Net developers" when compared with Java developers are little more than hobbyists (from a technical perspective) who happen to get paid for their work. They don't architect solutions. They aren't software engineers. Many seem to be adept at ctrl-c followed by ctrl-v.

I've recently been interviewing people for a development role within the company I work for and it scared me how little people seem to know about basic concepts.

Am I a better developer than they are?

For the last few years a colleague of mine ( a Java Dev) has pushed me deeper and deeper into the world of open source .NET projects such as Spring.NET and introduced me to concepts as inversion of control, loose coupling, designing immutably, design patterns etc etc. Using different frameworks such as ROR has introduced me to entirely new approaches to building web applications (MVC).

It's been quite an eye opener and, as a result, I'm now a much better developer.

Are you a .Net developer? Stop thinking of yourself as a .Net developer! Start developing with other frameworks and languages alongside .Net!

Our next project is being brought to you using Ruby on Rails (mmm bandwagon goodness).

Using regular expressions with JavaScript can be frustrating.

Specifically, trying to use regular expressions consistently with the different web browsers (Explorer and Firefox) is frustrating.

For example, using the match function upon a string yields different objects.

Firefox returns an Array containing the matches and Explorer returns an Array with the additional properties :

• The original string


• The matches (attached to properties 0 to n-1 ( where n = number of matches)


• A bunch of indexes.

var myMatches = mystring.match(myRegEx);
for(var match in matches ) {
alert(matches[match]);
}So, the code above will alert out just the matches in Firefox.
Explorer will alert out the matches and the values of the additional properties too.

Which is correct? Time to read the specs (http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf).

Section 15.10.6.2 is pretty explicit. Explorer implements the functionality described in the specs, but the additional properties we mentioned above, are bespoke.

Damn you Microsoft.

Functional programming languages, such as Lisp (a mathematical notation for computer programs), are very different from procedural languages such as Java and early versions of C#.

Functional programming is a style of programming that emphasizes the evaluation of expressions, rather than execution of commands. The expressions in these language are formed by using functions to combine basic values. A functional language is a language that supports and encourages programming in a functional style.

What is this "functional" style?

Consider the following quadratic equation:

x^2 + 5 = y

Given a specific value for x, we always expect the same value for y.

So, if x= 2, y= 9.

This is always true.

In many cases, particulary in a multi-threaded environment, it is helpful to develop functions that, for a given set of arguments, will always return the same result.

Why? It helps us develop stateless imutable functions that are thread safe.

NOTE: Something always has to be mutable otherwise our Lisp application couldnt do" anything!

Events like mouse clicks and keystrokes are handled in a single dedicated thread.

If the handling JavaScript, which also runs in its own thread, is unresponsive, the browser remains responsive, but eventually willl inform the user of the problem and give them the option of cancelling the handler!

This means that long running client side code cannot be executed without implementing a continuation style approach.

Example 1
function clickHandler() {<br />process1();<br />process2();<br />process3();<br />}<br />could be replaced with:
Example 2
function longRunningProcess() {<br />runProcess1();<br />}<br /><br />function clickHandler() {<br />process1();<br />setTimeout(runProcess2(), 10);<br />}<br /><br />function runProcess2() {<br />process2();<br />setTimeout(runProcess3(), 10);<br />}<br /><br />function runProcess3() {<br />process3();<br />}<br />

This is horrible. However, the javascript library Narrative enables blocking capabilities for asynchronous event callbacks. This makes asynchronous code more readable.

So, it will convert the following code into Example 2:

<br />function longRunningProcess() {<br />process1();<br />sleep(10);<br />process2();<br />sleep(10);<br />process3();<br />}<br />
Fortunately, it is rare that the client code should ever have to execute long processes.

SetTimeout can appear to provide concurrent code execution. While SetTimeout is not part of the ECMA-262 language specification, it is still implemented within a single threaded environment.

Here is a test I performed to try and understand whats going on.

function() test1{
window.setTimeout( function(){alert('TimeOut function');},100);
for(var i=0; i<=10000000;i++) { if(i==10000000) {alert('long running process');} }} window.onload = Test1; 

In this example, the alert 'long running process' always executes before 'TimeOut function'.

From this test we can conclude that the anonymous function is not guaranteed to be executed when the time period passed to the setTimeout method has elapsed.

So whats happening?

As I pointed out previously, JavaScript is single threaded and as such can only ever execute one piece of code. This means that if an asynchronous event such as a mouse click or a setTimeout call to anonymous function, occurs during the excution of a code block (not necessarily an atomic call), the event is queued.

At the next available point during execution the queue will be queried and the next atomic code block executed.

However, if we remove the "if" statement from the loop in the test we get a different result.

function() test2 {
window.setTimeout( function(){alert('TimeOut function');},100);
for(var i=0; i<1000000;i++){alert(i);}}>

window.onload = test2; 

The "TimeOut function" can be alerted before, during and after the alerts within the loop, for example:

• 0
• 1
• 2
• 3
• 4
• 5
• setTimeOutCalled
• 6
• 7
• 8
• 9
• ...
  

Does this contradict our previous conclusions? No.

The javascript "alert" method is not responsible for creating the popup box that we see. The browsers UI thread does this. Which means that we have two threads, the javascript (containing our async queue) and the browsers thread for rendering UI. So, we cant guarantee the order we see the alerts, is the order that the javascript executes the alert statement

So, to truely understand whats going on, we need to try to remove interactions with the UI thread.

We will place our "alerts" into an array and render (note that I have used attached an add method to the Array constructor to help with manipulating arrays).

var var messages = [];
function test2_Revisited()  {
window.setTimeout( function(){Array.add(messages,"setTimeOutCalled");},100);
for(var i=0; i<1000000;i++) {Array.add(messages,i);}} 
window.onload = test2_Revisited; 

• 0
• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• ...
• 999999
  
• setTimeOutCalled

Conslusions

1. We cannot guarantee that the delay requested in a setTimeout will be honoured.
2. The seperate UI thread means that alerts and console.logs calls from javascript are not synchronised with the user experience.

For more information  see this excellent article written by Jphn Resig 

Javascript and Microsoft's JScript are compliant with the ECMA-262 language specification.

I looked at the specification available at http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf and it doesnt mention threading.

There are no keywords or intrinsic ECMAScript objects available to manage threads, or to provide for thread synchronization.

So, if threading is available in some or all browsers, the implementation is notcontrolled by this spec.

Javascript Engines

I couldnt find any details of the implementation of IE 7s Javascript engine, however, SpiderMonkey, the engine used by Firefox is documented at http://www.mozilla.org/js/spidermonkey/

The SpiderMonkey Javascript engine supports native threading!!

However, it appears that firefox only uses one JSContext for Javascript.

Without Threading, how does AJAX work?

It seems that many people blogging about AJAX mistake asynchronous requests for threading.

This is not the case. Consider the following:

I post an aysnchronous request to a web service. This request is associated with a callback function.

Since this request is asynchronous and is now being handled by the browser rather than the JavaScript engine, I can execute JavaScript code while this request is being processed by the server.

The code I choose to execute is a long running process and server will respond before the process is complete.

Does the asynchronous request and callback function execute in their own thread?

No. The callback function is not called until the long running process is complete even though the server responded quicker. Javascript queues asynchronous code when it is blocked.

Implementing our own threading model

There are libraries, written in Javascript, that imitate the programming conventions associated with threading, but these "threads" still cannot execute code concurrently.

http://www.neilmix.com/2007/02/07/threading-in-javascript-17/ uses "python style" generators introduced in Javascript 1.7

Conclusions:

1. Microsoft need to start documenting the inner workings of their code. No more blackboxes!!!
2. JavaScript does not support native threading.
   


3. Since we cannot guarantee the order of asynchronous events, coding 'functionally' and 'immutabily' is still useful even in this single threaded environment.
   
   We avoid the use of global variables since asynchronous code mutating the same variable can lead to unwanted "side effects".
   
   We should always have a good reason to mutate!