I'll begin with a fundamental philosophical question. I'll state my position, but move on, because in many ways it's a matter of belief, even of faith. You either effectively accept it or not.

To me, and I know to many others, the web is something more, indeed much more than anything which runs in a browser. It is much more than something simply built with the technologies of the web. It is something open, interoperable, something which aims to reach as much of the world as possible.

Approaches to developing for the web which are inherently antithetical to these core principles of the web, are to me, not really about the web.

But as I said, these are issues of belief, and not what I want to focus on today.

Let's think for a moment about what we build, when we build things for the web

We build traditional web sites, still I'd argue the vast majority of the content of the web. These are document-centric, though may, and indeed increasingly do include interactive, application-like content.

We also are now frequently building more application-like, less document-like experiences. This as we know has been going on for quite some time, but for most of that time, much of our application logic has resided on the server, with the client being a largely dumb interface.

In recent years (though examples of this, like Google Maps and Gmail go back quite some way) our client-side applications have become more sophisticated, aided in particularly by the ability to send data rather than entire pages to and from the server.

In the last 3 or 4 years, with the success of mobile application platforms like iOS and Android, we've seen the rise in the expectation that particularly mobile applications which run in the browser will be more native-like.

And in response to this, a particular architecture has emerged to build these kind of native-like applications, the Single Page Application

Developers building these SPAs have found that there are particular challenges to developing them with web technologies. In particular, performance, and the ability to work offline.

Before we continue, I'd like to note that even if all of these concerns are completely true, and that there are no alternatives to addressing the shortcomings of traditional web technologies for building SPAs, we are still only focussing on

• a small subset of all the types of application we are building for the web
• a certain historical set of circumstances (currently poor browser engine in Android, lack of JIT compilers in webviews in iOS, first iteration of appcache)

I'm not saying the criticisms, the challenges that James has outlined are not real, not painful, and don't need solving.

It's understandable with all the frustration we have in solving a specific problem, we look to make something new, break the past's shackles, get it right this time.

But, I am suggesting we need to be very careful when looking to make profound changes to systems that have served us well for a long time now.

I fel this for two key reasons.

The trite one is, "the second system effect", as described by Fred Brooks in his classic of Software Engineering, The Mythical Man Month, which seems to apply very much to many of the suggestions we see in regards to how to do the web over.

The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one

The second, hopefully less trite response is rather than trying to solve a specific problem we have right now by throwing out many years of robust, solid, well thought out, well understood technologies and patterns, we should at least weigh very carefully what it is that will be gone if we just inject everything into the body.

I'm going to be quite up front here. Most of the rest of this presentation is inspired by a presentation by Rich Hickey, developer of the language Clojure, on the difference between something being easy and something being simple. You should find it and watch it. It's fantastic

(By the way, if you haven't picked up on the vibe yet, this is really a presentation about Software Engineering and the web.)

Hickey observes that we often conflate easiness with simplicity, but that these are quite profoundly different.

Simplicity (and it's obverse, complexity) is a feature of the thing we build, of our code. Is origins come from the latin meaning "single folded", as opposed to complexity, whose origins mean "many folded", or intertwined.

Easiness on the other hard is a subjective experience that we as developers have, unrelated to what we actually build.

As Dijkstra notes, simplicity is hard work. But absolutely vital to building reliable systems.

There have been a great many quotations about simplicity, we all know many, from Einstein, Leonardo, Frank Lloyd Wright, but I'll leave the last word to the master, Bruce Lee. "It is not a daily increase, but a daily decrease. Hack away at the inessentials"

• We've known for a long time that complex systems become increasingly expensive, take increasingly longer to build, often are never completed
• we know adding new features is difficult
• we know maintaining them is hard

And I'm going to argue that we've got away with not paying enough attention to issues like complexity for too long when we build for the web, particularly on the client side for a number of reasons

• For a long time, what we've been building has been intrinsically quite simple
• A lot of what we build is quite trivial (these are not the same thing), so their complexity, and its implications have been hidden
• We've long been used to throwing things out and starting again, rather than having to maintain systems over long periods of time

In the world of traditional software development, you simply can't get away with complexity. Many systems are decades old. How many of the things we build have even lasted years?

For most of the history of the web, the tools we've used to develop with have been quite simple

• We've used HTML for our content
• We've used CSS to style that content
• We've used JavaScript and the DOM to add behavior

But using these technologies isn't necessarily easy

• Sometimes because of the underlying ways in which they work. For instance CSS works quite differently from almost any technology most developers will be familiar with. We treat this difference as a bug, which we try to solve with technologies like SASS, which I'll return to a little later
• Sometimes, because the implementations of these technologies is poor, or inconsistent (if there'd only ever been IE6, developing for the web would doubtless be easier)

So, over time we've added layers of technology and practice to help address the difficulty of developing for the web. Now, not all of these are "bad", but all of these add complexity.

We stumbled on UA sniffing, which made addressing the challenge of cross browser inconsistency easier to deal with.

We developed CSS hacks, to target different browsers with the same CSS code base. These are largely no longer needed, but add complexity to learning and maintaining code bases We added doctype switching, to allow us to develop a single CSS code base that would work in older browsers and newer ones which supported a standards based implementation of CSS and HTML.

all of these added a complexity that continues to impact years later. They make it harder to learn to be a web developer, they make our code larger, more difficult to read and to maintain.

but they did address what we considered at the time fundamental challenges. Pages needed to look as nearly identical as possible across all browsers.

This is a belief we now no longer hold strongly, and indeed as we address the challenges and opportunities of fragmented viewing contexts, from small mobile devices, to tablets, to laptops and desktops, to high resolution monitors to True HD TVs, we have rejected it.

All of this complexity, which still haunts us today was in a way about premature optimisation. Complexity has a long half life, and we need to be very careful about adding it to what we do.

With the rise of web applications around a decade ago, we encountered the challenge of inconsistent DOM implementations (we'd already discovered these problems years earlier with DHTML, but they re emerged as what we tried to do in the browser became more complex, and as browsers other than Internet Explorer gained some sort of traction).

Prototype, Dojo, Mootools, JQuery, and other monolithic JavaScript libraries emerged to help us manage the challenges of these inconsistent implementations. They also added inconsistent features to address the shortcomings of the DOM, and JavaScript. They made our lives easier. but they added complexity as well.

• On top of learning JavaScript and the DOM, we needed to learn the features, concepts and often new syntaxes of these libraries
• We needed to manage versioning issues
• We needed to start considering their impact on performance, and so developed minifying as a way of reducing their impact on downloads

And increasingly meta toolsets (e.g. Yeoman. HTML5 Boilerplate, Mobile BoilerPlate) for helping us work with all these other toolsets

And to even use them you need Ruby on Rails, or Node.js set up

And this is just for client side development!

And I think we are travelling along a very slippery path of ever increasing complexity, in service of ostensible ease.

Because I'm old, and having been around the block a few times, I frequently come back to this observation by George Santayana

Progress, far from consisting in change, depends on retentiveness. When change is absolute there remains no being to improve and no direction is set for possible improvement: and when experience is not retained…, infancy is perpetual. Those who cannot remember the past are condemned to repeat it

The term "Software Crisis" was originally coined used at first NATO Software Engineering Conference. in 1968

Some years later, Dijkstra wrote

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.

Edsger Dijkstra, The Humble Programmer (EWD340), Communications of the ACM

You should read The Humble Programmer, it's not too long, and blends Dijkstra's personal story, with that of the infancy of our industry (Software Engineering) and deeper ideas about our wat it is we do.

As we saw, Dijkstra's counter intuitive response to this was to simplify (not to make things easier)

How did the software engineering world respond to this?

Perhaps the key principle, again a term coined by Dijkstra was the "separation of concerns"

"Separation of concerns (SoC) is the process of breaking a computer program into distinct features that overlap in functionality as little as possible. A concern is any piece of interest or focus in a program. Typically, concerns are synonymous with features or behaviors. Progress towards SoC is traditionally achieved through modularity and encapsulation, with the help of information hiding.

Sound familiar? You know that thing some web folks have been banging on about since the 1990s? HTML for content, CSS for presentation, JavaScript for Behaviour Not only was this good practice, it was baked directly into core technologies of the web

When we use HTML, CSS, JS in the way they were designed, we get separation of concerns out of the box. For free.

yet we do things like this

And this

And this

• and we have
• • "web sites" like ReadWrite Web that have 19 iframes on a single page
  • and load up to 20 scripts!
  • and uses Bootstrap
• it's just a website people

• Now, this is not simply about methodological purity. Complexity has costs.

learnability

For me, the first challenge for computing science is to discover how to maintain order in a finite, but very large, discrete universe that is intricately intertwined. And a second, but not less important challenge is how to mould what you have achieved in solving the first problem, into a teachable discipline: it does not suffice to hone your own intellect …, you must teach others how to hone theirs. … teaching yourself is discovering what is teachable.

you guessed it (yeah Dijkstra)

• Once upon a time, we could incrementally approach the challenges of learning to develop for the web.
• • Learn the principles of markup with HTML
  • Learn CSS, and master it over time
  • Learn JavaScript and DOM programming
• we could view source, and work out how something had been built

That's not to say this is perfect, but to observe this is something that has worked well for a long time. We discard such things at our peril

• With the SPA style of development
• • developers need to be increasingly very good at JavaScript, which in itself is not a bad thing,
  • but also have strong knowledge of specific frameworks, build tools, and on, and on

when concerns were strictly separated - markup, presentation, behavior, then working on each in isolation, with specialized skills or areas of focus was far easier than having these concerns conflated

build cycles (WTF!)

Maybe I'm old fashioned, but the ability to simply drop a file onto a browser, and refresh a window to rebuild is one of the pure joys of working on the web. As someone who comes from (and ocasionally still experiences) the edit compile run, cycle, the immediacy of developing for the web with nothing more than HTML, CSS and JavaScript files is frankly nearly miraculous

An don't get me started on installing and keeping up to date build tools, ruby gems, preprocessors, …

But ironically, here I've fallen into Rich Hickey's trap. I'm talking about ease, and process, not the simplicity or complexity of what we are building. So let's turn to this issue, and the impact of JavaScript everywhere on the artefacts we're building

debuggability

• Developers make simple mistakes like typographical errors
• We do complex things
• Really difficult bugs to fix are subtle. They don't throw syntax errors. They just do weird stuff
• Really complex bugs are difficult to reproduce, because of the stateful nature much of development. What this means is the same code can produce different outcomes based on the state of the system its applied to.

debuggability

As developers, we've long been frustrated with CSS errors and bugs. But at least the bugs are reproducible and deterministic. Same HTML + Same CSS gives the same DOM and values.

With CSS and HTML, discovering why this element has these properties set to these values is relatively straightforward. Our tools are well adapted to this problem, but the truth is the problem really isn't complicated.

But

Now, there are times when we will need to change the DOM programmatically. But I'd argue that we should resort to this only when it is essentially impossible to do with CSS and HTML alone.

And in order to make that decision, you really need to understand HTML and CSS well. Not just their syntax, but their essence.

Mastery

to obtain the deepest benefit from learning a new language, you must learn to think in the new language, not just learn to translate your favourite programming language syntax and idioms into it.

(Not Dijkstra) Eugene Wallingford

• Of late, there's been an explosion in languages which target JavaScript. CoffeeScript being the recent dominant player.
• Which introduces significant challenges in each of these areas (learnability - can you just know CoffeeScript and not JavaScript?) but in particular debuggability.
• Without sourcemaps (more complexity) the challenging of debugging returns to the stone age of console.log
• The tools we've built, and practices we've developed over years to help make building web sites and applications more robust, are obviated overnight.

In and of themselves, each of these innovations - CoffeeScript, SASS, various MVC frameworks, various tools may appear to bring much, and the case can be made for exchanging ease for some additional, but manageable complexity.

but as we pile innovation on innovation, we have an increasingly fragmented, non interoperable, fragile stack of technologies that rely on one another (I need node.js to use yeoman to give me ...

And there's a far bigger issue lurking than just debuggability.

On the web, particularly the front end, we're used to throwing things away and starting over again with alarming frequency.

We build all kinds of one off, essentially disposable apps - throw it against the wall, see if it has traction.

And let's face it, in particular the "native like" SPAs we build often don't do all that much.

But as more and more of what was traditionally desktop software becomes web based, this luxury of build to dispose will soon pass.

It's well known in software engineering that rewriting applications, rather than incrementally improving them is highly corelated with negative outcomes. Some of the most significant applications of their day have essentially vanished because of massive rewrites.

In the Web, this is precisely what happened with Netscape, ceding Internet Explorer almost complete dominance of the browser for nearly a decade.

You might argue that ultimately Firefox demonstrates that this was the right response, but essentially that was a new application, not a new version of Netscape.

• are we
• • building for this maintainability?
  • choosing technologies that will be around for a long time?
  • choosing technologies which are widely understood, and will be for the foreseeable future?

• or are we
• • forever chasing the new?
  • failing to learn from the past?
  • condemning ourselves to "perpetual infancy"?

BTW, everything I'm talking about applies, I think not just to the issue of building everything directly with JS, it also applies to

languages which compile to JS solutions to the problems with CSS, like LESS and SASS

basically, what we are trying to do, is shoe horn into these languages, CSS and JavaScript ideas, idioms, principles that aren't core to them.

Remember Eugene Wallingford's observation

... to obtain the deepest benefit from learning a new language, you must learn to think in the new language, not just learn to translate your favourite programming language syntax and idioms into it.

And what we are really missing is one of the genuinely revolutionary aspects of developing for the web.

Its declarative nature (particularly HTML and CSS).

And what we are really missing is one of the genuinely revolutionary aspects of developing for the web.

• With imperative programming we have
• • statefulness
  • variables
  • side effects
• It is about specifying how the computer should perform a task

• With declarative programming we have
• • statelessness
  • no side effects
  • deterministic outcomes
• It is about specifying what tasks a computer should perform
• It's associated with more robust, maintainable systems

And we want to throw all this out? Essentially for our own convenience?

An example: SASS

Just to prove it's not all about JavaScript,

• CSS is a declarative programming paradigm
• • you state what you want to happen in various conditions
  • when those conditions occur, the browser applies the CSS
  • there are no side effects
  • the same CSS on the same DOM tree produces the same outcomes

SASS

• We have variables
• We have loops
• we have calculations

• In essence, we have state
• We have turned a declarative programming language into an imperative one

Why?

• For Convenience
• To make our lives easier
• We've made our applications
• • harder to debug
  • less readable
  • less maintainable
• Not to mention the setup and maintenance of our build systems

Are we insane? I'm quite serious, are we fucking insane?

Easiness versus Complexity

We now have

1. a total lock-in into the future to one non-standard even if widely adopted technology
2. a vastly diminished pool of people who can work on this project
3. significantly increased complexity in debugging
4. significantly increased development environment complexity

not to mention increasing the complexity of our field.

and for what?

OK, enough of the negativity

This a is not a call to do nothing new, to not innovate in the areas I've been discussing.

Rather, it's a call to start appreciating the real nature of what we are doing

Increasingly, the web is the platform for enterprise, consumer, client server, mobile, pretty much all kinds of application

While the web's long status among many as a toy, at best a publishing medium has also given us the upside of not having the harsh glare of rigorous criticism being turned on what we do.

And that's not going to last.

The things we build are going to be required to do more and more. Not just additional functionality, but last longer, be more robust and secure and performant.

We have to grow up, and take these challenges seriously

The good news

• We already have the foundations
• • HTML
  • CSS
  • JavaScript
  • The DOM
  • The increasing sophistication of javascript engines, and just in time compilers
  • The increasing power of HTML5 and other device apis and capabilities.

Now is not the time to turn our backs on these. To wish we could do the web over (again and again). To boil the ocean.

Good engineering is about making well researched and reasoned decisions about the tools and technologies we use, for the specific purpose we are using them for.

Which means really understanding these technologies

Not relying on tools and scaffolding and syntactic sugar.

And now is not the time to seek out easiness, at the price of the increasing complexity of the systems we build.

• Development is much more than writing code
• Reducing the number of keystrokes may feel like we are being more efficient, but it is measuring the wrong thing
• We need to be measuring the complexity of our systems
• We need to be measuring the robustness of our systems

• And now is not the time to seek out easiness, at the price of the increasing complexity of the systems we build.
• It is the time to double down on them, continue to improve them, all the while understanding what makes them so powerful
• It's time for the web to take its place at the grown ups table.
• And it's the responsibility of people like us to make that happen
• Let's not throw that responsibility, and opportunity away.