Neon: Node + Rust = 💖

Dec 23rd, 2015

If you’re a JavaScript programmer who’s been intrigued by Rust’s hack without fear theme—making systems programming safe and fun—but you’ve been waiting for inspiration, I may have something for you! I’ve been working on Neon, a set of APIs and tools for making it super easy to write native Node modules in Rust.

TL;DR:

  • Neon is an API for writing fast, crash-free native Node modules in Rust;
  • Neon enables Rust’s parallelism with guaranteed thread safety;
  • Neon-cli makes it easy to create a Neon project and get started; and finally…
  • Help wanted!

I Can Rust and So Can You!

I wanted to make it as easy as possible to get up and running, so I built neon-cli, a command-line tool that lets you generate a complete Neon project skeleton with one simple command and build your entire project with nothing more than the usual npm install.

If you want to try building your first native module with Neon, it’s super easy: install neon-cli with npm install -g neon-cli, then create, build, and run your new project:

% neon new hello
...follow prompts...
% cd hello
% npm install
% node -e 'require("./")'

If you don’t believe me, I made a screencast, so you know I’m legit.

I Take Thee at thy Word

To illustrate what you can do with Neon, I created a little word counting demo. The demo is simple: read in the complete plays of Shakespeare and count the total number of occurrences of the word “thee”. First I tried implementing it in pure JS. The top-level code splits the corpus into lines, and sums up the counts for each line:

1
2
3
4
5
6
7
8
function search(corpus, search) {
  var ls = lines(corpus);
  var total = 0;
  for (var i = 0, n = ls.length; i < n; i++) {
    total += wcLine(ls[i], search);
  }
  return total;
}

Searching an individual line involves splitting the line up into word and matching each word against the search string:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
function wcLine(line, search) {
  var words = line.split(' ');
  var total = 0;
  for (var i = 0, n = words.length; i < n; i++) {
    if (matches(words[i], search)) {
      total++;
    }
  }
  return total;
}

The rest of the details are pretty straightforward but definitely check out the code—it’s small and self-contained.

On my laptop, running the algorithm across all the plays of Shakespeare usually takes about 280 – 290ms. Not hugely expensive, but slow enough to be optimizable.

Fall Into our Rustic Revelry

One of the amazing things about Rust is that highly efficient code can still be remarkably compact and readable. In the Rust version of the algorithm, the code for summing up the counts for all the lines looks pretty similar to the JS code:

1
2
3
4
5
6
7
let mut total = 0;
for word in line.split(' ') {
    if matches(word, search) {
        total += 1;
    }
}
total // in Rust you can omit `return` for a trailing expression

In fact, that same code can be written at a higher level of abstraction without losing performance, using iteration methods like filter and fold (similar to Array.prototype.filter and Array.prototype.reduce in JS):

1
2
3
line.split(' ')
    .filter(|word| matches(word, search))
    .fold(0, |sum, _| sum + 1)

In my quick experiments, that even seems to shave a few milliseconds off the total running time. I think this is a nice demonstration of the power of Rust’s zero-cost abstractions, where idiomatic and high-level abstractions produce the same or sometimes even better performance (by making additional optimizations possible, like eliminating bounds checks) than lower-level, more obscure code.

On my machine, the simple Rust translation runs in about 80 – 85ms. Not bad—about 3x as fast just from using Rust, and in roughly the same number of lines of code (60 in JS, 70 in Rust). BTW, I’m being approximate here with the numbers, because this isn’t a remotely scientific benchmark. My goal is just to demonstrate that you can get significant performance improvements from using Rust; in any given situation, the particular details will of course matter.

Their Thread of Life is Spun

We’re not done yet, though! Rust enables something even cooler for Node: we can easily and safely parallelize this code—and I mean without the night-sweats and palpitations usually associated with multithreading. Here’s a quick look at the top level logic in the Rust implementation of the demo:

1
2
3
4
5
6
7
let total = vm::lock(buffer, |data| {
    let corpus = data.as_str().unwrap();
    let lines = lines(corpus);
    lines.into_iter()
         .map(|line| wc_line(line, search))
         .fold(0, |sum, line| sum + line)
});

The vm::lock API lets Neon safely expose the raw bytes of a Node Buffer object (i.e., a typed array) to Rust threads, by preventing JS from running in the meantime. And Rust’s concurrency model makes programming with threads actually fun.

To demonstrate how easy this can be, I used Niko Matsakis’s new Rayon crate of beautiful data parallelism abstractions. Changing the demo to use Rayon is as simple as replacing the into_iter/map/fold/ lines above with:

1
2
3
lines.into_par_iter()
     .map(|line| wc_line(line, search))
     .sum()

Keep in mind, Rayon wasn’t designed with Neon in mind—its generic primitives match the iteration protocols of Rust, so Neon was able to just pull it off the shelf.

With that simple change, on my two-core MacBook Air, the demo goes from about 85ms down to about 50ms.

Bridge Most Valiantly, with Excellent Discipline

I’ve worked on making the integration as seamless as possible. From the Rust side, Neon functions follow a simple protocol, taking a Call object and returning a JavaScript value:

1
2
3
4
5
fn search(call: Call) -> JS<Integer> {
    let scope = call.scope;
    // ...
    Ok(Integer::new(scope, total))
}

The scope object safely tracks handles into V8’s garbage-collected heap. The Neon API uses the Rust type system to guarantee that your native module can’t crash your app by mismanaging object handles.

From the JS side, loading the native module is straightforward:

1
var myNeonModule = require('neon-bridge').load();

Wherefore’s this Noise?

Hopefully this demo is enough to get people interested. Beyond the sheer fun of it, I think the strongest reasons for using Rust in Node are performance and parallelism. As the Rust ecosystem grows, it’ll also be a way to give Node access to cool Rust libraries. Beyond that, I’m hoping that Neon can make a nice abstraction layer that just makes writing native Node modules less painful. With projects like node-uwp it might even be worth exploring evolving Neon towards a JS-engine-agnostic abstraction layer.

There are lots of possibilities, but I need help! If you want to get involved, I’ve created a community slack (grab an invite from the Slackin app) and a #neon IRC channel on Mozilla IRC (irc.mozilla.org).

A Quick Thanks

There’s a ton of fun exploration and work left to do but I couldn’t have gotten this far without huge amounts of help already: Andrew Oppenlander’s blog post got me off the ground, Ben Noordhuis and Marcin Cieślak helped me wrestle with V8’s tooling, I picked up a few tricks from Nathan Rajlich’s evil genius code, Adam Klein and Fedor Indutny helped me understand the V8 API, Alex Crichton helped me with compiler and linker arcana, Niko Matsakis helped me with designing the safe memory management API, and Yehuda Katz helped me with the overall design.

You know what this means? Maybe you can help too!

Why Mozlandia Was So Effective

Dec 8th, 2014

When Chris Beard first announced that over a thousand Mozilla staff and contributors would be descending on Portland this month for an all-hands work week, I worried about two things. I knew a couple of the groups in my department would be approaching deadlines. And I was afraid that so many groups of people in one place would be chaotic and hard to coordinate. I wasn’t even wrong – but it didn’t matter.

The level of focus and effectiveness last week was remarkable. For Mozilla Research’s part, we coordinated with multiple groups, planned 2015 projects, worked through controversial technical decisions, removed obstacles, brought new contributors on board, and even managed to get a bunch of project work done all at the same time.

There were a few things that made last week a success:

Articulating the vision: Leaders have to continually retell their people’s story. This isn’t just about morale, although that’s important. It’s about onboarding new folks, reminding old-timers of the big picture, getting people to re-evaluate their projects against the vision, and providing a group with the vocabulary to help them articulate it themselves.

While Portland was primarily a work week, it’s always a good thing for leadership to grab the opportunity to articulate the vision. This is something that Mitchell Baker has always been especially good at doing, particularly in connecting our work back to Mozilla’s mission; but Chris and others also did a good job of framing our work around building amazing products.

Loosely structured proximity: The majority of the work days were spent without excessive organization, leaving broad groups of people in close proximity but with the freedom to seek out the specific contact they needed. Managers were able to set aside quieter space for the groups of people that needed to get more heads down work done, but large groups (for example, most of Platform) were close enough together that you could find people for impromptu conversations, whether on purpose or – just as important! – by accident.

Cross-team coordination: Remote teams are the life blood of Mozilla. We have a lot of techniques for making remote teams effective. But it can be harder to coordinate across teams, because they don’t have the same pre-existing relationships, or as many opportunities for face-to-face interaction. Last week, Mozilla Research got a bunch of opportunities to build new relationships with other teams and have higher-bandwidth conversations about tricky coordination topics.

I hope we do this kind of event again. There’s nontrivial overhead, and a proper cadence to these things, but every once in a while, getting everyone together pays off.

I’m Running for the W3C TAG

Jan 2nd, 2014

The W3C’s Technical Architecture Group (TAG) has two open seats for 2014, and I’m running for one of those seats.

In recent years a reform effort has been underway to help the TAG to improve the cohesiveness and transparency of the many moving parts of Web standards. Domenic Denicola and I would like to help continue that reform process. My particular interests in running focus on several themes:

Designing for Extensibility

I’m an original co-signer of the Extensible Web Manifesto, which urges Web standards to focus on powerful, efficient, and composable primitives, in order to allow developers — who are far more efficient and scalable than standards can ever be — to innovate building higher layers of the platform. The TAG has recognized the Extensible Web as a core principle. We need to build on this momentum to continue educating people about how the principles play out in practice for designing new APIs and platform capabilities that empower developers to extend the web forward.

Thinking Big and Working Collaboratively

For the Web to compete with native platforms, I believe we have to think big. This means building on our competitive strengths like URLs and dynamic loading, as well as taking a hard look at our platform’s weaknesses — lack of access to modern hardware, failures of the offline experience, or limitations of cross-origin communication, to name a few. My entire job at Mozilla Research is focused on thinking big: from ES6 modules to asm.js and Servo, my goal is to push the Web as far forward as possible. I’m running for TAG because I believe it’s an opportunity to set and articulate big goals for the Web.

At the same time, standards only work by getting people working together. My experience with open source software and standards work — particularly in shepherding the process of getting modules into ES6 — has taught me that the best way to build community consensus is the layers of the onion approach: bring together key stakeholders and subject experts and iteratively widen the conversation. It’s critical to identify those stakeholders early, particularly developers. Often we see requests for developer feedback too late in the process, at which point flawed assumptions are too deeply baked into the core structure of a solution. The most successful standards involve close and continuous collaboration with experienced, productive developers. Pioneers like Yehuda Katz and Domenic Denicola are blazing trails building better collaboration models between developers and platform vendors. Beyond the bully pulpit, the TAG should actively identify and approach stakeholders to initiate important collaborations.

Articulating Design Principles

When Alex Russell joined the TAG, he advocated for setting forth principles for idiomatic Web API design. We can do this in part by advising standards work in progress, which is the ongoing purview of the TAG. Web API creators are often browser implementors, who are under aggressive schedules to ship functionality, and don’t always have the firsthand experience of using the API’s they create. Worse yet, they sometimes break key invariants of JavaScript that the creators, who are often primarily C++ programmers, didn’t understand. One area of particular concern to me is data races: several API’s, including the File API and some proposed extensions to WebAudio introduce low-level data races into JavaScript, something that has been carefully avoided since the run-to-completion model was introduced on Day 1.

And there’s room to lead more proactively still. One area I’d like to help with is in evolving or reforming WebIDL, which is used by browser vendors to specify and implement Web API’s, but which carries a legacy of more C++- and Java-centric API’s. Several current members of TAG have begun investigating alternatives to WebIDL that can provide the same convenience for creating libraries but that lead to more idiomatic API’s.

If you’re a developer who finds my perspective compelling, I’d certainly appreciate your public expression of support. If you belong to a voting member organization, I’d very much appreciate your organization’s vote. I also highly recommend Domenic Denicola as the other candidate whose vision and track record are most closely aligned with my own. Thanks!

On “On Asm.js”

Nov 27th, 2013

On his impossibly beautiful blog (seriously, it’s amazing, take some time to bask in it), Steven Wittens expressed some sadness about asm.js. It’s an understandable feeling: he compares asm.js to compatibility hacks like UTF-8 and x86, and longs for the browser vendors to “sit down and define the most basic glue that binds their platforms”—referring to a computational baseline that could form a robust and portable VM for the web.

I get it: it’s surprising to see a makeshift VM find its way into the web via JavaScript, rather than through the perhaps more direct approach of a new bytecode language and standardization effort. What’s more, it has become clear to me that emotions always run high when it comes to JavaScript. It’s easy for observers to suspect that what we’re doing is the result of a blind fealty to JavaScript. But the fact is, our strategy has nothing to do with the success of JavaScript. It’s about the success of the web. On a shared medium like the web, where content has to run across all OSes, platforms, and browsers, backwards-compatible strategies are far more likely to succeed than discrete jumps. In short, we’re betting on evolution because it works: UTF-8 and x86 may be ugly hacks, but the reason we’re talking about them at all is that they’re success stories.

There’s more work to be done, but between sweet demos, rapid improvements in browser performance, and the narrowing gap to native via float32 and SIMD, I see plenty of reason to keep betting on evolution. The truth is, in my heart I’m an idealist. I love beautiful, clean designs done right from scratch. (I spent my academic years working in Scheme, after all!) But my head tells me that this is the right bet. In fact, I’ve spent my career at Mozilla betting on evolution: growing JavaScript with modules and classes, leveling up the internal architecture of browser engines with Servo, and kicking the web’s virtualization powers into high gear with asm.js.

So for developers like Steven who are put off by the web’s idiosyncratic twists of fate, let’s keep working to build better abstractions to extend the web forward. In particular, in 2014 I want to invest in LLJS, as James Long has been doing in his spare time, to build better developer tools for generating high-performance code—and asm.js can be our stepping stone to get there.

ECMAScript Doc Sprint next Thursday

Aug 17th, 2012

I’ve been working on a reboot to the ECMAScript web site lately, which you can preview at tc39wiki.calculist.org. One of the most important parts of this will be a set of high-level descriptions of the proposals for ES6.

We will be hosting a virtual doc sprint to work on these pages next Thursday, August 23rd. If you enjoy writing documentation or coming up with bite-sized example programs to demonstrate new language features, please join us! A few of us will be on US Eastern time, so starting around 9 - 10am UTC-5, and others will be coming online on US Pacific time, around 9am UTC-8. You’re welcome to join us for any part of the day.

We’ll be hanging out all day in the #jsdocs channel on irc.mozilla.org. Hope you can join us!