Unambiguous JavaScript Grammar

| Status | DRAFT | | --- | --- | | Authors | @jdalton @bmeck | | Date | June 14, 2016 |

Proposal Accepted!

On July 6, 2016 this proposal was accepted as section 5.1 of the Node ES6 Module Interoperability draft proposal :bangbang:

TL;DR

• CJS and ES modules just work without new extensions, extra ceremony, or excessive scaffolding
• Performance is generally on par with existing CJS module loading
• Performance is significantly improved for ES modules over transpilation workflows
• Change JS grammars for Script and Module to be unambiguous
• Determine grammar of

  .js

  files by parsing as one grammar and falling back to others

Problem

The Script and Module goal of ECMA262 have a grammatical ambiguity where some code can run in both goals, having the exact same source text, but produce different results. Unlike

"use strict"

Note: The following example highlights a few of the effects of ambiguous grammar and is by no means exhaustive.

Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);

Differences:

| | script | module | | ----------------------- | -------- | ----------- | | variable scope of

foo

arguments

this

foo

undefined

Since there is no way in source text to enforce the goal with the current grammar; this leads to the behavior of certain constructs being undefined by the programmer, and defined by the host environment. In turn, existing code could be run in the wrong goal and partially function, or function without errors but produce incorrect results.

ECMA262 Solution

Require that Module source text has at least one

import

export

import

export

export {}

import

export {}

Note: While the ES2015 specification does not forbid this extension, Node wants to avoid acting as a rogue agent. Node has a TC39 representative, @bmeck, to champion this proposal. A specification change or at least an official endorsement of this Node proposal would be welcomed. If a resolution is not possible, this proposal will fallback to the previous

.mjs

Script Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);

| | script | module (cannot parse) | | ----------------------- | -------- | --------------------- | | variable scope of

foo

arguments

this

foo

Module Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);
export {};

| | script (cannot parse) | module | | ----------------------- | --------------------- | -----------| | variable scope of

foo

arguments

this

foo

Problem

Node currently requires a means for programmers to signal what goal their code is written to run in.

Leading solutions have either hefty ecosystem tolls, ceremony, or scaffolding. They lack a way to define the intent of the source text from the ECMA262 standard.

Solution

A package opts-in to the Module goal by specifying

"module"

package.json

Algorithm

Parse (source, goal, throws)

The abstract operation to parse source text as a given goal.

1. Bootstrap

   source

   for

   goal

   .
2. Parse

   source

   as

   goal

   .
3. If success, return

   true

   .
4. If

   throws

   , throw exception.
5. Return

   false

   .

Operation

1. If a package parse goal is specified, then

   1. Let

      goal

      be the resolved parse goal.
   2. Call

      Parse(source, goal, true)

      and return.

2. Else fallback to multiple parse.

   1. If

      Parse(Source, Script, false)

      is

      true

      , then
   2. Return.
   3. Else
   4. Call

      Parse(Source, Module, true)

      .

Note: A host can choose either goal to parse first and may change their order over time or as new parse goals are introduced. Feel free to swap the order of Script and Module.

Implementation

To improve performance, host environments may want to specify a goal to parse first. This can be done in several ways:
cache on disk, a command line flag, a manifest file, HTTP header, file extension, etc.

Tooling Concerns

Some tools, outside of Node, may not have access to a JS parser (Bash programs, some asset pipelines, etc.). These tools generally operate on files as opaque blobs / plain text files and can use the techniques, listed under Implementation, to get parse goal information.

External Examples and Impact

• Esprima relies on a

  --module

  flag to signal the Module goal. However, this has proven to be unintuitive for many users. Unambiguous Script and Module goals would enable things to “just work” without flags.

• Facebook Flow performs a series of inferences to detect CJS and ES modules. Unambiguous Script and Module goals would improve its ability to determine module types.

• JSCS can accept input through stdin, so identification of parse goals by source text is ideal.

• Linters, like xo, could use unambiguous Script and Module goals to enable module specific linting rules without extra configuration.

• Microsoft packaged web applications can benefit from unambiguous Script and Module goals. The bytecode cache for a packaged web application is generated upon installation. When the application is running, files are loaded by script tags so their intended parse goals are understood. However, bytecode cache generation is done without running the application so the intended parse goals are unknown. Because of this, the bytecode cache is generated for the Script goal and ignored for ES modules.

• TypeScript has taken the stance from early on that a script becomes a module when it has at least one

  import

  or

  export

  declaration. Over the years they have experienced very few user issues with this approach.

Special Thanks

This proposal would not have been possible without the tireless effort, conviction, and collaboration of
@bmeck, @dherman, and @wycats.

While iterating on this proposal we have reached out to several people from areas affected by it.
Although opinions on this proposal are varied, I am grateful for feedback from:
* @AtOMiCNebula (Microsoft Edge) * @brendaneich (TC39) * @bterlson (Microsoft Chakra / TC39) * @chrisdickinson (Node) * @hzoo (Babel) * @indutny (Node) * @leobalter (jQuery Foundation / TC39) * @ljharb (TC39) * @loganfsmyth (Babel) * @mathiasbynens (Lodash) * @rvagg (Node) * @sheerun (Bower) * @sindresorhus (AVA / Chalk / Yeoman) * @travisleithead (Microsoft Edge / W3C) * @trevnorris (Node)

Thank you! :heart:
— @jdalton