| Status | DRAFT |
|---|---|
| Authors | @jdalton @bmeck |
| Date | June 14, 2016 |
- CJS and ES modules just work without new extensions, extra ceremony, or excessive scaffolding
- Performance is generally on par or better than existing CJS module loading
- Performance is significantly improved for ES modules over transpilation workflows
- Change JS grammars for Script and Module to be unambiguous
- Determine grammar of
.jsfiles by parsing as one grammar and falling back to others
The Script and Module goal of ECMA262 have a grammatical ambiguity where some
code can run in both goals, having the exact same source, but produce different
results. Unlike "use strict", the signal to have a specific behavior is not in
the code, thus the code has a multitude of possible effects which are not
controlled by the programmer.
function foo(value) {
value = value || '';
var args = [].slice.call(arguments);
args.unshift(this);
return args;
}
foo(null);Differences:
| script | module
--- | --- | --- | ---
variable scope of foo | global | local
arguments object | modified | unmodified
this binding of foo | global | undefined
Since there is no way in source text to enforce the goal with the current grammar; this leads to the behavior of certain constructs being undefined by the programmer, and defined by the host environment. In turn, existing code could be run in the wrong goal and partially function, or function without errors but produce incorrect results.
Require a structure in the Module goal that does not parse in the Script goal. Having this requirement would prevent any source text written for the Module goal from being executed in the Script goal by removing the ambiguity at parse time.
While the ES2015 specification does not forbid this extension, Node wants to avoid further ecosystem fragmentation. This proposal is on the agenda for the July 2016 ECMA TC39 meeting.
The proposal is to require that Module source text has at least one import or
export declaration. This should feel natural to developers as most modules import
dependencies and or export APIs. A module with only an import declaration and
no export declaration is valid. However, it is our recommendation that modules
are explicit with export. Modules, that do not export anything, should specify
an export {} to make their intentions clear and avoid accidental parse errors
while removing import declarations.
Note: The export {} is not new syntax and does not export an empty object.
It is the standard ES2015 way to specify exporting nothing.
function foo(value) {
value = value || '';
var args = [].slice.call(arguments);
args.unshift(this);
return args;
}
foo(null);| script | module (cannot parse)
--- | --- | --- | ---
variable scope of foo | global | n/a
arguments object | modified | n/a
this binding of foo | global | n/a
function foo(value) {
value = value || '';
var args = [].slice.call(arguments);
args.unshift(this);
return args;
}
foo(null);
export {};| script (cannot parse) | module
--- | --- | --- | ---
variable scope of foo | n/a | local
arguments object | n/a | unmodified
this binding of foo | n/a | undefined
Node currently requires a means for programmers to signal what goal their code is written to run in.
Leading solutions have either hefty ecosystem tolls, ceremony, or scaffolding. They lack a way to define the intent of the source text from the ECMA262 standard.
A package opts-in to the Module goal by specifying "module" as the parse goal
field (name not final) in its package.json. Package dependencies are not affected
by the opt-in and may be a mix of CJS and ES module packages. If a parse goal is
not specified, then parse source text as either goal. If there is a parse error
that may allow the other goal to parse, then parse as the other goal. After this,
the goal is known unambiguously and the environment can safely perform initialization
without the possibility of the source text being run in the wrong goal.
The abstract operation to parse source text as a given goal.
- Bootstrap
sourceforgoal. - Parse
sourceasgoal. - If success, return
true. - If
throws, throw exception. - Return
false.
-
If a package parse goal is specified, then
-
Let
goalbe the resolved parse goal. -
Call
Parse(source, goal, true)and return. -
Fallback to multiple parse.
-
If
Parse(Source, Script, false)istrue, then 1. Return. -
Else 1. Call
Parse(Source, Module, true).
Note: A host can choose either goal to parse first and may change their order over time or as new parse goals are added. Feel free to swap the order of Script and Module.
To improve performance, host environments may want to specify a goal to parse
first. This could take several forms:
a command line flag, a manifest file, HTTP header, file extension, etc.
The recommendation for Node is to store this in a cache on disk.
Both @trevnorris and @indutny believe caching is doable. Caching removes the sting of parsing and can actually improve on existing performance through techniques like bytecode caching. While investigations are in their early stages, there is plenty of room for improvements and optimizations in this space.
The workflow for loading files could look like:
- Get path to load as
filename. - If cache has
filename, then - Set
goalfrom cached value. - Validate cache against file for
filename. - If valid, return cache.
- Else
- Set
goalto a preferred goal (Script for now since most modules are CJS) - Load file for
filenameassource. - Bootstrap source for
goalasbootstrapped_source. - Parse
bootstrapped_sourceusinggoalgrammar. - If success, then
- Cache and return results.
- Else
- Change
goalto opposite grammar. - Bootstrap source for
goalasbootstrapped_source. - Parse
bootstrapped_sourceusinggoalgrammar. - If success, cache and return results.
- Throw error.
Some tools, outside of Node, may not have access to a JS parser (Bash programs, some asset pipelines, etc.). These tools generally operate on files as opaque blobs / plain text files and can use the noted methods, listed under Implementation, to get information about file grammar.
-
Facebook Flow performs a series of inferences to detect CJS and ES modules. Unambiguous Script and Module goals would improve its ability to determine module types.
-
Microsoft packaged web applications can benefit from unambiguous Script and Module goals. The bytecode cache for a packaged web application is generated upon installation. When the application is running, files are loaded by script tags so their intended parse goals are understood. However, bytecode cache generation is done without running the application so the intended parse goals are unknown. Because of this, the bytecode cache is generated for the Script goal and ignored for ES modules.
This proposal would not have been possible without the tireless effort, conviction,
and collaboration of
@bmeck, @dherman,
and @wycats.
While iterating on this proposal we have reached out to several people from areas
affected by it.
Although opinions on this proposal are varied, I am grateful for
feedback from:
- @AtOMiCNebula (Microsoft Edge)
- @brendaneich (TC39)
- @bterlson (Microsoft Chakra / TC39)
- @chrisdickinson (Node)
- @hzoo (Babel)
- @indutny (Node)
- @leobalter (jQuery Foundation / TC39)
- @ljharb (TC39)
- @loganfsmyth (Babel)
- @mathiasbynens (Lodash)
- @rvagg (Node)
- @sheerun (Bower)
- @sindresorhus (AVA / Chalk / Yeoman)
- @travisleithead (Microsoft Edge / W3C)
- @trevnorris (Node)
Thank you! ❤️
— @jdalton