Editorial: support built-in async functions

[michaelficarra]

This adds support for built-in async functions, as needed by the async iterator helpers and Array.fromAsync proposals.

[jmdyck]

This says that a built-in function is exotic (and synchronous) unless otherwise specified. So then a built-in would be ordinary (i.e., an ECMAScript function object) only if specified to be so. But, of course, none of them are, because the spec explicitly says that built-ins can be ordinary or exotic.

[bakkot]

@jmdyck I don't understand this comment. I think I'm missing some assumption. Can you rephrase?

[jmdyck]

• 10.3 "Built-in Function Objects" says that built-in functions may be implemented as either ECMAScript function objects or as exotic function objects.

• 10.2 "ECMAScript Function Objects" says that an ECMAScript function object has the [[Call]] internal method defined in 10.2.1.

• 10.3 "Built-in Function Objects" says (in the status quo) that a built-in function that isn't implemented as an ECMAScript function object must have the [[Call]] internal method defined in 10.3.1.

So, in the status quo, a built-in's [[Call]] can be either 10.2.1 or 10.3.1, and the spec leaves the choice to the implementation.

• This PR replaces the third point with a sentence that says a built-in function object must use 10.3.1 "unless otherwise specified".

• But for all of the existing built-in functions, the spec never specifies whether to use 10.3.1 or 10.2.1 (because it's leaving that choice to the implementation), so it is never "otherwise specified", so the "unless otherwise specified" clause never holds.

So this new sentence implies that all existing built-in functions must use 10.3.1.

[jmdyck]

(And I know that, at some point, the idea was floated to get rid of the 10.2.1 possibility for built-ins, but that's clearly not what this PR is trying to accomplish. That would presumably be a separate PR.)

[michaelficarra]

This is not an observable difference. I am okay with it. Besides, we wanted to eliminate all the implications (and explicit statements) that "built-ins can be ECMAScript code, too" because it doesn't matter.

[michaelficarra]

Resolution from editor call is to change the definition of ordinary object to explicitly allow these [[Call]] and [[Construct]] methods so they're all considered ordinary for now. We'll do this in a separate PR.

[michaelficarra]

Resolved in #2969.

[jmdyck]

#2969 expands the definition of "ordinary function" to include all built-in functions (however implemented). But I don't think that resolves this, because my concern doesn't hinge on the definition of "ordinary function". If you look at my response to bakkot above, it all still holds in a post-2969 spec (except that "exotic" in the first point now becomes "non-ECMAScript").

(Yes, it was during discussion of this concern that the editors call decided to expand the definition of "ordinary", but maybe that was just to deal with the discovery that built-ins could be exotic?)

[jmdyck]

I think PR #3007 would be a basis for resolving my concern.

[jmdyck]

PR #3007 was superseded by PR #3145, which (among other things) established that all built-in functions had 10.3.1 [[Call]] semantics, thus resolving this discussion.

[syg]

lgtm % question and nits

[syg]

What's the verbiage for describing an async AC? "Let c be a new async Abstract Closure"? Or "Async Abstract Closure" with capital A?

[bakkot]

I would say "async Abstract Closure", I think; the idea is what we will say "async function" or "async method" for the built-in function case (as in proposal-iterator-helpers).

We don't actually have any async ACs at the moment so it's not established yet.

[bakkot]

Per plenary, built-in async functions will continue to have Function.prototype as their prototype.

[jmdyck]

Roughly speaking, the status quo says that a function is ordinary if its [[Call]] conforms to either 10.2.1 or 10.3.1. This PR defines a distinct [[Call]] for built-in async function objects, which implies that these objects are not ordinary. Is that intentional, or do you need to tweak the definition of ordinary to include them?

[syg]

Recording my thoughts here for the record. While being able to specify built-ins as async functions in spec text significantly aids spec authors and many readers, I've found that it simultaneously really hinders understanding by implementers that work in a non-self hosting implementation, like V8.

For non-self hosting implementations, the implementer would need to do a manual transform to some kind of state machine. This manual transformation process increases the likelihood of bugs and subtle non-interop (like getting the number of Promise ticks wrong or something).

There are possible paths forward. The long-term solution is for V8 to be build something so that async functions can be written as straightline code as well. The spec can also include the transformed version and offer an alternate formulation of async builtins.

[js-choi]

Once tc39/proposal-array-from-async#50’s reviews are finished and it is merged, Array.fromAsync will be ready to make a pull request against this pull request for formal Stage 3 reviews, as per tc39/proposal-array-from-async#14 (comment). [Edit: Now in #3581.]

However, this pull request now seems to have merge conflicts with the main ecma262 branch.

Should I write the new Array.fromAsync pull request against this pull request as it is now, or should I wait until this pull request’s conflicts are resolved?

[jmdyck]

I think it's fairly clear that "built-in async functions" are a subset of "built-in functions". However, there are a few things that make it less clear:

• CreateBuiltinFunction has a dichotomy between creating "a new built-in async function object" vs "a new built-in function object". The latter could instead be "a new built-in sync function object" (or whatever you want to call it, I don't think there's a precedent).

• 10.3.1 [[Call]] and 10.3.2 [[Construct]] say they are for "a built-in function object F". Similarly, 10.3.3 BuiltinCallOrConstruct says that its parameter _F_ is "a built-in function object". But presumably these are only for synchronous built-ins.

• The new section 10.4 Built-in Async Function Objects is a sibling of 10.3 Built-in Function Objects. You could instead make it a subsection. At the same time, it might be helpful to factor out the sync-specific parts of 10.3 and put them in a complementary subsection. That is, something like this:

10.3 Built-in Function Objects
     10.3.1 CreateBuiltinFunction
     10.3.2 Built-in Sync Function Objects
            10.3.2.1 [[Call]]
            10.3.2.2 [[Construct]]
            10.3.2.3 BuiltinCallOrConstruct
     10.3.3 Built-in Async Function Objects
            10.3.3.1 [[Call]]

[michaelficarra]

Addressed comments from @jmdyck except for the relationship between the terms "async built-in function object" and "built-in function object". We'll have to figure that one out together in editor call.

[syg]

Editor call: for 10.3.1 let's go with branching in the algorithm steps on whether the built-in is async or not, instead of defining a new kind of built-in function and having to answer all the other questions that entails.

[michaelficarra]

Okay this should be updated as requested.

[github-actions]

The rendered spec for this PR is available at https://tc39.es/ecma262/pr/2942.

[michaelficarra]

Side note: this ID isn't referenced anywhere in 262. Is there another spec that references it? I hope not. Maybe we can get rid of it.

[jmdyck]

Formerly, [[Construct]] for built-in functions was defined as "the same as [[Call]] [for built-in functions], except that [one step] is replaced by [a different step]", and it used href="#step-call-builtin-function-result" to refer to the one step in [[Call]]. That cross-reference disappeared when PR #2637 factored out BuiltinCallOrConstruct.

But there might be another spec (or web-page, or bookmark) that references it.

[jmdyck]

It slightly bugs me that the "Push" step is factored out, but the "Remove" step isn't. You could factor it out by ending the async branch with

1. Let _result_ be NormalCompletion(_promiseCapability_.[[Promise]]).

but then there's the question of whether the AC's _result_ shadows that outer one.

But that's mostly a matter of style. I think the PR is okay to go as is.

I think the PR is okay to go as is.

[michaelficarra]

@jmdyck Yeah we try not to shadow. We could use different aliases if we really cared, but I think it's nice and clear how it is.

[bakkot]

On re-reading I don't love this specific wording, because I'm worried about DOM or other specs having "async functions" which aren't intended to take this branch; possibly "as an async function or async method"?

[bakkot]

This doesn't obviously generalize to anonymous built-in functions (e.g. created from Abstract Closures), but the current wording doesn't really either - if you do Let _F_ be CreateBuiltInFunction(...) there's not really a place to "describe [it] as 'async'".

[michaelficarra]

Let _F_ be CreateBuiltInFunction(...). _F_ is async.

[bakkot]

I don't love that, but I guess if it were factored out to CreateBuiltInAsyncFunction or something it could be ok. At which point we could say in the description for CreateBuiltInAsyncFunction that it "creates a built-in async function" (and maybe have it link to this step), and then it would satisfy the alternative wording I proposed.

[jmdyck]

Alternatively,

• CreateBuiltinFunction would take another parameter, indicating whether the function is sync or async,
• the value of that param would be put in a new internal slot of the function object, and
• BuiltinCallOrConstruct here would check the value of that internal slot.

[bakkot]

LGTM other than the wordsmithing comment, which I guess could be addressed later.