Data to back up claims of usabilty favoring Hack

[runarberg]

The README includes a couple of section for why the Hack pipelines were chosen over the competing F# proposal. There are two sections in particular that interest me:

1. Hack pipes favor more common expressions, and
2. Hack pipes might be simpler to use.

Both of these claims are made without evidence to back them up, instead seemingly rely on speculations by the author. The former of these could easily be found out by searching for code samples in the real world and by asking a random sample of language users for typical code samples. This data would need to be considered in the context of where pipelines are likely to be used, so I would be interested in hearing some user stories about that. E.g. how common is it that a developer already has an n-ary function that they want to use in a pipeline? How much does it bother them that they need to wrap it in an arrow function? How often would you expect an arbitrary expression to show up in a pipeline? Are developers more likely to use libraries which will accommodate these pipelines by providing functions in place of operations. And if not, why?

The latter claim that “[h]ack pipes might be simpler to use” would probably need some in depth usability studies where participants are asked to use either pipelines and later ask participants if the following caused them any issues. E.g. how many participants experienced the issue of someFunction + 1 not being a unary function? I would also be interested in seeing some statistics of which third party tools likely users are using. E.g. the point that x |> someFunction + 1 is a runtime error is void if the majority of likely users are using static analysis tools that will point out that this code will result in a runtime type error.

To take a few of the real world examples provided in the explainer:

// Status quo
var minLoc = Object.keys( grunt.config( "uglify.all.files" ) )[ 0 ];

// With pipes
var minLoc = grunt.config('uglify.all.files') |> Object.keys(^)[0];

• How many users favor the competing style of:

  var minLoc = grunt.config('uglify.all.files') |> Object.keys |> (key) => key[0];

• Are users bothered by the syntax tax of (key) => key[0] over ^[0].
• If they are, how likely are they to write a utility functions to accommodate this:

  const nth = (n, arr) => arr[n];
  const first = (arr) => nth(0, arr);
  
  var minLoc = grunt.config('uglify.all.files') |> Object.keys |> first;

// Status quo
const json = await npmFetch.json(npa(pkgs[0]).escapedName, opts);

// With pipes
const json = pkgs[0] |> npa(^).escapedName |> await npmFetch.json(^, opts);

• How many users favor the following:

  const json = pkgs[0] |> (pkg) => npa(pkg).escapedName |> (name) => npmFetch.json(name, opts) |> await;

• Are they bothered by the special case of |> await?
• Do they value having the parameter named over the non-descriptive topic marker?

// Status quo
return _reduce(xf(typeof fn === 'function' ? _xwrap(fn) : fn), acc, list);

// With pipes
return fn
|> (typeof ^ === 'function' ? _xwrap(^) : ^)
|> xf(^)
|> _reduce(^, acc, list);

• How many users peref the following:

   return fn
   |> (x) => (typeof x === 'function' ? _xwrap(x) : x)
   |> xf
   |> (x) => _reduce(x, acc, list);

• Are they likely to use a library that will accommodate for unary function preference:

  import { ifNotTypeof, partial } from 'some-util';
  
  return fn
  |> ifNotTypeof('function', _xwrap)
  |> xf
  |> partial(_reduce, [, acc, list]);

I’m sure the committee has done some studies in order to reach the decision that they did. However, as somebody that would like the best decision to be made for the future of the language, I would love for these studies to be made publicly available so that they can be scrutinized by experts and that other members of the community—such as my self—can be reassured that the best decision was made. So far I remain unconvinced.

[lightmare]

the point that x |> someFunction + 1 is a runtime error

What's the point of having |> bind less tightly than arithmetic operators, when they're useless on the right-hand-side. Just to avoid parentheses on the left-hand-side?

[sandren]

I completely agree.

Those arguments for Hack-style seem to depend on some very bad assumptions, namely that the pipeline operator would be deployed in every instance possible and that absolutely no other changes would be made while doing so.

In this extremely restrictive and unrealistic scenario, yes, the Hack-style pipes would technically result in less verbose code overall, but in reality—as @runarberg points out—any code rewritten to use the pipeline operator would very likely see other changes as well, such as the creation and usage of more utility functions, eliminating much of the x=> syntax tax.

The two competing proposals shouldn't be evaluated based on universal adoption of the pipeline operator in an artificially simplified "codemod"-like manner. The two should be compared based on how they would impact the language long-term in the real world.

I don't think anyone would argue against the fact that there would be fewer lines of JavaScript in a world with F#-style pipes compared to Hack-style pipes based on the patterns they facilitate and I think that is how the two should be compared, not if we effectively ran a codemod on all existing JavaScript to use one of the two proposals.

[tabatkins]

JS language changes do not typically require extensive user studies to be done; in fact, it's virtually never done. Nor, to the best of my knowledge, is it common in any programming language community to gate changes behind user studies. Instead, they're typically based on some study of the language corpus, and arguments and reasoning from HCI and, indeed, language-designers intuition. Since TC39 is a large committee, and this topic in particular has the interest of a number of committee members from diverse programming backgrounds, if one champion had a very unusual intuition or background experience, the rest of the champions would discover this, and the unusual ideas would be limited in their effect.

As it happens, one of the committee members did do a user study (comparing F#-style to "smart mix"-style), and found that the differences were relatively small, but within that context, people seemed to prefer the smart-mix style for readability, but made less mistakes in translating code to F# pipelines. Again, tho, the effects were small each way; not enough to be conclusive in either direction.

---

That said, this committee does not gate proposals behind in-depth user studies. If you wish to run some on your own and share the study and results, we'd welcome the information.

[runarberg]

This is a shame. Especially in a hotly debated topic like this one. This means that a very convincing rhetoric can make claims based on speculation which might not reflect the real world well at all. When I develop web apps in the real world the design team is hard at work conducting all kinds of usability studies to get the best experience for the user. If it wasn’t for them I’m sure I would have made a very convincing case multiple times for a bad design choice because I speculated it would be better.

Not every feature needs an in depth user studies, but here we have a case of two competing proposals, one was chosen over the other based on arbitrary speculation by committee members. The proposal took many years to advance despite that only rudimentary studies were preformed.

Since the only evidence you have for the claims of usability are speculative, I recommend that you remove these two sections from the README and say the real reason for why Hack pipes were choose: “Committee members believe Hack pipelines are better suited for the language”.