Deprecate <keygen>

[bblfish]

The following
commit 1b438067d84da2ee95988842cdd8fe2655444936 named "deprecate <keygen> and appcache" has not be registered as an issue, meaning it may not be visible to people who should care about it, and the discussion may get lost later on, so I am posting this in the issues database.

[annevk]

Well, I have certainly reviewed the blink-dev thread and equivalent thread from Mozilla before making that change. Coupled with Microsoft not being interested in shipping it, this seems like the only way forward.

[bblfish]

Keygen never worked in Microsoft (MS) . They have an ActiveX controller to do the same thing, which we use. It's easy using JS to switch between them. (it makes code more difficult to write, and it would be nice if keygen were improved to the point where it was implemented in MS browsers too).
What is the thread in Mozilla you are referring to?

[bblfish]

Let me note that it's pretty odd to commit something to master without first opening an issue on the subject that people of the HTML5 WG can first discuss.

[annevk]

Please be respectful. Abbreviations like "M$" have no place here. And this is also not the HTML WG or has any relationship with it.

[bblfish]

Does the WhatWG not have a process of first discussing an issue on its list or in an issue DB and then making commits? The discussions referred to in commit 1b43806 may not be visible to your members. What is the official way to bring up an issue here?

[annevk]

The editor is in charge of making changes within the WHATWG. And typically the editor does so after consulting with the community in some manner. In this case, a PR was supplied: #26.

https://groups.google.com/d/msg/mozilla.dev.platform/pAUG2VQ6xfQ/FKX63BwOIwAJ is the discussion by Mozilla by the way. There's at least one more thread somewhere around there.

[foolip]

Closing this issue, since now there is an issue, namely this issue. FWIW, I support deprecating <keygen> in the spec and in Blink.

[bblfish]

@foolip you don't leave much time for the public and other interested parties to participate in a debate: 3.5 hours it seems to be precise. Is it so important to remove a feature that has been around for 15 years this quickly?

[Ms2ger]

Nothing was removed. The only thing that happened here is that a warning was added to the specification to point out that browser vendors are trying to figure out whether the feature can be removed.

[csarven]

@Ms2ger It sounds a bit more than a warning to me (after all, it is marked as "critical"). Otherwise, is there any need to make a recommendation to use service workers?

If browser vendors / people are trying to figure things out, then wait for actual justification or an appropriate alternate to present itself. As I understand the threads, there only appears to be hypothetical promises.

[domenic]

Fundamentally, this is about making the spec reflect reality. The reality is that keygen is:

1. Never supported in one major rendering engine
2. Being actively deprecated in two more, with the end goal of removal in the reasonably-near future.

This is ample justification for adding a warning, and indeed a critical one, to the spec, to reflect the fact that this feature is going away.

Besides, per spec the feature never actually had to work; UAs never had to support any keygen algorithms.

[Yay295]

Apple has no intention of supporting WebM, and Microsoft doesn't seem too keen on it either. Should we remove WebM from the spec as well? The reality of keygen is that many people currently use it, and the lack of support from ONE browser should not be seen as a total failure. Microsoft Edge's developer website states their reason for not supporting it is "...significant interoperability issues exist with the DOM interfaces and protocols for this feature."[1] Would it not make more sense to fix these issues and get Microsoft's support rather than throwing it out?

[1] https://dev.modern.ie/platform/status/keygenelement/?filter=f3f0000bf&search=keygen

[domenic]

WebM isn't in the spec.

[Yay295]

Huh, you're right (well, half right: https://html.spec.whatwg.org/multipage/embedded-content.html#sourcing-in-band-text-tracks:media-resource-6). My bad. My point was that a lack of support from one or more browsers does not mean it should be removed.

[bblfish]

@domenic argued:

Fundamentally, this is about making the spec reflect reality.
The reality is that keygen is:
Never supported in one major rendering engine

It was supported in Firefox, Opera, Safari for over 10 years and then Chrome came along and supported it too. With IE there is a JS work around using their Certificate Enrolment API as mentioned at the top of this issue. So all desktop browsers at least have this functionality.

Being actively deprecated in two more, with the end goal of removal in the reasonably-near future.

The deprecations there are being pushed through without regard to the arguments held up. One of the arguments was that this is in the standard. As a result this change was surreptitiously made to html5 in order to help win the argument in the browser space. The two arguments are not independent.

This is ample justification for adding a warning, and indeed a critical one, to the spec, to reflect the fact that this feature is going away.
Besides, per spec the feature never actually had to work; UAs never had to support any keygen algorithms.

They work quite well: we tested them in all the browsers. There is nothing yet equivalent to it.

The argument here and the one @timbl put forward in the discussions and now at the Technical Architecture Group of the W3C is that you should first wait to have something to replace <keygen> which can be verified to have the right properties, and then you can deprecate it, when all are happy. Currently there is reference to an unknown and unfinished spec, which may change any moment, which may fail, and which may not be as successful as hoped ( the FIDO alliance). If it does offer the right features all will be for the best, and we'll be happy to deprecate. Deprecating features before hand is too put to much trust in human processes, and is not allowing others in this process to make up their mind.

[domenic]

As a result this change was surreptitiously made to html5 in order to help win the argument in the browser space.

This is inaccurate. The change was made to the HTML Standard in response to the browser depredations.

[ABISprotocol]

I'd like to request that this issue be reopened (edit... Derp because it is late... PULL REQUEST BE RECONSIDERED, because apparently there is already the commit discussed by this thread was already made before an issue was opened and pull request were only opened later)... seems odd it was closed out so quickly. I get there is a pull request related that was also rapidly closed. Seems to require more public input. Should have been an issue, pull request, etc... then merge of some kind after substantial public discussion... That isn't what happened here. So what's going on?

[foolip]

There isn't anything unusual going on here, the way things usually happen is:

1. A browser engine developer notices that a feature is problematic and gets an itch to remove it from the code base.
2. The developer investigates support in other browsers, and what usage of the feature on the web looks like.
3. If browser interoperability is bad enough or usage is low enough, the developer proposes that the feature be removed from the code base and the spec, often with some deprecation period.
4. If everything works out, the browser in question ships without the feature in a stable release.
5. It is now likely safe for other browsers to remove their support, and then feature can be removed from the spec if it hasn't already been.

We are currently in step 3, where the a warning has been added in the spec but the feature hasn't been deprecated in any browser yet.

Until it looks like the tide is turning and the relevant browser developers commit to keeping <keygen>, it's appropriate to warn against using the feature in the spec.

[csarven]

Can we at least agree not to abuse the term warning meanwhile marking it with class critical at 1b43806 ?

[bblfish]

@foolip there was not unanimity in the browser discussions about removing it, and the arguments were not coherently made, which should not be surprising as this is a complex topic involving many different groups from JS crypto, UI design, TLS, HTTP and HTTP2.0, Web App Security, certificate design, authentication APIs such as OpenID, OAuth, Mozilla Persona, etc... other standards organisations such as FIDO that tie into hardware and OSes, Web Privacy debates, etc. etc....

As such this is a discussion that requires coordination between a large number of diverse groups, with very different skills, on a topic that is fraught with complex international political agendas ( think Snowden). Hopefully the discussion at the Technial Architecture Group (TAG) will help bring more light to this:

https://lists.w3.org/Archives/Public/www-tag/2015Sep/thread.html

[Hixie]

Consensus is not part of how WHATWG specs are written. See https://whatwg.org/faq if you are not familiar with the WHATWG process.

class="critical" is the class name that is used for anything that represents something we expect to change in the spec. class=note and class=warning represent things where we (currently) expect the spec to not change. Here, since the warning is specifically about the fact that the spec may in fact eventually change, "critical" is the appropriate class name. Note that class names are opaque non-semantic strings and it could be class="bluefish" and have the same meaning. It's just a key into the CSS. (class=warning actually renders a more noticeable message than class=critical.)

[bblfish]

We were finally told what the reason for the "security risk" of is meant to be. Since this issue is closed I opened issue 102: & "md5WithRSAEncryption" for that. That debate is more interesting than what the process of the WHATWG is.

[akuckartz]

Why was this issue closed?

[foolip]

@akuckartz, the stated reason for opening the issue was to have an issue tracking the deprecation commit, so there isn't anything actionable we can do with the issue if open. Please file a new issue if there is a specific spec change you are proposing.

[akuckartz]

@foolip No, I will raise this outside the WHATWG.

[bblfish]

Perhaps the most details logic overview I have yet gone into about the logical security aspect of at multiple layers can be found here now: https://groups.google.com/a/chromium.org/d/msg/blink-dev/pX5NbX0Xack/dn_7RguGAAAJ

[bblfish]

There may be an issue with the MD5 weakness in spkac as communicated to me by Prof David Chadwick in a personal e-mail ( I asked him if I could quote him )

the only attack I can think of in your given example is that, acting as
MITM, I substitute the certificate on a signed message with my false
certificate containing a different subject and then let the message
continue on its way. In this way the recipient MAY (only may, not will,
because it depends upon the signed content not containing the real
details of the sender) believe that the message came from someone other
than the real sender.

Here is where it might be useful. You submit a patent application
electronically and I, acting as a MITM, substitute your certificate with
one containing me as the subject, and the patent gets registered in my
name and not yours.

I expanded on that in the blink-dev thread Re: beyond MD5 to security logic -- Re: (Pre-)Intent to Deprecate: element and application/x-x509-*-cert MIME handling. But it really comes to show that one should only trust that a signature belongs to a person after that person has proven that they have the private key of a given public key. In TLS authentication this happens automatically. In the described situation the patent office should first have required the person to authenticate with the certificate before accepting the submission.

In any case all of this makes a case for improving spkac, not for removing .

[minfrin]

Given that there is no standard to replace keygen, what the actual effect of this change is to remove client certificates from the world wide web.

Microsoft's lack of support for keygen was irrelevant, as they provided their activex alternative. This is not therefore a justification for the current proposal.

Why is there an attempt to ram through the removal of a key security feature from the web?

[Hixie]

There is no such attempt, at least not from the WHATWG. You'll notice that the element is still in the spec. The only change that has been made to the spec is to point out that browser vendors desire to stop supporting this feature. That's just a statement of fact. If you want to convince them not to do so, you have to discuss it with them directly.

[minfrin]

Who are "browser vendors?".

To quote from the commit: "This feature is in the process of being removed from the Web platform. (This
is a long process that takes many years.) Using the keygen element at this time is highly discouraged."

Microsoft disagreed with the spec for 15 years and everyone else decided to follow the spec anyway. Suddenly a different vendor disagrees with the spec and everyone scrambles to remove it from the spec?

Why the deprecation of client side private keys? What is going on?

[Hixie]

Chrome and Mozilla are the vendors who recently indicated their desire to drop it in future versions of their respective products, as I understand it.

Actually, the spec post-dates all the implementations. <keygen> was non-standard when Netscape invented it, and the W3C refused to spec it, and it was only when a few years ago I finally decided to find what little documentation there was and put it in the WHATWG spec that there was anything resembling a spec for it. I don't believe any of the implementations post-date the spec.

[bblfish]

Chrome and Mozilla are the vendors who recently indicated their desire to drop it in future versions of their respective products, as I understand it.
Why the deprecation of client side private keys? What is going on?

Ryan Sleevi, the person who started the blink-dev thread where the discussion is happening and which I think is being used by Firefox as the discussion thread, contributes to Firefox security as well as to Google Chrome's , and was editor of the JS Crypto API. He believes that everything that can be done with keygen can actually be done with the crypto API, and he points to the Fido alliance as the future cure. We have argued that these are not necessarily incompatible with , and that we should wait, but he is in a hurry. It seems to have to be done real soon. Many people have spoken out. Tim Berners Lee brought this up on the Technical Architecture Group this month, and there is discussion on the WebAppsSec group too, eg concerning the relevance of Same Origin Policy

As for @Hixie's point that <keygen> was non standard, I'll refer to @dirkx point on the mailing list yesterday which makes the point quite well that running code and rough consensus are the foundation of a standard, and that was there long before it was speced out by the whatwg.

But in all fairness - keygen is used as an (early) step in enrolling for a generally quite persistent client side x.509 certificate. And pretty much only for that. So yes; we need to discuss the whole x509 mess; as that is what we break.

Now the docs may be crap or non-existent. Which is a problem and should be fixed so (in)compliance becomes measurable. If I had to implement it from scratch - I would loathe the inordinate amount of reverse engineering and testing it would take.

But at the same time - people have gotten this to work rather reliable; over very long periods of time; sometimes at a fairly large scale; across quite a few generations of browsers & servers - in a very wide range of settings. With little control over their environments; environments that evolved.

Now arguably a lot of that is thanks to PKCS; a lot of ‘hardcoded’ assumptions in OpenSC and downright cargo culting between the various implementations UI wise. So, while often as badly documented as keygen, they keep the backbone common & stable.

Some of the oldest ones I can think off date back to ’98*; and are still operable enough for enterprises; across over 15 years of windows, Solaris, and later OSX and Linux; and have been able to move along the various key lengths and algorithms as well.

So despite the crappy docs & specs - folks have kept it all working.

Dw

*: The oldest commercial deployment I still have docs for was with Netscape Gold 3.0.1 though.

[rektide]

I'm new to the Keygen feature, but if I understand it right, it enables a very important use case: link-local secure networking. I would like to see a future where I and other people about me advertise HTTP services, for example "Top 5 albums of the week" or some such. Keygen would let me also advertise/negotiate a self-signed certificate on the HTTP page, that when installed would let me upgrade the viewer to HTTPS. Previously it was a nice to have, but now that getUserMedia works over secure origin only, I consider this (a means to agree to a non browser-blessed-CA certificate) a must have feature.

What do you think is an appropriate way to allow a link-local web? Does everyone everywhere wanting to share ambiently with other people need to procure a cert from a centralized, controlled CA? That seems antithetical, a burden to projects like Eddystone which are trying to increase the degrees to which we can form connections with one another, especially over link local. Do you have any other affordances in the pipeline? I don't know how else to interpret this deprecation as anything but pushing us from a place where users can connect with each other, to a scary, centralized web. I'm not a fan of this spec, but based on this one use case, it seems vital to peer to peer scenarios, it has a rough concensus of running code, it's technical nits that Microsoft object to can be remedied, and it has no replacements lined up. I'd like this PR reverted, and a genuine discussion on what should happen to occur if the existing feature was found inadequate/undesirable to implement.

Chrome issue on getUserMedia() being unusable over link-local,

http://code.google.com/p/chromium/issues/detail?id=531675&thanks=531675&ts=1442264648

[rektide]

Peter Bowen kindly corrected my false understanding of Keygen. Keygen was never an aid for the server to assert authorization in a link-local scenario. Apologies, please disregard above.

[dirkx]

Do not feel to guilty - over the years many people have used keygen in odd experiments as a way to create public/private key pairs - to then trigger some (self signed) scenario.

[bblfish]

@rektide perhaps you may want to look at WebID-TLS as something close to what @dirkx is saying. Not sure how close it comes to your use case. It also has an actual description of how the <keygen> protocol works, which is missing in the html5 spec.

[xeor]

If <keygen> dies, is there anything that replaces what it does? Making the browser generate a private-key, sending off a signing request, storing the key securely?
We need a way to do that, using <keygen> or another way.

What if you could do this using Javascript, but forcing the browser to popup one of those messages first. Just like the geolocation api and "This page wants to know your location", we could have a "This page wants to generate a private certificate in your browser.".

[bblfish]

The W3C TAG have a report entitled "Keygen and Client Certificates" edited by
Microsoft's Travis Leithead, that forms a balanced and principled review of
the discussion:

http://w3ctag.github.io/client-certificates/

[ABISprotocol]

TY @bblfish

[deiu]

Is anyone else able to confirm that the latest (stable) build of Chrome has indeed disabled the KEYGEN element?

[perj]

The current stable version of Chrome indeed seems to have disabled keygen. We noticed this by our certificate form breaking for Chrome users, which is the first I've heard of any of this.

[dirkx]

On 08 Apr 2016, at 09:45, Per Johansson [email protected] wrote:

The current stable version of Chrome indeed seems to have disabled keygen. We noticed this by our certificate form breaking for Chrome users...

Which version - we are looking into recent bug-reports at two of our Medical (public service) customers which seem to say this too - but are absolutely unable to reproduce it (up to and including 48.0.2564.103).

Dw.

[annevk]

I'm going to lock this thread since it's receiving a bunch of off-topic conversation not relevant to the HTML Standard. Please use stackoverflow.com or some such for support questions.