net/url: RequestURI encoded path should not encode '!'

[ghost]

see https://groups.google.com/forum/#!topic/golang-nuts/5er6Ud_V0-U

[ghost]

Comment 1:

see http://play.golang.org/p/X6LGcNbHzA, it's more obvious.
it's affected by all struts2 framework, it's serious.

[davecheney]

Comment 2:

Here is a smaller reproduction,
http://play.golang.org/p/xPQ61lbUqE
The bone of contention is the encoding of !. I am not sure if this is a problem or not.

[ghost]

Comment 3:

The https://www.shipin7.com/user/userAction%21goRegister.action page is incorrect.
https://www.shipin7.com/user/userAction!goRegister.action page is ok.
you can compare above in Browser.
http.Client.Do(), http.Get() internal encode '!' and send whole Request to server.
I think '!' should not encode because of RFC3986

[gopherbot]

Comment 4 by hongruiqi:

In RFC 2396:
      reserved    = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" |
                    "$" | ","
      unreserved  = alphanum | mark
      mark        = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
In RFC 3986:
      reserved    = gen-delims / sub-delims
      gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
      sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="
      unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"
https://code.google.com/p/go/source/detail?r=6b46fb967ca4a48caf486f4452c4358251f91aad
The CL above only removes !*\() from unreserved part(the \ may be wrong, it should be
"'"), 
but doesn't add []!'()* to the reserved part. So I think it's a bug.

[gopherbot]

Comment 5 by hongruiqi:

sorry, I mistake '\'' as '\', nothing wrong here.

[rsc]

Comment 6:

Labels changed: added go1.3maybe.

[dsymonds]

Comment 7:

Labels changed: added priority-soon, packagebug, removed priority-triage.

[rsc]

Comment 8:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 9:

Labels changed: added repo-main.

[gopherbot]

Comment 10:

This also triggers with the hashbang style single-page web app url fragments:
http://play.golang.org/p/-kx5yULrzl
    u, err := url.Parse("http://foo.bar/#!quux")
    // http://foo.bar/#%21quux
See https://developers.google.com/webmasters/ajax-crawling/ for more.

[minux]

Comment 11:

https://golang.org/cl/31400043/
Does anyone know why ! is left out in the first place? was that intentional?

Status changed to Started.

[andybalholm]

Comment 12:

Some examples of URLs where parentheses don't work if they're escaped:
http://web.signaltiretrader.com/(S(5iexcz551ptpgo45g03mgz45))/Themes/css/ploneColumns.css
and the LinkedIn API URLs discussed at
https://groups.google.com/forum/#!searchin/golang-nuts/url$20escaping/golang-nuts/Mro8TGrb3y8/eW8QCx_iFYMJ

[gopherbot]

Comment 13:

jkbbwr on IRC pointed out that slashes in queries get quoted too:
http://play.golang.org/p/EiRhkOT8im
Relevant RFC: http://tools.ietf.org/html/rfc3986#section-3.4

[gopherbot]

Comment 14 by hongruiqi:

Slashes not quoted in queries may causes some server failed to handle?

[andybalholm]

Comment 15:

This appears to be a duplicate of 5684.

[andybalholm]

Comment 16:

Referenced on StackOverflow:
http://stackoverflow.com/questions/20847357/golang-http-client-always-escaped-the-url/

[bradfitz]

Comment 17:

Status changed to Duplicate.

Merged into issue #5684.