Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin

Cookies

Round Trip Behaviour

The following tables document how a value is sent in a Set-Cookie header, what gets stored by a typical browser, the Cookie header that is generated by the browser and then the final value returned to a Servlet application.

The browser tested here is Chrome-31

Default Configuration (no properties set)

Implementation Progress

I started work on this in a local branch. Patches for the changes made there can be found here:
http://people.apache.org/~jboynes/patches/
Of these, patches 01 to 12 have been applied.

There is substantial refactoring in there to simply the current implementation. Actual changes are:

  • C3 '=' is now disallowed in Netscape cookie names (it was already not allowed in RFC2109 names)
  • C4 Attribute names are allowed as cookies names
  • Cookie names starting with '$' are allowed in Netscape and RFC6265 mode and will still throw an IAE in RFC2109 mode

Round Trip Behaviour

The following tables document how a value is sent in a Set-Cookie header, what gets stored by a typical browser, the Cookie header that is generated by the browser and then the final value returned to a Servlet application.

The browser tested here is Chrome-31

Default Configuration (no properties set)

 

Generation

 

Browser Value

Parsing

 

Version

Value

Set-Cookie Header

Cookie Header

Resulting Value

0

 

Generation

 

Browser Value

Parsing

 

Version

Value

Set-Cookie Header

Cookie Header

Resulting Value

0

bar

test=bar

bar

test=bar

bar

0

"bar"

test="bar"

"bar"

test="bar"

bar

0

""

test=""

""

test=""

emptyString

0

a"b

test="a\"b"; Version=1

"a\"b"

test="a\"b"

a"b

0

a\b

test="a\b"; Version=1

"a\b"

test="a\b"

ab

0

a?b

test="a?b"; Version=1

"a?b"

test="a?b"

a?b

1

bar

test=bar; Version=1

bar

test=bar

bar

...

The intent of the following changes is improve interoperability of cookies with other servers and with client-side JavaScript. The primary changes are a switch to the RFC6265 format for transmission of V0 cookies to be more in line with browser behaviour, and support for UTF-8 encoded values that are now specified by HTML-5.

This is very preliminary and intended primary to focus discussion on something concrete now the behavior has been clarified.

Changes to Cookie class

C1 Stricter default validation of name::

  • *
indent

Change the default value of STRICT_NAMING to be true even if STRICT_SERVLET_COMPLIANCE is false. Application impact is that applications that wish to set cookies with names that are valid per Netscape's rules but that are not valid "tokens" per RFC2109 or RFC6265 will need to explicitly set this system property. The intent of the change is to notify application developers that they are using a cookie name that is likely to have interoperability issues.
  • *
indent

*Alternative C1a:* remove option for Netscape naming entirely. Applications that need to set names that do not comply with RFC2109 and RFC6265 would need to sub-class Cookie themselves. If this is common, then we could provide a default implementation of that behaviour (e.g. o.a.t.NetscapeCookie).

C2 Always allow "/" in Netscape cookie names::

  • *
indent

Discontinue use of FWD_SLASH_IS_SEPARATOR to configure whether a "/" character can appear in a name when STRICT_NAMING is false and instead always allow it. No negative application impact and matches the behaviour of the RI. This property was introduced to prevent quoting of tokens used in Path values as that is not supported by IE but that behaviour is not needed for names.

C3 Always disallow "=" in Netscape cookie names::

  • *
indent

Now throw IllegalArgumentException if a "=" character is present. Application impact is that an attempt to use "=" will now trigger an IAE before the cookie is sent rather than having the browser set a cookie with an inconsistent name and value. When parsing the received Set-Cookie header, browsers treat all characters up to the first "=" character as the name and the remainder as the value. Having a "=" character in the name will result in an incorrect split.

Changes to generation of Set-Cookie header

-5.

This is very preliminary and intended primary to focus discussion on something concrete now the behavior has been clarified.

Changes to Cookie class

C1 Stricter default validation of name::

  • *
indent

Change the default value of STRICT_NAMING to be true even if STRICT_SERVLET_COMPLIANCE is false. Application impact is that applications that wish to set cookies with names that are valid per Netscape's rules but that are not valid "tokens" per RFC2109 or RFC6265 will need to explicitly set this system property. The intent of the change is to notify application developers that they are using a cookie name that is likely to have interoperability issues.
  • *
indent

*Alternative C1a:* remove option for Netscape naming entirely. Applications that need to set names that do not comply with RFC2109 and RFC6265 would need to sub-class Cookie themselves. If this is common, then we could provide a default implementation of that behaviour (e.g. o.a.t.NetscapeCookie).
  • *
indent

*Alternative C1b:* Make STRICT_NAMING a enum specifying which standard's rules to enforce: values are "netscape" "rfc2109" or "rfc6265" with the default being "rfc6265." Maintain compatibilty by allowing "true" as an alias for "rfc2109" and "false" as an alias for "netscape" with the option defaulting to "rfc6265" or to "rfc2109" if STRICT_SERVLET_COMPLIANCE is true. "rfc2109" and "rfc6265" are both based on "token" rules, except "rfc2109" disallows values starting with '$' character.

C2 Always allow "/" in Netscape cookie names::

  • *
indent

Discontinue use of FWD_SLASH_IS_SEPARATOR to configure whether a "/" character can appear in a name when STRICT_NAMING is false and instead always allow it. No negative application impact and matches the behaviour of the RI. This property was introduced to prevent quoting of tokens used in Path values as that is not supported by IE but that behaviour is not needed for names.

C3 Always disallow "=" in Netscape cookie names::

  • *
indent

Now throw IllegalArgumentException if a "=" character is present. Application impact is that an attempt to use "=" will now trigger an IAE before the cookie is sent rather than having the browser set a cookie with an inconsistent name and value. When parsing the received Set-Cookie header, browsers treat all characters up to the first "=" character as the name and the remainder as the value. Having a "=" character in the name will result in an incorrect split.

C4 Always allow attribute names (e.g. "Expires") as cookie names::

  • *
indent

Stop throwing IAE if an attribute name is used as the cookie name. No application impact as more values are allowed. No confusion with cookie protocols as they are unambiguous in Set-Cookie and are never used as part of a Cookie header (attributes in the RFC2109 Cookie header begin with '$').

C5 Allow unnamed cookies in C1b "netscape" mode::

  • *
indent

Allow cookies whose name is null or the empty string. Browsers will store a single cookie that has no name whose value is sent as simply «value» (i.e. without any '=' delimiter). This would now be supported if STRICT_NAMING is set to "netscape" but would remain disallowed in "rfc2109" or "rfc6265" modes. If allowed, the Set-Cookie header would contain just the value (no '=' present and an IAE if value contained an '=') and any such cookie found during parsing would be included in the result of [HttpServletRequest]#getCookies().

Changes to generation of Set-Cookie header

G1 Use RFC6265 format header for V0 cookies::

  • *
indent

When version == 0 always generate a RFC6265 header, raising an exception from addCookie if the value is invalid rather than attempting to upgrade to a RFC2109 header to use quoting. Application impact is that they will now fail fast with an error rather than inconsistent data as described in Bug 55920; applications that do not set invalid values will not be impacted.
  • *
indent

*Alternative G1a:* Generate an RFC6265 header if possible but provide an option (disabled by default) to allow switching to an RFC2109 header if a valid RFC6265 header is not possible.

G2 Use RFC2109 format header only for V1 G1 Use RFC6265 format header for V0 cookies::

  • *
indent
When version == 01 always generate a RFC6265RFC2109 header, raising an exception from addCookie if the value is invalid. ratherThis thanpreserves attemptingexisting tobehaviour upgradefor toapplications athat RFC2109use header to use quoting. Application impact is that they will now fail fast with an error rather than inconsistent data as described in Bug 55920; applications that do not set invalid values will not be impacted.
  • *
indent

*Alternative G1a:* Generate an RFC6265 header if possible but provide an option (disabled by default) to allow switching to an RFC2109 header if a valid RFC6265 header is not possible.

G2 Use RFC2109 format header only for V1 cookies::

  • *
indent

When version == 1 always generate a RFC2109 header, raising an exception from addCookie if the value is invalid. This preserves existing behaviour for applications that use V1 cookies.
V1 cookies.

G3 Stop adding quotes or escaping to values::

  • *
indent

The value supplied by the application will be validated to the relevant specification and will result in a IAE if it does not conform. The value will never be modified to add quotations or escape characters, Application impact is that an attempt to set an invalid value will result in an early error rather than inconsistent data.
  • *
indent

*Alternative G3a:* Quotes and/or escaping only to be added to RFC2109 headers. API to remain symmetric and quoting/escaping to remain transparent to applicatons.

G4 Use UTF-8 encoding for G3 Stop adding quotes or escaping to values::

  • *
indent

The value supplied by the application will be validated to the relevant specification and will result in a IAE if it does not conform. The value will never be modified to add quotations or escape characters, Application impact is that an attempt to set an invalid value will result in an early error rather than inconsistent data
The value (which is a UCS-16 Java String) will be encoded using UTF-8 when being added to the header. Application impact is that non-ASCII characters will no longer cause an IAE. For V0 cookies, this is an extension to RFC6265 required to support HTML-5. V1 cookies already allow 8-bit characters if quoted and this is likely to be needed to avoid an IAE as the value would still be validated; it would be the application's responsibility to quote the value.
  • *
indent

*Alternative G3a:* Quotes and/or escaping only to be added to RFC2109 headers. API to remain symmetric and quoting/escaping to remain transparent to applicatons.

G4 Use UTF-8 encoding for values::

  • *


_kkolinko_: Using UTF-8 in HTTP headers is not allowed by RFC 2616. On page 32 it says:


Wiki Markup
 {{message-header = field-name ":" \[field-value \]}}

field-value = *( field-content | LWS )

field-content = <the OCTETs making up the field-value and consisting of either *TEXT or combinations of token, separators, and quoted-string>

The tokens are US-ASCII (0-127 minus CTLs or separators) (pages 16-17).

Wiki Markup
 The TEXT is defined on page 16 where it says: "Words of \*TEXT MAY contain characters from character sets other than ISO-8859-1 \[22\] only when encoded according to the rules of RFC 2047 \[14\]."

The quoted-string is TEXT in double quotes (page 16).

  • *
indent


_kkolinko_: Javadoc for [HttpServletResponse].setHeader() method also mentions that the value of a header should be encoded according to RFC 2047. http://www.ietf.org/rfc/rfc2047.txt
indent

The value (which is a UCS-16 Java String) will be encoded using UTF-8 when being added to the header. Application impact is that non-ASCII characters will no longer cause an IAE. For V0 cookies, this is an extension to RFC6265 required to support HTML-5. V1 cookies already allow 8-bit characters if quoted and this is likely to be needed to avoid an IAE as the value would still be validated; it would be the application's responsibility to quote the value.

G5 Validate domain per RFC6265::

...

indent
Stop modifying the header in-situ as part of the de-escaping process ([Bug 57896|https://bz.apache.org/bugzilla/show_bug.cgi?id=57896]) so that an application can elect to perform its own parsing by calling getHeader("Cookie"). Eliminate the need for the PRESERVE_COOKIE_HEADER property that currently controls whether a copy of the header is made if modifications are needed. Perform de-escaping during the copy needed to convert the MessageBytes to the String in Cookie#value, possibly during any conversation process needed to handle UTF-8.

Impact of proposal on existing issues

Issue

Impact

Bug 55917

Parsing will no longer cause an IAE. 8-bit values will be interpreted as a UTF-8 value and the cookie would be dropped if they are not a valid encoding.

Bug 55918

The cookie would be dropped rather than accepted.

Bug 55920

Valid values would be round tripped including quotes supplied by the application. Attempts to set invalid values would result in a IAE from addCookie. Invalid values sent by the browser would result in the cookie being ignored.

Bug 55921

Attempts to set a cookie containing raw JSON would results in an IAE due to the DQUOTE characters. A cookie sent from the browser containing JSON would be accepted although any semicolons in the data would result in early termination (note, browsers other than Safari do not allow semicolons in values anyway).

...

Issue

Current behaviour (8.0.0-RC10/7.0.50)

Proposed new behaviour

Servlet + Netscape + RFC2109

Servlet + RFC 6265

0x80 to 0xFF in cookie value (Bug 55917)

IAE

TBD

Netscape yes. RFC2109 requires quotes.

RFC 6265 never allowed.

CTL allowed in quoted cookie values (Bug 55918)

Allowed

TBD

Not allowed.

Not allowed.

Quoted values in V0 cookies (Bug 55920)

Quotes removed.

TBD

Netscape - quotes are part of value.

Quotes are not part of value.

Raw JSON in cookie values (Bug 55921)

TBD

TBD

TBD

TBD

Allow equals in value

Not by default. Allowed if property set.

TBD

Netscape is ambiguous. RFC2109 requires quoting.

Allowed.

Allow separators in V0 names and values

Not by default. Allowed if property set.

TBD

Yes except semi-colon, comma and whitespace.

Never in names. Yes in values except semi-colon, comma and whitespace, double-quote and backslash.

Always add expires

Enabled by default. Disabled by property.

TBD

Netsacpe uses expires. RFC2109 uses Max-Age.

Allows either, none or both.

/ is separator

Enabled by default. Disabled by property.

TBD

Netscape allowed in names and values. RFC2109 allowed in values if quoted.

Allowed in values.

Strict naming (as per Servlet spec)

Enabled by default. Disabled by property.

TBD

Netscape allows names the Servlet spec does not. RFC2109 is consistent with the Servlet spec.

Consistent with the Servlet spec.

Allow name only

Disabled by default. Enabled by property.

TBD

Netscape allowed and equals sign expected before empty value. RFC2109 not allowed.

Allowed but equals sign required before empty value.

Issues to add to the table above

  • Bug 55951 regarding UTF-8 encoded values from HTML5
  • Any further issues raised on mailing lists

...