Parse raw identifiers.

[allevato]

The parser implementation of SE-0451.

[ahoppen]

I think adding parenthesis here would help readability.

Suggested change

	if isEmpty && !scalar.isOperatorStartCodePoint || !scalar.isOperatorContinuationCodePoint {
	if (isEmpty && !scalar.isOperatorStartCodePoint) || !scalar.isOperatorContinuationCodePoint {

[allevato]

Done.

[ahoppen]

Do we diagnose raw identifiers that only consist of whitespace? As far as I can tell, they are forbidden by the proposal.

[allevato]

We weren't; thanks for catching that. Fixed.

[grynspan]

I think I have a hard need for determining if an identifier wrapped in backticks is raw or just an escaped keyword. Having a member such as TokenKind.isRawIdentifier would be very useful here.

[grynspan]

Would it be better to express this as isWhitespace && !isPermittedRawIdentifierWhitespace?

[allevato]

The parser explicitly avoids the use of Unicode.Scalar.Properties because they're not necessarily stable between Unicode versions and may change depending on the Unicode tables that are built into libswiftCore, so we don't want the behavior of the parser to change depending on things like what version of the operating system the compiler is running on.

It's probably exceedingly rare that new whitespace will be added (or worse, that a non-whitespace code point would become whitespace or vice versa), but hardcoding the code points means the behavior is deterministic and we can choose to make breaks via new language modes, if needed in the future.

[allevato]

I think I have a hard need for determining if an identifier wrapped in backticks is raw or just an escaped keyword. Having a member such as TokenKind.isRawIdentifier would be very useful here.

I think you can do this today, in a somewhat roundabout fashion, using existing APIs from SwiftParser:

let id = "<some keyword or raw identifier without backticks>"
let quotedID = "`\(id)`"
let isKeywordish = id.isValidSwiftIdentifier(for: .memberAccess)
  && quotedID.isValidSwiftIdentifier(for: .memberAccess)

...since x.class and x.`class` are both valid for escaped keywords but x.some raw identifier and x.`some raw identifier` would fail the first test. There may be some failure cases I'm missing here, though...

It's not the most efficient though since it kicks off two parsers. I agree it's probably a good idea to have a separate API for this that does the same single-pass over the identifier that the lexer does.

[allevato]

@swift-ci please test