Enum syntax::tokenstream::TokenTree

pub enum TokenTree {
    Token(Span, Token),
    Delimited(Span, Delimited),
}

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

When the main rust parser encounters a syntax-extension invocation, it parses the arguments to the invocation as a token-tree. This is a very loose structure, such that all sorts of different AST-fragments can be passed to syntax extensions using a uniform type.

If the syntax extension is an MBE macro, it will attempt to match its LHS token tree against the provided token tree, and if it finds a match, will transcribe the RHS token tree, splicing in any captured macro_parser::matched_nonterminals into the SubstNts it finds.

The RHS of an MBE macro is the only place SubstNts are substituted. Nothing special happens to misnamed or misplaced SubstNts.

Variants

Token(Span, Token)

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

A single token

Delimited(Span, Delimited)

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

A delimited sequence of token trees

Methods

impl TokenTree

pub fn parse(
    cx: &ExtCtxt,
    mtch: &[TokenTree],
    tts: TokenStream
) -> NamedParseResult

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Use this token tree as a matcher to parse given tts.

pub fn eq_unspanned(&self, other: &TokenTree) -> bool

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Check if this TokenTree is equal to the other, regardless of span information.

pub fn span(&self) -> Span

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Retrieve the TokenTree's span.

pub fn set_span(&mut self, span: Span)

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Modify the TokenTree's span inplace.

pub fn eq_token(&self, t: Token) -> bool

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Indicates if the stream is a token that is equal to the provided token.

pub fn joint(self) -> TokenStream

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

Trait Implementations

impl Debug for TokenTree

fn fmt(&self, __arg_0: &mut Formatter) -> Result

Formats the value using the given formatter.

impl Clone for TokenTree

fn clone(&self) -> TokenTree

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl PartialEq for TokenTree

fn eq(&self, __arg_0: &TokenTree) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, __arg_0: &TokenTree) -> bool

This method tests for !=.

impl Eq for TokenTree

impl Encodable for TokenTree

fn encode<__S: Encoder>(&self, __arg_0: &mut __S) -> Result<(), __S::Error>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

impl Decodable for TokenTree

fn decode<__D: Decoder>(__arg_0: &mut __D) -> Result<TokenTree, __D::Error>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?

impl Hash for TokenTree

fn hash<__H: Hasher>(&self, __arg_0: &mut __H)

Feeds this value into the given [Hasher].

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given [Hasher].

impl From<TokenTree> for TokenStream

fn from(tt: TokenTree) -> TokenStream

Performs the conversion.

impl ToTokens for TokenTree

Important traits for Vec<u8>

impl Write for Vec<u8>

fn to_tokens(&self, _cx: &ExtCtxt) -> Vec<TokenTree>

🔬 This is a nightly-only experimental API. (rustc_private)

this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via Cargo.toml instead?