# Class: YARD::Parser::Ruby::TokenResolver

Inherits:
  Object
  - Object
  - YARD::Parser::Ruby::TokenResolver
  show all

Includes:
  Enumerable, CodeObjects::NamespaceMapper

Defined in:
  lib/yard/parser/ruby/token_resolver.rb

## Overview

Supports #each enumeration over a source’s tokens, yielding the token and a possible CodeObjects::Base associated with the constant or identifier token.

## Class Method Summary collapse

- .state_attr(*attrs) ⇒ Object

## Instance Method Summary collapse

- #each {|token, object| ... } ⇒ Object

  Iterates over each token, yielding the token and a possible code object that is associated with the token.

- #initialize(source, namespace = Registry.root) ⇒ TokenResolver constructor

  Creates a token resolver for given source.

### Methods included from CodeObjects::NamespaceMapper

#clear_separators, #default_separator, on_invalidate, #register_separator, #separators, #separators_for_type, #separators_match, #types_for_separator, #unregister_separator_by_type

## Constructor Details

### #initialize(source, namespace = Registry.root) ⇒ `TokenResolver`

Creates a token resolver for given source.

Parameters:

- source (`String`) —

  the source code to tokenize

- namespace (`CodeObjects::Base`) (defaults to: `Registry.root`) —

  the object/namespace to resolve from

Raises:

- (`ParserSyntaxError`)

    16
    17
    18
    19
    20

    # File 'lib/yard/parser/ruby/token_resolver.rb', line 16

    def initialize(source, namespace = Registry.root)
      @tokens = RubyParser.parse(source, '(tokenize)').tokens
      raise ParserSyntaxError if @tokens.empty? && !source.empty?
      @default_namespace = namespace
    end

## Class Method Details

### .state_attr(*attrs) ⇒ `Object`

    92
    93
    94
    95
    96
    97
    98

    # File 'lib/yard/parser/ruby/token_resolver.rb', line 92

    def self.state_attr(*attrs)
      attrs.each do |attr|
        define_method(attr) { @states.last[attr.to_sym] }
        define_method("#{attr}=") {|v| @states.last[attr.to_sym] = v }
        protected attr, :"#{attr}="
      end
    end

## Instance Method Details

### #each {|token, object| ... } ⇒ `Object`

Iterates over each token, yielding the token and a possible code object that is associated with the token.

#### Examples:

#####

Yielding code objects

    r = TokenResolver.new("A::B::C")
    r.each do |tok, obj|
      if obj
        puts "#{tok[0]} -> #{obj.path.inspect}"
      else
        puts "No object: #{tok.inspect}"
      end
    end

    # Prints:
    # :const -> "A"
    # No object: [:op, "::"]
    # :const -> "A::B"
    # No object: [:op, "::"]
    # :const -> "A::B::C"

Yield Parameters:

- token (`Array(Symbol,String,Array(Integer,Integer))`) —

  the current token object being iterated

- object (`CodeObjects::Base`, `nil`) —

  the fully qualified code object associated with the current token, or nil if there is no object for the yielded token.

    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90

    # File 'lib/yard/parser/ruby/token_resolver.rb', line 46

    def each
      @states = []
      push_state
      @tokens.each do |token|
        yield_obj = false

        if skip_group && [:const, :ident, :op, :period].include?(token[0])
          yield token, nil
          next
        else
          self.skip_group = false
        end

        case token[0]
        when :const
          lookup(token[0], token[1])
          yield_obj = true
          self.last_sep = nil
        when :ident
          lookup(token[0], token[1])
          yield_obj = true
          self.last_sep = nil
        when :op, :period
          self.last_sep = token[1]
          unless CodeObjects.types_for_separator(token[1])
            self.object = nil
            self.last_sep = nil
          end
        when :lparen
          push_state
        when :rparen
          pop_state
        else
          self.object = nil
        end

        yield token, (yield_obj ? object : nil)

        if next_object
          self.object = next_object
          self.next_object = nil
        end
        self.skip_group = true if yield_obj && object.nil?
      end
    end