Token representing piece of Robot Framework data. More...

Inheritance diagram for robot.parsing.lexer.tokens.Token:

Public Member Functions
def	__eq__ (self, other)

def	__init__ (self, type=None, value=None, lineno=-1, col_offset=-1, error=None)

def	__repr__ (self)

def	__str__ (self)

def	set_error (self, error, fatal=False)

def	tokenize_variables (self)
	Tokenizes possible variables in token value. More...

Public Attributes
	col_offset

	error

	lineno

	type

	value

Static Public Attributes
	ALLOW_VARIABLES

string	ARGUMENT = 'ARGUMENT'

string	ARGUMENTS = 'ARGUMENTS'

string	AS = 'AS'

string	ASSIGN = 'ASSIGN'

string	BREAK = 'BREAK'

string	COMMENT = 'COMMENT'

string	COMMENT_HEADER = 'COMMENT HEADER'

string	CONFIG = 'CONFIG'

string	CONTINUATION = 'CONTINUATION'

string	CONTINUE = 'CONTINUE'

string	DEFAULT_TAGS = 'DEFAULT TAGS'

string	DOCUMENTATION = 'DOCUMENTATION'

string	ELSE = 'ELSE'

string	ELSE_IF = 'ELSE IF'

string	END = 'END'

string	EOL = 'EOL'

string	EOS = 'EOS'

string	ERROR = 'ERROR'

string	EXCEPT = 'EXCEPT'

string	FATAL_ERROR = 'FATAL ERROR'

string	FINALLY = 'FINALLY'

string	FOR = 'FOR'

string	FOR_SEPARATOR = 'FOR SEPARATOR'

string	FORCE_TAGS = 'FORCE TAGS'

	HEADER_TOKENS

string	IF = 'IF'

string	INLINE_IF = 'INLINE IF'

string	KEYWORD = 'KEYWORD'

string	KEYWORD_HEADER = 'KEYWORD HEADER'

string	KEYWORD_NAME = 'KEYWORD NAME'

string	KEYWORD_TAGS = 'KEYWORD TAGS'

string	LIBRARY = 'LIBRARY'

string	METADATA = 'METADATA'

string	NAME = 'NAME'

	NON_DATA_TOKENS

string	OPTION = 'OPTION'

string	RESOURCE = 'RESOURCE'

string	RETURN = 'RETURN'

string	RETURN_SETTING = RETURN

string	RETURN_STATEMENT = 'RETURN STATEMENT'

string	SEPARATOR = 'SEPARATOR'

string	SETTING_HEADER = 'SETTING HEADER'

	SETTING_TOKENS

string	SETUP = 'SETUP'

string	SUITE_SETUP = 'SUITE SETUP'

string	SUITE_TEARDOWN = 'SUITE TEARDOWN'

string	TAGS = 'TAGS'

string	TASK_HEADER = 'TASK HEADER'

string	TEARDOWN = 'TEARDOWN'

string	TEMPLATE = 'TEMPLATE'

string	TEST_SETUP = 'TEST SETUP'

string	TEST_TEARDOWN = 'TEST TEARDOWN'

string	TEST_TEMPLATE = 'TEST TEMPLATE'

string	TEST_TIMEOUT = 'TEST TIMEOUT'

string	TESTCASE_HEADER = 'TESTCASE HEADER'

string	TESTCASE_NAME = 'TESTCASE NAME'

string	TIMEOUT = 'TIMEOUT'

string	TRY = 'TRY'

string	VARIABLE = 'VARIABLE'

string	VARIABLE_HEADER = 'VARIABLE HEADER'

string	VARIABLES = 'VARIABLES'

string	WHILE = 'WHILE'

string	WITH_NAME = 'WITH NAME'

Properties
	end_col_offset = property

Private Member Functions
def	_tokenize_no_variables (self)

def	_tokenize_variables (self, variables)

def	end_col_offset (self)

Private Attributes
	_add_eos_after

	_add_eos_before

Static Private Attributes
list	__slots__

Detailed Description

Token representing piece of Robot Framework data.

Each token has type, value, line number, column offset and end column
offset in :attr:`type`, :attr:`value`, :attr:`lineno`, :attr:`col_offset`
and :attr:`end_col_offset` attributes, respectively. Tokens representing
error also have their error message in :attr:`error` attribute.

Token types are declared as class attributes such as :attr:`SETTING_HEADER`
and :attr:`EOL`. Values of these constants have changed slightly in Robot
Framework 4.0 and they may change again in the future. It is thus safer
to use the constants, not their values, when types are needed. For example,
use ``Token(Token.EOL)`` instead of ``Token('EOL')`` and
``token.type == Token.EOL`` instead of ``token.type == 'EOL'``.

If :attr:`value` is not given when :class:`Token` is initialized and
:attr:`type` is :attr:`IF`, :attr:`ELSE_IF`, :attr:`ELSE`, :attr:`FOR`,
:attr:`END`, :attr:`WITH_NAME` or :attr:`CONTINUATION`, the value is
automatically set to the correct marker value like ``'IF'`` or ``'ELSE IF'``.
If :attr:`type` is :attr:`EOL` in this case, the value is set to ``'\\n'``.

Definition at line 39 of file tokens.py.

Constructor & Destructor Documentation

◆ init()

def robot.parsing.lexer.tokens.Token.__init__	(	self,
		type = `None`,
		value = `None`,
		lineno = `-1`,
		col_offset = `-1`,
		error = `None`
	)

Definition at line 157 of file tokens.py.

Member Function Documentation

◆ eq()

def robot.parsing.lexer.tokens.Token.__eq__	(	self,
		other
	)

Definition at line 229 of file tokens.py.

◆ repr()

def robot.parsing.lexer.tokens.Token.__repr__ ( self )

Definition at line 223 of file tokens.py.

◆ str()

def robot.parsing.lexer.tokens.Token.__str__ ( self )

Definition at line 220 of file tokens.py.

◆ _tokenize_no_variables()

def robot.parsing.lexer.tokens.Token._tokenize_no_variables ( self )

private

Definition at line 204 of file tokens.py.

◆ _tokenize_variables()

def robot.parsing.lexer.tokens.Token._tokenize_variables	(	self,
		variables
	)

private

Definition at line 207 of file tokens.py.

◆ end_col_offset()

def robot.parsing.lexer.tokens.Token.end_col_offset ( self )

private

Definition at line 179 of file tokens.py.

◆ set_error()

def robot.parsing.lexer.tokens.Token.set_error	(	self,
		error,
		fatal = `False`
	)

Definition at line 184 of file tokens.py.

◆ tokenize_variables()

def robot.parsing.lexer.tokens.Token.tokenize_variables ( self )

Tokenizes possible variables in token value.

    Yields the token itself if the token does not allow variables (see
    :attr:`Token.ALLOW_VARIABLES`) or its value does not contain
    variables. Otherwise yields variable tokens as well as tokens
    before, after, or between variables so that they have the same
    type as the original token.

Definition at line 196 of file tokens.py.

Member Data Documentation

◆ slots

list robot.parsing.lexer.tokens.Token.__slots__

staticprivate

Initial value:

= ['type', 'value', 'lineno', 'col_offset', 'error',

'_add_eos_before', '_add_eos_after']

Definition at line 154 of file tokens.py.

◆ _add_eos_after

robot.parsing.lexer.tokens.Token._add_eos_after

private

Definition at line 174 of file tokens.py.

◆ _add_eos_before

robot.parsing.lexer.tokens.Token._add_eos_before

private

Definition at line 173 of file tokens.py.

◆ ALLOW_VARIABLES

robot.parsing.lexer.tokens.Token.ALLOW_VARIABLES

static

Initial value:

=  frozenset((
        NAME,
        ARGUMENT,
        TESTCASE_NAME,
        KEYWORD_NAME
    ))

Definition at line 147 of file tokens.py.

◆ ARGUMENT

string robot.parsing.lexer.tokens.Token.ARGUMENT = 'ARGUMENT'

static

Definition at line 78 of file tokens.py.

◆ ARGUMENTS

string robot.parsing.lexer.tokens.Token.ARGUMENTS = 'ARGUMENTS'

static

Definition at line 70 of file tokens.py.

◆ AS

string robot.parsing.lexer.tokens.Token.AS = 'AS'

static

Definition at line 92 of file tokens.py.

◆ ASSIGN

string robot.parsing.lexer.tokens.Token.ASSIGN = 'ASSIGN'

static

Definition at line 79 of file tokens.py.

◆ BREAK

string robot.parsing.lexer.tokens.Token.BREAK = 'BREAK'

static

Definition at line 96 of file tokens.py.

◆ col_offset

robot.parsing.lexer.tokens.Token.col_offset

Definition at line 170 of file tokens.py.

◆ COMMENT

string robot.parsing.lexer.tokens.Token.COMMENT = 'COMMENT'

static

Definition at line 100 of file tokens.py.

◆ COMMENT_HEADER

string robot.parsing.lexer.tokens.Token.COMMENT_HEADER = 'COMMENT HEADER'

static

Definition at line 46 of file tokens.py.

◆ CONFIG

string robot.parsing.lexer.tokens.Token.CONFIG = 'CONFIG'

static

Definition at line 102 of file tokens.py.

◆ CONTINUATION

string robot.parsing.lexer.tokens.Token.CONTINUATION = 'CONTINUATION'

static

Definition at line 101 of file tokens.py.

◆ CONTINUE

string robot.parsing.lexer.tokens.Token.CONTINUE = 'CONTINUE'

static

Definition at line 95 of file tokens.py.

◆ DEFAULT_TAGS

string robot.parsing.lexer.tokens.Token.DEFAULT_TAGS = 'DEFAULT TAGS'

static

Definition at line 60 of file tokens.py.

◆ DOCUMENTATION

string robot.parsing.lexer.tokens.Token.DOCUMENTATION = 'DOCUMENTATION'

static

Definition at line 51 of file tokens.py.

◆ ELSE

string robot.parsing.lexer.tokens.Token.ELSE = 'ELSE'

static

Definition at line 88 of file tokens.py.

◆ ELSE_IF

string robot.parsing.lexer.tokens.Token.ELSE_IF = 'ELSE IF'

static

Definition at line 87 of file tokens.py.

◆ END

string robot.parsing.lexer.tokens.Token.END = 'END'

static

Definition at line 84 of file tokens.py.

◆ EOL

string robot.parsing.lexer.tokens.Token.EOL = 'EOL'

static

Definition at line 103 of file tokens.py.

◆ EOS

string robot.parsing.lexer.tokens.Token.EOS = 'EOS'

static

Definition at line 104 of file tokens.py.

◆ ERROR

string robot.parsing.lexer.tokens.Token.ERROR = 'ERROR'

static

Definition at line 106 of file tokens.py.

◆ error

robot.parsing.lexer.tokens.Token.error

Definition at line 171 of file tokens.py.

◆ EXCEPT

string robot.parsing.lexer.tokens.Token.EXCEPT = 'EXCEPT'

static

Definition at line 90 of file tokens.py.

◆ FATAL_ERROR

string robot.parsing.lexer.tokens.Token.FATAL_ERROR = 'FATAL ERROR'

static

Definition at line 107 of file tokens.py.

◆ FINALLY

string robot.parsing.lexer.tokens.Token.FINALLY = 'FINALLY'

static

Definition at line 91 of file tokens.py.

◆ FOR

string robot.parsing.lexer.tokens.Token.FOR = 'FOR'

static

Definition at line 82 of file tokens.py.

◆ FOR_SEPARATOR

string robot.parsing.lexer.tokens.Token.FOR_SEPARATOR = 'FOR SEPARATOR'

static

Definition at line 83 of file tokens.py.

◆ FORCE_TAGS

string robot.parsing.lexer.tokens.Token.FORCE_TAGS = 'FORCE TAGS'

static

Definition at line 59 of file tokens.py.

◆ HEADER_TOKENS

robot.parsing.lexer.tokens.Token.HEADER_TOKENS

static

Initial value:

=  frozenset((
        SETTING_HEADER,
        VARIABLE_HEADER,
        TESTCASE_HEADER,
        TASK_HEADER,
        KEYWORD_HEADER,
        COMMENT_HEADER
    ))

Definition at line 139 of file tokens.py.

◆ IF

string robot.parsing.lexer.tokens.Token.IF = 'IF'

static

Definition at line 85 of file tokens.py.

◆ INLINE_IF

string robot.parsing.lexer.tokens.Token.INLINE_IF = 'INLINE IF'

static

Definition at line 86 of file tokens.py.

◆ KEYWORD

string robot.parsing.lexer.tokens.Token.KEYWORD = 'KEYWORD'

static

Definition at line 80 of file tokens.py.

◆ KEYWORD_HEADER

string robot.parsing.lexer.tokens.Token.KEYWORD_HEADER = 'KEYWORD HEADER'

static

Definition at line 45 of file tokens.py.

◆ KEYWORD_NAME

string robot.parsing.lexer.tokens.Token.KEYWORD_NAME = 'KEYWORD NAME'

static

Definition at line 49 of file tokens.py.

◆ KEYWORD_TAGS

string robot.parsing.lexer.tokens.Token.KEYWORD_TAGS = 'KEYWORD TAGS'

static

Definition at line 61 of file tokens.py.

◆ LIBRARY

string robot.parsing.lexer.tokens.Token.LIBRARY = 'LIBRARY'

static

Definition at line 62 of file tokens.py.

◆ lineno

robot.parsing.lexer.tokens.Token.lineno

Definition at line 169 of file tokens.py.

◆ METADATA

string robot.parsing.lexer.tokens.Token.METADATA = 'METADATA'

static

Definition at line 54 of file tokens.py.

◆ NAME

string robot.parsing.lexer.tokens.Token.NAME = 'NAME'

static

Definition at line 76 of file tokens.py.

◆ NON_DATA_TOKENS

robot.parsing.lexer.tokens.Token.NON_DATA_TOKENS

static

Initial value:

=  frozenset((
        SEPARATOR,
        COMMENT,
        CONTINUATION,
        EOL,
        EOS
    ))

Definition at line 109 of file tokens.py.

◆ OPTION

string robot.parsing.lexer.tokens.Token.OPTION = 'OPTION'

static

Definition at line 97 of file tokens.py.

◆ RESOURCE

string robot.parsing.lexer.tokens.Token.RESOURCE = 'RESOURCE'

static

Definition at line 63 of file tokens.py.

◆ RETURN

string robot.parsing.lexer.tokens.Token.RETURN = 'RETURN'

static

Definition at line 73 of file tokens.py.

◆ RETURN_SETTING

string robot.parsing.lexer.tokens.Token.RETURN_SETTING = RETURN

static

Definition at line 74 of file tokens.py.

◆ RETURN_STATEMENT

string robot.parsing.lexer.tokens.Token.RETURN_STATEMENT = 'RETURN STATEMENT'

static

Definition at line 94 of file tokens.py.

◆ SEPARATOR

string robot.parsing.lexer.tokens.Token.SEPARATOR = 'SEPARATOR'

static

Definition at line 99 of file tokens.py.

◆ SETTING_HEADER

string robot.parsing.lexer.tokens.Token.SETTING_HEADER = 'SETTING HEADER'

static

Definition at line 41 of file tokens.py.

◆ SETTING_TOKENS

robot.parsing.lexer.tokens.Token.SETTING_TOKENS

static

Initial value:

=  frozenset((
        DOCUMENTATION,
        SUITE_SETUP,
        SUITE_TEARDOWN,
        METADATA,
        TEST_SETUP,
        TEST_TEARDOWN,
        TEST_TEMPLATE,
        TEST_TIMEOUT,
        FORCE_TAGS,
        DEFAULT_TAGS,
        KEYWORD_TAGS,
        LIBRARY,
        RESOURCE,
        VARIABLES,
        SETUP,
        TEARDOWN,
        TEMPLATE,
        TIMEOUT,
        TAGS,
        ARGUMENTS,
        RETURN
    ))

Definition at line 116 of file tokens.py.

◆ SETUP

string robot.parsing.lexer.tokens.Token.SETUP = 'SETUP'

static

Definition at line 65 of file tokens.py.

◆ SUITE_SETUP

string robot.parsing.lexer.tokens.Token.SUITE_SETUP = 'SUITE SETUP'

static

Definition at line 52 of file tokens.py.

◆ SUITE_TEARDOWN

string robot.parsing.lexer.tokens.Token.SUITE_TEARDOWN = 'SUITE TEARDOWN'

static

Definition at line 53 of file tokens.py.

◆ TAGS

string robot.parsing.lexer.tokens.Token.TAGS = 'TAGS'

static

Definition at line 69 of file tokens.py.

◆ TASK_HEADER

string robot.parsing.lexer.tokens.Token.TASK_HEADER = 'TASK HEADER'

static

Definition at line 44 of file tokens.py.

◆ TEARDOWN

string robot.parsing.lexer.tokens.Token.TEARDOWN = 'TEARDOWN'

static

Definition at line 66 of file tokens.py.

◆ TEMPLATE

string robot.parsing.lexer.tokens.Token.TEMPLATE = 'TEMPLATE'

static

Definition at line 67 of file tokens.py.

◆ TEST_SETUP

string robot.parsing.lexer.tokens.Token.TEST_SETUP = 'TEST SETUP'

static

Definition at line 55 of file tokens.py.

◆ TEST_TEARDOWN

string robot.parsing.lexer.tokens.Token.TEST_TEARDOWN = 'TEST TEARDOWN'

static

Definition at line 56 of file tokens.py.

◆ TEST_TEMPLATE

string robot.parsing.lexer.tokens.Token.TEST_TEMPLATE = 'TEST TEMPLATE'

static

Definition at line 57 of file tokens.py.

◆ TEST_TIMEOUT

string robot.parsing.lexer.tokens.Token.TEST_TIMEOUT = 'TEST TIMEOUT'

static

Definition at line 58 of file tokens.py.

◆ TESTCASE_HEADER

string robot.parsing.lexer.tokens.Token.TESTCASE_HEADER = 'TESTCASE HEADER'

static

Definition at line 43 of file tokens.py.

◆ TESTCASE_NAME

string robot.parsing.lexer.tokens.Token.TESTCASE_NAME = 'TESTCASE NAME'

static

Definition at line 48 of file tokens.py.

◆ TIMEOUT

string robot.parsing.lexer.tokens.Token.TIMEOUT = 'TIMEOUT'

static

Definition at line 68 of file tokens.py.

◆ TRY

string robot.parsing.lexer.tokens.Token.TRY = 'TRY'

static

Definition at line 89 of file tokens.py.

◆ type

robot.parsing.lexer.tokens.Token.type

Definition at line 158 of file tokens.py.

◆ value

robot.parsing.lexer.tokens.Token.value

Definition at line 168 of file tokens.py.

◆ VARIABLE

string robot.parsing.lexer.tokens.Token.VARIABLE = 'VARIABLE'

static

Definition at line 77 of file tokens.py.

◆ VARIABLE_HEADER

string robot.parsing.lexer.tokens.Token.VARIABLE_HEADER = 'VARIABLE HEADER'

static

Definition at line 42 of file tokens.py.

◆ VARIABLES

string robot.parsing.lexer.tokens.Token.VARIABLES = 'VARIABLES'

static

Definition at line 64 of file tokens.py.

◆ WHILE

string robot.parsing.lexer.tokens.Token.WHILE = 'WHILE'

static

Definition at line 93 of file tokens.py.

◆ WITH_NAME

string robot.parsing.lexer.tokens.Token.WITH_NAME = 'WITH NAME'

static

Definition at line 81 of file tokens.py.

Property Documentation

◆ end_col_offset

robot.parsing.lexer.tokens.Token.end_col_offset = property

static

Definition at line 177 of file tokens.py.

The documentation for this class was generated from the following file:

src/robot/parsing/lexer/tokens.py

Public Member Functions

Public Attributes

Static Public Attributes

Properties

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ __eq__()

◆ __repr__()

◆ __str__()

◆ _tokenize_no_variables()

◆ _tokenize_variables()

◆ end_col_offset()

◆ set_error()

◆ tokenize_variables()

Member Data Documentation

◆ __slots__

◆ _add_eos_after

◆ _add_eos_before

◆ ALLOW_VARIABLES

◆ ARGUMENT

◆ ARGUMENTS

◆ AS

◆ ASSIGN

◆ BREAK

◆ col_offset

◆ COMMENT

◆ COMMENT_HEADER

◆ CONFIG

◆ CONTINUATION

◆ CONTINUE

◆ DEFAULT_TAGS

◆ DOCUMENTATION

◆ ELSE

◆ ELSE_IF

◆ END

◆ EOL

◆ EOS

◆ ERROR

◆ error

◆ EXCEPT

◆ FATAL_ERROR

◆ FINALLY

◆ FOR

◆ FOR_SEPARATOR

◆ FORCE_TAGS

◆ HEADER_TOKENS

◆ IF

◆ INLINE_IF

◆ KEYWORD

◆ KEYWORD_HEADER

◆ KEYWORD_NAME

◆ KEYWORD_TAGS

◆ LIBRARY

◆ lineno

◆ METADATA

◆ NAME

◆ NON_DATA_TOKENS

◆ OPTION

◆ RESOURCE

◆ RETURN

◆ RETURN_SETTING

◆ RETURN_STATEMENT

◆ SEPARATOR

◆ SETTING_HEADER

◆ SETTING_TOKENS

◆ SETUP

◆ SUITE_SETUP

◆ SUITE_TEARDOWN

◆ TAGS

◆ TASK_HEADER

◆ TEARDOWN

◆ TEMPLATE

◆ TEST_SETUP

◆ TEST_TEARDOWN

◆ TEST_TEMPLATE

◆ init()

◆ eq()

◆ repr()

◆ str()

◆ slots