All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class jp.kyasu.awt.util.LanguageTokenizer

java.lang.Object
   |
   +----jp.kyasu.graphics.TextCharacterIterator
           |
           +----jp.kyasu.awt.util.LanguageTokenizer

public abstract class LanguageTokenizer
extends TextCharacterIterator
implements Serializable
The LangTokenizer class is an abstract base class for all objects that parse a text into "tokens". The text should represent a program.

Version:
10 Jun 1998
Author:
Kazuki YASUMATSU

Variable Index

 o COMMENT
A constant indicating that a comment token has been read.
 o CONSTANT
A constant indicating that a constant token has been read.
 o EOT
A constant indicating that the end of the array has been read.
 o inMultiLineComment
True, if the parsing context is in the multiline comment.
 o inMultiLineConstant
True, if the parsing context is in the multiline constant.
 o KEYWORD
A constant indicating that a keyword token has been read.
 o MULTILINE_COMMENT
A constant indicating that a multiline comment token has been read.
 o MULTILINE_CONSTANT
A constant indicating that a multiline constant token has been read.
 o OTHER
A constant indicating that a token has been read.
 o tokenBegin
The beginning index of the token, inclusive.
 o tokenEnd
The ending index of the token, exclusive.

Constructor Index

 o LanguageTokenizer(Text, int, int)
Construct a LanguageTokenizer with the specified text and range.

Method Index

 o getPreferredParseBegin(Text, int, int, int)
Returns the preferred beginning index to parse after the specified text has been changed.
 o getPreferredParseEnd(Text, int, int, int)
Returns the preferred ending index to parse after the specified text has been changed.
 o nextToken()
Parses the next token from the text of this tokenizer.

Variables

 o tokenBegin
 public int tokenBegin
The beginning index of the token, inclusive.

 o tokenEnd
 public int tokenEnd
The ending index of the token, exclusive.

 o inMultiLineConstant
 public boolean inMultiLineConstant
True, if the parsing context is in the multiline constant.

 o inMultiLineComment
 public boolean inMultiLineComment
True, if the parsing context is in the multiline comment.

 o EOT
 public static final int EOT
A constant indicating that the end of the array has been read.

 o OTHER
 public static final int OTHER
A constant indicating that a token has been read.

 o KEYWORD
 public static final int KEYWORD
A constant indicating that a keyword token has been read.

 o CONSTANT
 public static final int CONSTANT
A constant indicating that a constant token has been read.

 o MULTILINE_CONSTANT
 public static final int MULTILINE_CONSTANT
A constant indicating that a multiline constant token has been read.

 o COMMENT
 public static final int COMMENT
A constant indicating that a comment token has been read.

 o MULTILINE_COMMENT
 public static final int MULTILINE_COMMENT
A constant indicating that a multiline comment token has been read.

Constructors

 o LanguageTokenizer
 public LanguageTokenizer(Text text,
                          int begin,
                          int end)
Construct a LanguageTokenizer with the specified text and range.

Parameters:
text - the text to be parsed.
begin - the beginning index to parse, inclusive.
end - the ending index to parse, exclusive.

Methods

 o getPreferredParseBegin
 public static int getPreferredParseBegin(Text text,
                                          int tokenType,
                                          int tokenBegin,
                                          int changeBefore)
Returns the preferred beginning index to parse after the specified text has been changed.

Parameters:
text - the text to be parsed.
tokenType - the token type.
tokenBegin - the beginning index of the token.
changeBefore - the index before a change has been made.
 o getPreferredParseEnd
 public static int getPreferredParseEnd(Text text,
                                        int tokenType,
                                        int tokenEnd,
                                        int changeAfter)
Returns the preferred ending index to parse after the specified text has been changed.

Parameters:
text - the text to be parsed.
tokenType - the token type.
tokenEnd - the ending index of the token.
changeAfter - the index after a change has been made.
 o nextToken
 public abstract int nextToken()
Parses the next token from the text of this tokenizer.

Returns:
the type of the token.
See Also:
tokenBegin, tokenEnd, EOT, OTHER, KEYWORD, CONSTANT, MULTILINE_CONSTANT, COMMENT, MULTILINE_COMMENT

All Packages  Class Hierarchy  This Package  Previous  Next  Index