All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class com.micronova.util.ITokenizer 

java.lang.Object
   |
   +----com.micronova.util.ITokenizer
public class ITokenizer
extends Object
Simple tokenizer for delimited data (such as TAB/NEWLINE-delimited table). It can also handle multiple quote pairs and escapes.

Version:
1.0

Variable Index

o _buffer
buffer used for parsing.
o _EOF
returned by _parse() when EOF is read.
o _map
character map for characters 0 - 127.
o BEGINQUOTE
bitflags for 'begin qoute' characters.
o ENDQUOTE
bit flag for 'end quote' characters.
o ESCAPE
flag for escape characters (e.g., \)
o IGNORED
flag for ignored characters.
o NONE
flag for ordinary characters.
o QUOTECODE
bit flag for quote pair identifier.
o SEMISEPARATOR
flag for semi-separator characters; semi-separators are only significant after a word or quote.
o SEPARATOR
flag for separator characters (e.g., TAB)
o TERMINATOR
flag for terminator characters (e.g., NEWLINE)

Constructor Index

o ITokenizer()
Constructs an empty Tokenizer.

Method Index

o _parse(Reader)
Parses and returns 'result' code, setting up the 'buffer'.
o _processEscape(StringBuffer, char, char)
Processes escapes.
o _processQuote(StringBuffer, int, StringBuffer)
Processes quotes.
o getRow(Reader)
Gets a row of words (up to a terminator) as an IStringVector.
o getTable(Reader)
Gets all rows (up to EOF) into IStringTable.
o getWord(Reader)
Gets the next 'word' from the reader.
o setCharacters(String, int)
Set character flag for each one in given string.

Variables

o NONE 
 public static final byte NONE
flag for ordinary characters.

o IGNORED 
 public static final byte IGNORED
flag for ignored characters.

o ESCAPE 
 public static final byte ESCAPE
flag for escape characters (e.g., \)

o SEPARATOR 
 public static final byte SEPARATOR
flag for separator characters (e.g., TAB)

o SEMISEPARATOR 
 public static final byte SEMISEPARATOR
flag for semi-separator characters; semi-separators are only significant after a word or quote.

o TERMINATOR 
 public static final byte TERMINATOR
flag for terminator characters (e.g., NEWLINE)

o _EOF 
 protected static final byte _EOF
returned by _parse() when EOF is read.

o BEGINQUOTE 
 public static final byte BEGINQUOTE
bitflags for 'begin qoute' characters. Lower bits idenifies the matching pair.

o ENDQUOTE 
 public static final byte ENDQUOTE
bit flag for 'end quote' characters. Lower bits identifies the matching pair.

o QUOTECODE 
 public static final byte QUOTECODE
bit flag for quote pair identifier.

o _buffer 
 protected StringBuffer _buffer
buffer used for parsing.

o _map 
 protected byte _map[]
character map for characters 0 - 127.

Constructors

o ITokenizer 
 public ITokenizer()
Constructs an empty Tokenizer.

Methods

o setCharacters 
 public void setCharacters(String s,
                           int flag)
Set character flag for each one in given string.

o _processQuote 
 protected void _processQuote(StringBuffer buffer,
                              int quoteCode,
                              StringBuffer quoteBuffer)
Processes quotes. You can replace the buffer content here if necessary.

o _processEscape 
 protected void _processEscape(StringBuffer buffer,
                               char escapeChar,
                               char escaped)
Processes escapes. You can modify the buffer content here if necessary (e.g., to handle '\n' etc.)

o _parse 
 protected int _parse(Reader in)
Parses and returns 'result' code, setting up the 'buffer'.

o getWord 
 public String getWord(Reader in)
Gets the next 'word' from the reader.

o getRow 
 public IStringVector getRow(Reader in)
Gets a row of words (up to a terminator) as an IStringVector.

o getTable 
 public IStringTable getTable(Reader in)
Gets all rows (up to EOF) into IStringTable. 

All Packages  Class Hierarchy  This Package  Previous  Next  Index