org.jeditor.scripts.tokenmarkers

Class CsvTokenMarker

java.lang.Object

org.jeditor.scripts.base.TokenMarker

org.jeditor.scripts.tokenmarkers.CsvTokenMarker

Direct Known Subclasses:

Csv2TokenMarker

public class CsvTokenMarker
extends TokenMarker

CSV token marker with commas as delimiters.

Nested Class Summary

Nested classes/interfaces inherited from class org.jeditor.scripts.base.TokenMarker

TokenMarker.LineInfo

Field Summary

Fields

Modifier and Type	Field and Description
static byte	VALUE

Fields inherited from class org.jeditor.scripts.base.TokenMarker

currentTokenOffset, firstToken, lastLine, lastToken, length, lineIndex, lineInfo, nextLineRequested, offsetGap, syntaxListener

Constructor Summary

Constructors

Constructor and Description
CsvTokenMarker()
CsvTokenMarker(char delim)

Method Summary

Modifier and Type	Method and Description
protected byte	markTokensImpl(byte token, javax.swing.text.Segment line, int lineIndex, boolean externalTokenizer) Splits a line up into tokens.
boolean	supportsMultilineTokens() Returns true if the token marker supports tokens that span multiple lines.

Methods inherited from class org.jeditor.scripts.base.TokenMarker

addToken, deleteLines, ensureCapacity, getLineCount, getLineIndex, getMaxLineWidth, getOffsetGap, getSyntaxListener, insertLines, isNextLineRequested, markTokens, markTokens, setLineWidth, setOffsetGap, setSyntaxListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VALUE

public static final byte VALUE

See Also:

Constant Field Values

Constructor Detail

CsvTokenMarker

public CsvTokenMarker(char delim)

CsvTokenMarker

public CsvTokenMarker()

Method Detail

markTokensImpl

protected byte markTokensImpl(byte token,
                              javax.swing.text.Segment line,
                              int lineIndex,
                              boolean externalTokenizer)

Splits a line up into tokens.

Specified by:

markTokensImpl in class TokenMarker

Parameters:

token - The initial token type for this line

line - The line to be tokenized

lineIndex - The index of the line in the document, starting at 0

externalTokenizer - true for an external tokenizer

Returns:

The initial token type for the next line

supportsMultilineTokens

public boolean supportsMultilineTokens()

Description copied from class: TokenMarker

Returns true if the token marker supports tokens that span multiple lines. If this is true, the object using this token marker is required to pass all lines in the document to the markTokens() method (in turn). The default implementation returns true. It should be overridden to return false on simpler token markers for increased speed.

Overrides:

supportsMultilineTokens in class TokenMarker

Returns:

true