PosToken

The PosToken class is a Token associated with the position of the token in the document.

These Tokens are created by the TokenizedDocument through the TokenizedDocument.applySyntax() method.

Methods

The PosToken has the following methods:

• The PosToken.getId() method return the id of the Token
• The PosToken.getLength() method return the length of the Token
• The PosToken.offset field is the position of the Token from the beginning of the document
• The PosToken.getText() method return the text of the token