a b/@s`ddlmZddlZejeeeedddlmZddlmZm Z m Z eZ GdddeZ dS) )absolute_importN)BOLEOLEOF NOT_FOUND)Errors)rrrc@sdeZdZdZdddZddZdd Zd d Zd d ZddZ ddZ ddZ dddZ ddZ dS)Scannera A Scanner is used to read tokens from a stream of characters using the token set specified by a Plex.Lexicon. Constructor: Scanner(lexicon, stream, name = '') See the docstring of the __init__ method for details. Methods: See the docstrings of the individual methods for more information. read() --> (value, text) Reads the next lexical token from the stream. position() --> (name, line, col) Returns the position of the last token read using the read() method. begin(state_name) Causes scanner to change state. produce(value [, text]) Causes return of a token value to the caller of the Scanner. NcCsd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ ||_ ||_ ||_ g|_d|_|dd|_d|_d|_t|_d|_|dur|d|d |_|_dS)a Scanner(lexicon, stream, name = '') |lexicon| is a Plex.Lexicon instance specifying the lexical tokens to be recognised. |stream| can be a file object or anything which implements a compatible read() method. |name| is optional, and may be the name of the file being scanned or any other identifying string. rr rN)tracebuffer buf_start_posnext_poscur_poscur_line start_pos start_line start_coltext state_namelexiconstreamnamequeue initial_statebegincur_line_startrcur_char input_state)selfrrrZ initial_posr!d/Users/vegardjervell/Documents/master/model/venv/lib/python3.9/site-packages/Cython/Plex/Scanners.py__init__Is0  zScanner.__init__cCsh|j}|sV|\|_}|dur4|d|q|||j}|dur||q|d}|d=|S)a" Read the next lexical token from the stream and return a tuple (value, text), where |value| is the value associated with the token as specified by the Lexicon, and |text| is the actual string read from the stream. Returns (None, '') on end of file. Nr)r scan_a_tokenrproduceeofZperform)r ractionvalueresultr!r!r"readqs   z Scanner.readcCs|j|_|j|_|j|j|_|}|durp|jrJtd||j|jf|j |j|j |j|j }||fS|j|jkr|j t ur| |j dus|j turdSt||jdS)z Read the next input sequence recognised by the machine and return (text, action). Returns ('', None) on end of file. Nz"Scanner: read: Performing %s %d:%d)r N)rrrrrrrun_machine_inlinedr printr rrr next_charrrZUnrecognizedInputr)r r'rr!r!r"r$s*     zScanner.scan_a_tokencCs|j}|j}|j}|j}|j}|j}|j}|j}|j} t |} d\} } } }}}}|j }|rvt d|d||t |f|d}|dur|||||||f\} } } }}}}|}| |t}|tur|o| d}|r|rt d|d|}|dkr|}|| }|| kr||}|d7}nh|j| }|jd }|j|d|}||_| |7} | |_t |} ||8}|r||}|d7}nd }|d krt}d }n|st}d }n|}nP|d krd }d}n<|dkr|d7}|}}t}d}n|d krt}d}nd }qV|rt d| dur:| | | ||||f\}}}}}}}nd}qDqV||_||_||_||_||_||_|r|durt d||S)z; Inlined version of run_machine for speed. )Nrrrr rrzState %d, %d/%d:%s -->numberr'NelsezState %drir  r ZblockedzDoing %s)rrrrrrrr rlenr r,reprgetrrrr*rrr)r staterrrrrrr rbuf_lenZb_actionZ b_cur_posZ b_cur_lineZb_cur_line_startZ b_cur_charZ b_input_stateZ b_next_posr r'cZ new_stateZ buf_indexdiscarddatar!r!r"r+s               zScanner.run_machine_inlinedcCs|j}|jr tdd||jf|dkrh|j|_|}|dkrNt|_d|_q|s`t|_d|_q||_nd|dkr~d|_d|_nN|dkr|jd7_|j|_ |_t |_d|_n|dkrt |_d|_nd |_|jrtd ||j|jfdS) NzScanner: next: %s [%d] %dz rr0r r1r2r3r z--> [%d] %d %r) rr r,rrZ read_charrrrrrr)r rr9r!r!r"r- s6zScanner.next_charcCs|j|j|jfS)am Return a tuple (name, line, col) representing the location of the last token read using the read() method. |name| is the name that was provided to the Scanner constructor; |line| is the line number in the stream (1-based); |col| is the position within the line of the first character of the token (0-based). )rrrr r!r!r"position)s zScanner.positioncCs|S)zOPython accessible wrapper around position(), only for error reporting. )r=r<r!r!r" get_position4szScanner.get_positioncCs|j||_||_dS)z8Set the current state of the scanner to the named state.N)rZget_initial_staterr)r rr!r!r"r9s z Scanner.begincCs"|dur|j}|j||fdS)a Called from an action procedure, causes |value| to be returned as the token value from read(). If |text| is supplied, it is returned in place of the scanned text. produce() can be called more than once during a single call to an action procedure, in which case the tokens are queued up and returned one at a time by subsequent calls to read(), until the queue is empty, whereupon scanning resumes. N)rrappend)r r(rr!r!r"r%?s zScanner.producecCsdS)z_ Override this method if you want something to be done at end of file. Nr!r<r!r!r"r&Nsz Scanner.eof)r N)N)__name__ __module__ __qualname____doc__r#r*r$r+r-r=r>rr%r&r!r!r!r"r s1 (j  r ) __future__rZcythonZdeclareobjectr rZRegexpsrrrrr r!r!r!r" s