Searched refs:_tokenize (Results 1 – 1 of 1) sorted by relevance
212 for tok in self._tokenize(source):215 def _tokenize(self, source): member in CTokenizer