tokenization
The act or process of tokenizing.
Noun
- The act or process of tokenizing.
- Something tokenized.
- This was an unlikely tokenization of the input string.
Origin
From token + -ization.
The act or process of tokenizing.
From token + -ization.