Skip to content
CCC Python Course

String processing II: tokenizing with state

Stage
3
Module
M3.4
Lessons
1

In this module

  • Parse a string into tokens based on separators or delimiters.
  • Track state while building tokens to handle multi-character delimiters or contextual rules.
  • Build a list of tokens by appending complete tokens when a delimiter is found.

Before this module

M3.3String processing I: counting, runs and building

Lessons

  1. 1String processing II (tokenizing with state)

Practice

Try these on the judges. Each link opens the problem on WMOJ or DMOJ.

  1. 2019 J3
    Cold Compress (opens on DMOJ in a new tab) DMOJ

    Decompress a run-length encoded string one token at a time.

    Why DMOJ: A state-tracking tokenizing problem from the same era as this lesson's examples.

  2. 2023 J3
    Special Event (opens on WMOJ in a new tab) WMOJ

    Read a line of tokens and track the maximum, joining names on a tie.

  3. 2025 J3
    Product Codes (opens on WMOJ in a new tab) WMOJ

    Tokenize a line into distinct character classes, including signed multi-digit numbers.