Code RoomEnglish word tokenizer
MediumPrep Room Coding #1023

English word tokenizer

CodingAlgorithms & data structuresMid–Senior~25 min

Tokenize a line of plain English into word tokens. A word is a maximal run of letters (A-Z, a-z) and apostrophes ('), where an apostrophe only counts as part of a word if it sits BETWEEN two letters (so contractions like don't and possessives like cat's stay whole, but leading/trailing quotes are dropped). All other characters (digits, spaces, punctuation) are separators. Lowercase every token. Return the list of word tokens in order.

Implement
tokenize_words(text: str) → list[str]
Examples
in["Don't stop, it's fine."]out["don't","stop","it's","fine"]
What a strong answer looks like

State your approach and its time/space complexity out loud before you optimize. Handle the edge cases (empty input, duplicates, overflow), and say why you chose this over the brute force. Green tests are the floor, not the grade.

0:00 of about 25 min
InputExpectedGot
["Don't stop, it's fine."]["don't","stop","it's","fine"]not run yetsample