Chatbot fuzzy token match
Your task
Exact match is too strict for chatbot answers, so your eval also scores fuzzy token matches. For each output/reference pair: lowercase both strings and split them on whitespace into tokens. Take the reference's distinct tokens; count how many of them appear among the output's tokens (hits). The pair matches when 100 * hits >= threshold_pct * distinct_count, using integer arithmetic. A reference with no tokens always matches. Given equal-length lists outputs and references plus an integer threshold_pct, return the number of matching pairs. Example: outputs = ["the capital of france is paris"], references = ["paris france"], threshold_pct = 100 returns 1.
Examples
fuzzy_match_count(outputs: list[str], references: list[str], threshold_pct: int) -> intExample 1
Input
fuzzy_match_count(["the capital of france is paris"], ["paris france"], 100)Output
1