Fix the streaming row counter for S3 CSV uploads
Your task
Your Lambda counts the rows in a CSV file that lands in S3. The first version read the whole file into memory and split it into lines, so real uploads of a few gigabytes ran out of memory. The new version reads the object as a stream of chunks and keeps only a few counters.
An assistant wrote count_csv_rows_streaming(chunks, has_header). It is in the editor. It has a bug. Find it and fix it.
Input and output:
chunks: the file's text as a list of strings, in order. The file is all the chunks joined together. A chunk can be empty, and a chunk can end anywhere, even in the middle of a row.has_header: true when the first row of the file is a header row.- Return the number of data rows, as an integer.
Rules the tests check:
- Only
\nends a row. Every other character is part of a row,\rincluded. - A file with no characters has 0 rows.
- Otherwise, the number of rows is the number of
\ncharacters in the file, plus 1 when the file's last character is not\n. - So a final
\ndoes not start an extra empty row, but every other\nends a row, even when the row is empty. - When
has_headeris true and there is at least 1 row, the first row is the header and is not counted.
Examples
count_csv_rows_streaming(chunks: list[str], has_header: bool) -> intExample 1
Input
count_csv_rows_streaming(["a,b\n1,2\n3,4\n"], True)Output
2Example 2
Input
count_csv_rows_streaming(["a,b\n1", ",2\n3,", "4"], False)Output
3