Mercurial > hg > Lemuriformes
comparison lemuriformes/chunk.py @ 6:244c29f46554
add module for chunking data
| author | Jeff Hammel <k0scist@gmail.com> |
|---|---|
| date | Sun, 10 Dec 2017 11:52:37 -0800 |
| parents | |
| children |
comparison
equal
deleted
inserted
replaced
| 5:dbbf5344868c | 6:244c29f46554 |
|---|---|
| 1 def chunk(data, chunksize=100000): | |
| 2 """ | |
| 3 chunk `data` into smaller sizes of `chunksize` | |
| 4 """ | |
| 5 | |
| 6 offset = 0 | |
| 7 chunk = data[0:chunksize] | |
| 8 while chunk: | |
| 9 yield chunk | |
| 10 offset += chunksize | |
| 11 chunk = data[offset:offset+chunksize] | |
| 12 |