Split a List Into Evenly Sized Chunks in Python Using Collections

In this blog, we will explore an elegant and efficient Python program to split a list into evenly sized chunks using collections. Specifically, we will focus on using the deque class from the collections module to achieve our goal. The deque class provides a convenient and performant way to pop and append elements from both ends of a collection.

Whether you’re working with large datasets, implementing parallel processing, or dealing with tasks that require dividing data into manageable portions, having a method to split a list into chunks can be invaluable. Python, being a versatile and expressive programming language, offers various approaches to achieve this task.

Let’s dive into the code and explore how to split a list into evenly sized chunks in Python using collections.

Python Program to Split a List Into Evenly Sized Chunks Using Collections

# Split a List Into Evenly Sized Chunks in Python Using Collections

from collections import deque
 
def split_list(input_list, chunk_size):
  # Create a deque object from the input list
  deque_obj = deque(input_list)
  # While the deque object is not empty
  while deque_obj:
      # Pop chunk_size elements from the left side of the deque object
      # and append them to the chunk list
      chunk = []
      for _ in range(chunk_size):
        if deque_obj:
          chunk.append(deque_obj.popleft())
         
      # Yield the chunk
      yield chunk
input_list = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
chunk_size = 3
chunks = list(split_list(input_list, chunk_size))
print(chunks)

Code Explanation

Importing Required Modules

In this step, we import the deque class from the collections module. The deque class is a double-ended queue that allows efficient popping and appending of elements from both ends.

Defining the split_list Function

In this step, we define the split_list function that takes two parameters: input_list (the list to be split) and chunk_size (the size of each chunk). This function uses a generator to yield chunks of the input list.

Creating a deque Object

Inside the split_list function, we create a deque object deque_obj using the input list. The deque object allows us to efficiently remove elements from the left side.

Splitting the List into Chunks

In this step, we iterate over the deque_obj while it’s not empty. We initialize an empty list chunk to store the elements of each chunk.

Popping Elements and Appending to Chunk

Within the iteration, we use a for loop to pop chunk_size elements from the left side of the deque_obj. These elements are then appended to the chunk list.

Yielding the Chunk

Once we have a complete chunk, we use the yield keyword to yield the chunk. This allows the function to return the chunk as a generator object.

Accepting Input and Calling the Function

In this step, we define the input_list and chunk_size variables with the desired values. We then call the split_list function with the input_list and chunk_size as arguments, and assign the result to the chunks variable.

Printing the Result

Finally, we print the chunks list, which contains the split chunks of the input list.

Output:

[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]]

The input list [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] is split into chunks of size 3 using the split_list function. Each chunk is represented as a sublist within the main list.

The output shows the resulting list of chunks, where each sublist contains three elements. In this case, the input list is divided into five chunks: [1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], and [13, 14, 15].

In Python, there are multiple ways to achieve the same result of splitting a list into evenly sized chunks. Here are a few alternative methods:

Using Yield:

This approach uses a generator function with the yield keyword to produce chunks one at a time. It is memory-efficient because it generates chunks on-the-fly without storing the entire result in memory. This method is useful when dealing with large lists or when memory usage is a concern.

Using for Loop:

With a for loop and slicing, we can iterate over the list and extract chunks of the desired size. This method is simple and intuitive to understand. It works well for small to medium-sized lists.

Using List Comprehension:

List comprehension offers a concise way to split a list into chunks using slicing notation. It creates a new list by iterating over the original list and extracting chunks of the desired size. This method is compact and readable, suitable for situations where code brevity is preferred.

Using NumPy:

NumPy, a powerful library for numerical computations, provides the array_split function to split arrays into evenly sized chunks. It is efficient for handling large datasets and performing complex numerical operations. This method is advantageous when dealing with numerical data and performing subsequent calculations.

Using itertools:

The islice function from the itertools module can be used to split a list into chunks. It offers various iteration tools for efficient and memory-friendly operations. This method is beneficial when working with iterators or when more advanced iteration functionalities are required.

In this blog, we used ‘collections.deque‘ as it allows efficient popping and appending of elements from both ends of the list. This is particularly useful when dealing with large lists or when memory usage is a concern. Also, Generators are memory-efficient as they produce the chunks on-the-fly without storing them in memory.

This is beneficial when dealing with very large lists or when memory resources are limited. It allows for more flexibility in handling the chunks, especially in scenarios where you may not need to access all the chunks simultaneously.

Conclusion

In this tutorial, the code demonstrates how to split a list into evenly sized chunks using the deque class from the collections module in Python. By utilizing the deque object, we can efficiently remove elements from the left side while appending them to individual chunks. The code showcases the power of generators, as it yields each chunk one at a time, allowing for memory-efficient processing of large lists.

Splitting a list into chunks can be beneficial in various scenarios, such as when working with large datasets or when parallel processing is required. The provided code offers a flexible and reusable function that can be applied to different lists and chunk sizes.

Overall, the code provides an effective solution for splitting lists into evenly sized chunks, offering a valuable tool in your Python programming toolkit.

Frequently Asked Questions