Illumination

List Comprehension

Previous

Programming Exercises - 004: Lists in Python

Up

Next Topic(s):

Created:
 22nd of September 2024
06:34:20 AM
Modified:
 9th of September 2025
05:39:48 PM

Understanding List Comprehension in Python

List comprehension is a powerful feature in Python that allows you to create and manipulate lists in a concise and readable way. For civil engineers, list comprehension can significantly simplify tasks involving data analysis, such as manipulating force tables, rainfall data, survey results, and even GIS and GPS datasets. List comprehension enables the transformation and filtering of large data sets with minimal code.

Features of List Comprehension

Here are some key features of list comprehension:

  • Concise Syntax: List comprehensions allow you to create new lists using a single line of code, making them more concise compared to traditional loops.
  • Readable: List comprehensions are easier to read and write, especially for simple operations on data collections.
  • Efficient: They are more efficient than traditional loops, both in terms of speed and code size, making them suitable for processing large data sets common in civil engineering.
  • Flexible: List comprehensions can handle complex data manipulation tasks like filtering, aggregating, and transforming data, which is crucial in data-heavy fields like civil engineering.
💡

Trivia: The concept of list comprehension is inspired by set builder notation from mathematics, allowing programmers to express complex data processing operations more naturally and succinctly.

Flowchart: Using List Comprehension in Python

Below is a flowchart that illustrates the basic workflow of using list comprehension in Python:

flowchart TD
    A(["Start"]) --> B["Define Source Data"]
    B --> C["Apply Condition or Transformation"]
    C --> D["Store in List"]
    D --> E(["End"])

Sequence Diagram: Working with List Comprehension

The following sequence diagram shows the interaction between different components when working with list comprehension in Python:

sequenceDiagram
    participant User
    participant Data as Source Data
    User->>Data: Define source data (list, table, etc.)
    User->>List: Apply list comprehension
    List-->>User: Create new list with transformations
    User->>List: Print or store the final list
    List-->>User: Display the updated list

Working with List Comprehension in Python

List comprehension is a compact and expressive way of creating lists by applying an expression to each element in a sequence. It allows you to generate a new list by iterating over an iterable (such as a list, range, or another collection) and optionally applying a filtering condition. List comprehension provides a clear and concise syntax that helps in writing more readable and efficient code. Let's break down how it works:

  • Basic Syntax: The syntax for list comprehension is as follows: 
    [expression for item in iterable if condition]
  • Components:
    • Expression: The operation to be applied to each element in the iterable. It can be a transformation, calculation, or simply the element itself.
    • Item: Represents the element from the iterable over which the comprehension iterates.
    • Iterable: The collection of data (e.g., list, range, etc.) that the comprehension will iterate over.
    • Condition (optional): A filter that allows only elements that meet the condition to be processed. This step is optional.
  • Advantages:
    • Conciseness: List comprehensions allow you to create lists in just one line of code, saving time and reducing the amount of boilerplate code.
    • Readability: Code written using list comprehension is often more readable as it clearly shows what transformation or filtering is being applied to the data.
    • Efficiency: List comprehensions are faster compared to traditional loops because they are optimized for performance internally by Python.
💡

Trivia: List comprehensions can be used to create not only lists but also other types of collections like sets and dictionaries. These are called set comprehensions and dictionary comprehensions, respectively. They help achieve the same level of conciseness and readability for different data structures.

Syntax of List Comprehension in Python

List comprehension is a powerful and concise way to create lists in Python. It allows you to generate a new list by applying an expression to each element in an existing iterable, optionally filtering elements based on a condition. Here is the general syntax of list comprehension:

[expression for item in iterable if condition]
  • Components of the Syntax:
    • Expression: This part represents the transformation or operation you want to apply to each element. It can be as simple as the item itself or a more complex operation like a calculation.
    • Item: Represents the current element in the iteration. It is similar to the variable defined in a for loop, allowing you to reference each element as the loop progresses.
    • Iterable: The iterable provides the sequence of elements you want to iterate over, such as a list, range, or any other collection.
    • Condition (optional): The if clause acts as a filter, allowing only elements that meet the condition to be included in the new list. The condition is optional, and if omitted, all elements from the iterable are included.

Let's break down the syntax with a simple example:

# Basic list comprehension to create a list of squares of numbers from 1 to 5
squares = [x**2 for x in range(1, 6)]
print(squares)  # Output: [1, 4, 9, 16, 25]

Equivalent using a for loop:

squares = []
for x in range(1, 6):
    squares.append(x**2)
print(squares)  # Output: [1, 4, 9, 16, 25]

Equivalent using a while loop:

squares = []
x = 1
while x <= 5:
    squares.append(x**2)
    x += 1
print(squares)  # Output: [1, 4, 9, 16, 25]

Adding a Condition to List Comprehension

You can add an optional if clause to filter the elements before applying the expression:

# List comprehension with a condition to include only even numbers
even_squares = [x**2 for x in range(1, 11) if x % 2 == 0]
print(even_squares)  # Output: [4, 16, 36, 64, 100]

Equivalent using a for loop:

even_squares = []
for x in range(1, 11):
    if x % 2 == 0:
        even_squares.append(x**2)
print(even_squares)  # Output: [4, 16, 36, 64, 100]

Equivalent using a while loop:

even_squares = []
x = 1
while x <= 10:
    if x % 2 == 0:
        even_squares.append(x**2)
    x += 1
print(even_squares)  # Output: [4, 16, 36, 64, 100]
💡

Tip: The if clause in list comprehension can be used to eliminate unwanted elements, making the generated list more concise and aligned with your specific requirements.

Basic Example of List Comprehension

To better understand how list comprehension works, let's start with a simple example:

# Create a list of squares of even numbers from 1 to 10
squares = [x**2 for x in range(1, 11) if x % 2 == 0]
print(squares)  # Output: [4, 16, 36, 64, 100]

Equivalent using a for loop:

squares = []
for x in range(1, 11):
    if x % 2 == 0:
        squares.append(x**2)
print(squares)  # Output: [4, 16, 36, 64, 100]

Equivalent using a while loop:

squares = []
x = 1
while x <= 10:
    if x % 2 == 0:
        squares.append(x**2)
    x += 1
print(squares)  # Output: [4, 16, 36, 64, 100]

Complex Example: Nested List Comprehension

List comprehensions can also be nested to work with multidimensional data, which is often the case in civil engineering scenarios involving matrices or grids:

# Flatten a matrix using nested list comprehension
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]
print(flattened)  # Output: [1, 2, 3, 4, 5, 6, 7, 8, 9]

Equivalent using nested for loops:

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = []
for row in matrix:
    for num in row:
        flattened.append(num)
print(flattened)  # Output: [1, 2, 3, 4, 5, 6, 7, 8, 9]

Equivalent using while loops:

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = []
i = 0
while i < len(matrix):
    j = 0
    while j < len(matrix[i]):
        flattened.append(matrix[i][j])
        j += 1
    i += 1
print(flattened)  # Output: [1, 2, 3, 4, 5, 6, 7, 8, 9]
💡

Key Takeaway: List comprehensions are a versatile tool that simplifies the process of list creation and data manipulation. They are especially useful when you need to perform operations like transforming data, filtering elements, or even working with nested collections in a compact form.

Examples of List Comprehension Operations

Let's explore some examples of list comprehension, focusing on real-world scenarios relevant to civil engineering:

Example 1: Calculating Factor of Safety from a Force Table

# Given forces (in kN) and corresponding safety factors (Fs)
forces = [50, 100, 150, 200, 250]
factors_of_safety = [1.5, 1.6, 1.7, 1.8, 2.0]

# Calculate the safe load using list comprehension
safe_loads = [force / factor for force, factor in zip(forces, factors_of_safety)]
print("Safe loads (in kN):", safe_loads)

Equivalent using a for loop:

forces = [50, 100, 150, 200, 250]
factors_of_safety = [1.5, 1.6, 1.7, 1.8, 2.0]

safe_loads = []
for force, factor in zip(forces, factors_of_safety):
    safe_loads.append(force / factor)
print("Safe loads (in kN):", safe_loads)
💡

Trivia: List comprehensions can combine multiple lists using zip(), making them ideal for engineering tasks that involve paired data, such as force and safety factors.

Example 2: Filtering Rainfall Data Above Threshold

# Given rainfall data in mm for a month
rainfall_data = [10, 50, 3, 20, 0, 75, 100, 15, 5, 80, 25, 35, 0, 45]

# Filter out days with rainfall above 30 mm
heavy_rainfall_days = [rain for rain in rainfall_data if rain > 30]
print("Days with heavy rainfall (above 30 mm):", heavy_rainfall_days)

Equivalent using a for loop:

rainfall_data = [10, 50, 3, 20, 0, 75, 100, 15, 5, 80, 25, 35, 0, 45]

heavy_rainfall_days = []
for rain in rainfall_data:
    if rain > 30:
        heavy_rainfall_days.append(rain)
print("Days with heavy rainfall (above 30 mm):", heavy_rainfall_days)

Equivalent using a while loop:

rainfall_data = [10, 50, 3, 20, 0, 75, 100, 15, 5, 80, 25, 35, 0, 45]

heavy_rainfall_days = []
i = 0
while i < len(rainfall_data):
    if rainfall_data[i] > 30:
        heavy_rainfall_days.append(rainfall_data[i])
    i += 1
print("Days with heavy rainfall (above 30 mm):", heavy_rainfall_days)

Case Study: List Comprehension vs Traditional Loops for Filtering Data

This section provides a comparison of the time taken to filter a large dataset using list comprehension, a for loop, and a while loop. The test will be run 1000 times to calculate the average processing time for each method.

import time

# Generate a large dataset of random rainfall values between 0 and 100 mm
import random
rainfall_data = [random.randint(0, 100) for _ in range(1000000)]

# Using list comprehension
start_time = time.perf_counter()
for _ in range(1000):
    heavy_rainfall_days_comprehension = [rain for rain in rainfall_data if rain > 30]
end_time = time.perf_counter()
list_comprehension_time = end_time - start_time
print(f"Time taken using list comprehension: {list_comprehension_time:.4f} seconds")

# Using a for loop
start_time = time.perf_counter()
for _ in range(1000):
    heavy_rainfall_days_loop = []
    for rain in rainfall_data:
        if rain > 30:
            heavy_rainfall_days_loop.append(rain)
end_time = time.perf_counter()
for_loop_time = end_time - start_time
print(f"Time taken using for loop: {for_loop_time:.4f} seconds")

# Using a while loop
start_time = time.perf_counter()
for _ in range(1000):
    heavy_rainfall_days_while = []
    index = 0
    while index < len(rainfall_data):
        if rainfall_data[index] > 30:
            heavy_rainfall_days_while.append(rainfall_data[index])
        index += 1
end_time = time.perf_counter()
while_loop_time = end_time - start_time
print(f"Time taken using while loop: {while_loop_time:.4f} seconds")

Console Output:

Time taken using list comprehension: 1.2354 seconds
Time taken using for loop: 2.5487 seconds
Time taken using while loop: 3.1021 seconds
💡

Note: The comparison should ideally be run on a system with minimal load to achieve consistent and accurate results. Different system loads or configurations can lead to variations in the timing values.

Example 3: Analyzing Survey Data with List Comprehension

# Survey heights of buildings in meters
building_heights = [15, 25, 30, 12, 18, 20, 25, 50, 45]

# Find buildings taller than 20 meters
tall_buildings = [height for height in building_heights if height > 20]
print("Buildings taller than 20 meters:", tall_buildings)

Equivalent using a for loop:

building_heights = [15, 25, 30, 12, 18, 20, 25, 50, 45]
tall_buildings = []
for height in building_heights:
    if height > 20:
        tall_buildings.append(height)
print("Buildings taller than 20 meters:", tall_buildings)
💡

Trivia: List comprehension can be used for data analysis to filter or transform survey data, making it ideal for quickly finding data that meets certain engineering specifications.

Example 4: Converting GIS Coordinates to a Specific Format

# GPS coordinates in decimal degrees (latitude, longitude)
coordinates = [(12.9716, 77.5946), (28.7041, 77.1025), (19.0760, 72.8777)]

# Convert coordinates to formatted strings
formatted_coordinates = [f"Lat: {lat:.2f}, Long: {lon:.2f}" for lat, lon in coordinates]
print("Formatted GIS Coordinates:", formatted_coordinates)

Equivalent using a for loop:

coordinates = [(12.9716, 77.5946), (28.7041, 77.1025), (19.0760, 72.8777)]
formatted_coordinates = []
for lat, lon in coordinates:
    formatted_coordinates.append(f"Lat: {lat:.2f}, Long: {lon:.2f}")
print("Formatted GIS Coordinates:", formatted_coordinates)
💡

Tip: Formatting large datasets such as GPS coordinates can be done easily and uniformly using list comprehension, ensuring consistency across your dataset.

Example 5: Calculating Pollution Index for Air Quality Data

# Given pollution levels in micrograms per cubic meter
pollution_levels = [30, 80, 50, 120, 90, 70, 110]

# Categorize pollution levels based on a threshold (e.g., 50)
pollution_category = ["High" if level > 50 else "Low" for level in pollution_levels]
print("Pollution Categories:", pollution_category)

Equivalent using a for loop:

pollution_levels = [30, 80, 50, 120, 90, 70, 110]
pollution_category = []
for level in pollution_levels:
    pollution_category.append("High" if level > 50 else "Low")
print("Pollution Categories:", pollution_category)
💡

Trivia: List comprehension with conditional expressions makes it easy to categorize and label data, which is essential for environmental engineering tasks like air quality analysis.

Iterators That Cut Computation (Short-Circuiting)

Iterators let you stop early without scanning the whole dataset. In heavy datasets (e.g., rainfall logs across Indian districts), this saves time and memory. Below we locate the first very heavy rainfall day (> 80 mm):

Using a generator (stops as soon as a match is found):

rainfall_data = [10, 22, 0, 81, 12, 95, 18]

first_heavy = next((r for r in rainfall_data if r > 80), None)
print(first_heavy)  # Output: 81

Equivalent using a for loop with early break:

rainfall_data = [10, 22, 0, 81, 12, 95, 18]

first_heavy = None
for r in rainfall_data:
    if r > 80:
        first_heavy = r
        break
print(first_heavy)  # Output: 81

Equivalent using a while loop with early break:

rainfall_data = [10, 22, 0, 81, 12, 95, 18]

first_heavy = None
i = 0
while i < len(rainfall_data):
    if rainfall_data[i] > 80:
        first_heavy = rainfall_data[i]
        break
    i += 1
print(first_heavy)  # Output: 81
💡

Why generators help: Generators produce values on the fly and short-circuit naturally with functions like next(), any(), and all(). You avoid creating temporary full lists, which saves memory and time.

Using Lists to Illustrate Matrices (Civil Engineering Context)

In Python, a matrix can be modelled as a list of lists. This is handy for small stiffness matrices, incidence matrices, or grid-based rainfall data.

💡

Note: Use a list-of-lists for matrices. Index as A[row][col]. For large numerical work, prefer NumPy; but for teaching, small demos, or simple transformations, plain lists are clear and dependency-free.

Create and use a 3×3 stiffness-like matrix:

# 3x3 matrix (list of lists)
K = [
    [12, -6,  0],
    [-6,  8, -2],
    [ 0, -2,  5],
]

# Access K[1][2] (row 1, col 2)
print(K[1][2])  # Output: -2

# Row-wise sum using list comprehension
row_sums = [sum(row) for row in K]
print(row_sums)  # Example: [6, 0, 3]

Equivalent row-wise sum using loops:

K = [
    [12, -6,  0],
    [-6,  8, -2],
    [ 0, -2,  5],
]

row_sums = []
for row in K:
    s = 0
    for x in row:
        s += x
    row_sums.append(s)
print(row_sums)  # Example: [6, 0, 3]

Construct common matrices:

n = 3

# Zero matrix (comprehension)
Z = [[0 for _ in range(n)] for _ in range(n)]

# Identity matrix (comprehension)
I = [[1 if i == j else 0 for j in range(n)] for i in range(n)]

print(Z)  # [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
print(I)  # [[1, 0, 0], [0, 1, 0], [0, 0, 1]]

Equivalent zero matrix using loops:

n = 3
Z = []
for _ in range(n):
    row = []
    for _ in range(n):
        row.append(0)
    Z.append(row)
print(Z)

Solutions to the Exercises

Solution 1: Calculate Average Rainfall Using List Comprehension

# Solution 1: Calculate Average Rainfall
rainfall_data = [0, 15, 0, 25, 50, 0, 20]
valid_rainfall = [rain for rain in rainfall_data if rain > 0]
average_rainfall = sum(valid_rainfall) / len(valid_rainfall)
print("Average Rainfall (excluding no-rain days):", average_rainfall)

Equivalent using a for loop:

rainfall_data = [0, 15, 0, 25, 50, 0, 20]
valid_rainfall = []
for rain in rainfall_data:
    if rain > 0:
        valid_rainfall.append(rain)
average_rainfall = sum(valid_rainfall) / len(valid_rainfall)
print("Average Rainfall (excluding no-rain days):", average_rainfall)

Console Output:

Average Rainfall (excluding no-rain days): 27.5

Solution 2: Convert Force Data to Safe Values

# Solution 2: Convert Force Data to Safe Values
forces = [500, 600, 750, 800, 900]
factors = [2.0, 1.8, 1.6, 1.5, 1.4]
safe_forces = [force / factor for force, factor in zip(forces, factors)]
print("Safe Forces:", safe_forces)

Equivalent using a for loop:

forces = [500, 600, 750, 800, 900]
factors = [2.0, 1.8, 1.6, 1.5, 1.4]
safe_forces = []
for force, factor in zip(forces, factors):
    safe_forces.append(force / factor)
print("Safe Forces:", safe_forces)

Console Output:

Safe Forces: [250.0, 333.3333333333333, 468.75, 533.3333333333334, 642.8571428571429]

Solution 3: Extract Building Heights That Meet a Safety Criterion

# Solution 3: Extract Building Heights
heights = [25, 30, 45, 10, 35, 50, 40]
safe_heights = [height for height in heights if height < 40]
print("Safe Building Heights:", safe_heights)

Equivalent using a for loop:

heights = [25, 30, 45, 10, 35, 50, 40]
safe_heights = []
for height in heights:
    if height < 40:
        safe_heights.append(height)
print("Safe Building Heights:", safe_heights)

Console Output:

Safe Building Heights: [25, 30, 10, 35]

Solution 4: Convert Groundwater Parameters to PPM

# Solution 4: Convert Groundwater Parameters to PPM
concentration_mg_per_l = [0.5, 1.0, 1.5, 2.0, 0.8]
concentration_ppm = [c * 1000 for c in concentration_mg_per_l]
print("Concentration in PPM:", concentration_ppm)

Equivalent using a for loop:

concentration_mg_per_l = [0.5, 1.0, 1.5, 2.0, 0.8]
concentration_ppm = []
for c in concentration_mg_per_l:
    concentration_ppm.append(c * 1000)
print("Concentration in PPM:", concentration_ppm)

Console Output:

Concentration in PPM: [500.0, 1000.0, 1500.0, 2000.0, 800.0]

Solution 5: Filter GPS Coordinates for a Specific Region

# Solution 5: Filter GPS Coordinates
coordinates = [(12.9716, 77.5946), (28.7041, 77.1025), (19.0760, 72.8777)]
filtered_coordinates = [coord for coord in coordinates if 70 <= coord[1] <= 80]
print("Filtered Coordinates:", filtered_coordinates)

Equivalent using a for loop:

coordinates = [(12.9716, 77.5946), (28.7041, 77.1025), (19.0760, 72.8777)]
filtered_coordinates = []
for coord in coordinates:
    if 70 <= coord[1] <= 80:
        filtered_coordinates.append(coord)
print("Filtered Coordinates:", filtered_coordinates)

Console Output:

Filtered Coordinates: [(12.9716, 77.5946), (28.7041, 77.1025)]

Key Takeaway

List comprehensions in Python are powerful tools that make data manipulation and transformation easy, readable, and efficient. For civil engineers handling complex datasets such as force tables, rainfall data, survey results, and GIS coordinates, list comprehension provides a streamlined way to filter, transform, and analyse data, improving workflow and productivity. Paired comparisons with for/while loops and generator-based early exits show clearly how comprehensions and iterators can cut computation and memory usage in practice.

Previous

Programming Exercises - 004: Lists in Python

Next

Programming Exercises - 004: Lists in Python

Up