Chapter 1: Introduction to Sparse Matrix-Vector Multiplication

In the realm of small-scale parallel programming, our focus lies on the efficiency of OpenMP and CUDA in executing sparse matrix-vector multiplication (SpMV). This operation is essential in scientific computing and presents numerous challenges that need to be tackled to enhance performance.

"In scientific computing, sparse matrix-vector multiplication (SpMV) is a fundamental yet intricate operation requiring optimal strategies for efficient execution."

Section 1.1: Overview of Sparse Matrices

Sparse matrices are prevalent in various scientific and engineering applications, including numerical simulations, machine learning, and data analysis. These matrices primarily consist of zero entries, and thus, the challenge lies in effectively processing them without incurring significant computational costs.

As defined by James Wilkinson, a sparse matrix is one where:

"Any matrix with enough zeros that it pays to take advantage of them."

In numerical terms, this is represented as the number of non-zero (NZ) elements being significantly smaller than the total number of elements (M × N), leading to a lower computational burden.

Yet, this sparsity complicates the parallelization of computations due to irregular workloads across matrix elements.

Section 1.2: Research Focus

This study investigates the performance of OpenMP and CUDA for SpMV on a hybrid CPU-GPU architecture. By implementing both programming models on various sparse matrices of differing sizes and densities, we aim to uncover the correlations between programming choices and matrix characteristics.

Chapter 2: Programming Frameworks in SpMV

Section 2.1: Comparing OpenMP and CUDA

OpenMP is a shared-memory parallel programming model that effectively utilizes multicore processors. In contrast, CUDA, developed by NVIDIA, leverages the massive parallel processing capabilities of GPUs, enabling efficient execution of SpMV on larger matrices.

Previous studies have indicated that while CUDA excels in handling larger datasets, OpenMP is more suited for smaller-scale problems or systems with fewer cores.

Section 2.2: Objectives of the Study

The goals of this report include:

1. Development of a parallel SpMV kernel for computing y ← Ax, where A represents a sparse matrix.
2. Parallelization of the kernel for both OpenMP and CUDA frameworks.
3. Creation of auxiliary functions for matrix data preprocessing and representation.
4. Performance testing on a selection of matrices from the Suite Sparse Matrix Collection.

Chapter 3: Methodological Framework

Section 3.1: Sparse Matrix Formats

Various formats for sparse matrix storage were employed in this project, including Coordinate List (COO), Compressed Sparse Row (CSR), and ELLPACK. Each format presents unique advantages and challenges for efficient computation.

Section 3.2: Optimization Techniques

Numerous optimization techniques, such as loop unrolling and matrix transposition, were explored to enhance the efficiency of the SpMV computations.

Chapter 4: Experimental Evaluation

The experiments employed matrices from the SuiteSparse Matrix Collection to evaluate the performance of various programming models.

Section 4.1: Hardware Setup

The study utilized the Tesla K40m GPU, which features 2,880 CUDA cores and is designed for high-performance computing tasks.

Section 4.2: Performance Results

The results indicated that the CSR format outperformed ELLPACK in OpenMP implementations, while CUDA showed improved performance with ELLPACK in larger matrices.

Section 4.3: Comparative Analysis

A comparative analysis between OpenMP and CUDA revealed that OpenMP is more effective for smaller matrices, whereas CUDA excels with larger, denser matrices.

Chapter 5: Conclusion

In summary, the performance analysis highlighted the strengths of the CSR format in OpenMP, while CUDA demonstrated superior efficiency with the ELLPACK format for large datasets. The findings underscore the importance of selecting the appropriate programming model based on matrix characteristics and computational requirements.

For further details on the implementation, refer to the code repository.

References

[1] T. Davis. Wilkinson's sparse matrix definition (2007)

[2] Salvatore Filippone et al. (2017), Sparse Matrix-Vector Multiplication on GPGPUs

[3] Nathan Bell, Michael Garland (2008), Efficient Sparse Matrix-Vector Multiplication on CUDA

[4] John Mellor-Crummey, John Garvin (2003), Optimizing Sparse Matrix-Vector Product Computations Using Unroll and Jam

[5] Salvatore Filippone (2023), Small Scale Parallel Programming, Cranfield University