In this study, we evaluate the performance of sparse matrix-vector multiplication (SpMV) using the compressed row storage (CRS) format on CPUs and MIC. We focus on the relationship between OpenMP scheduling and performance. The performance of SpMV is measured using various OpenMP scheduling settings and the results are analyzed, which show that OpenMP scheduling has a considerable effect on the performance of SpMV. We confirm that some scheduling settings resulted in performance improvements compared with default scheduling for particular matrices. The results of the evaluation show that the performance of SpMV is improved by up to 1.57 times compared with SPARC64 IXfx, 2.47 times compared with Xeon Ivy Bridge-EP, and 2.26 times compared with Knights Corner. Next, we modify the SpMV function of OpenATLib, an auto-tuned numerical library, to consider the scheduling of optimization as an additional SpMV implementation. We measure the performance of the GMRES solver and obtain performance improvements of up to 11.4%. These results will help to improve the performance of various numerical calculation applications.