{"componentChunkName":"component---src-templates-blog-post-js","path":"/matrix-multiplication-strassen","result":{"data":{"markdownRemark":{"html":"

The standard matrix multiplication for an n x n matrix results in a time complexity with a factor of 3. This means\nif you double the size of the matrix, you'll require eight times more time. Strassen's algorithm is one technique\nthat optimizes matrix multiplication, reducing time complexity to a factor of 2.8.

\n

The algorithm is named after the German mathematician Volker Strassen,\nwho introduced it.

\n

Conditions

\n\n

How it Works

\n
    \n
  1. If the matrices do not meet the above conditions, pad with rows and columns of zeros to get nice square matrices.
    2. \n
  2. Check if the matrix size is smaller than the threshold. If so use the standard multiplication.
    3. \n
  3. If not, proceed to Strassen's algorithm.
    4. \n
  4. Partition matrices into four equal size blocks.
    5. \n
  5. Use these partitions to calculate the seven matrices.
    6. \n
  6. Using the seven matrices, calculate the final matrix.
    7. \n
  7. Remove additional rows and columns (which are a result of padded zeros) to get the final matrix.
    8. \n
\n

Pros

\n

The final matrix is calculated in seven multiplication steps, whereas the standard method uses eight\nmultiplications. This results in reduced time complexity, hence optimizing multiplication.

\n

Cons

\n

The threshold at which the algorithm falls back to the standard method has a significant effect in optimization,\nparticularly when matrices of varying sizes need to be calculated. Threshold measures that work for large matrices\nmay not work for smaller matrices.

\n

Since most matrices usually require to be padded with zeros prior to applying the algorithm, the matrix size may\nbecome very large, hence increasing the time complexity.

\n

Implementation

\n

Here is a C++ implementation of Strassen's algorithm, using it to multiply matrices of size 1000.\nThe results can be compared with those from standard multiplication.

\n
//optimized matrix multiplication using Strassen algorithm\n    \n#include <iostream>\n#include <ctime>\n#include <cstdlib>\n#include <vector>\n#include <cmath>\n\nusing namespace std;\n\n//size at which the sequential multiplication is used instead of recursive Strassen\nint thresholdSize = 128;  \n\nvoid initMat(vector< vector<double> > &a, vector< vector<double> > &b, int n)\n{\n    // initialize matrices and fill them with random values\n    for (int i = 0; i < n; ++i) {\n        for (int j = 0; j < n; ++j) {\n            a[i][j] = (double)rand()/RAND_MAX*10;\n            b[i][j] = (double)rand()/RAND_MAX*10;\n        }\n    }\n}\n\nvoid multiplyMatStandard(vector< vector<double> > &a, vector< vector<double> > &b, vector< vector<double> > &c, int n)\n{\n    // standard matrix multiplication: C <- C + A x B\n    for (int i = 0; i < n; ++i) {\n        for (int j = 0; j < n; ++j) {\n            double temp  = 0;\n            for (int k = 0; k < n; ++k) {\n                temp += a[i][k] * b[k][j];\n            }\n            c[i][j]=temp;\n        }\n    }\n}\n\nint getNextPowerOfTwo(int n)\n{\n    return pow(2, int(ceil(log2(n))));\n}\n\nvoid fillZeros(vector< vector<double> > &newA, vector< vector<double> > &newB,\n       vector< vector<double> > &a, vector< vector<double> > &b, int n)\n{\n    //pad matrix with zeros\n    for (int i=0; i<n; i++){\n        for (int j=0; j<n; j++){\n            newA[i][j] = a[i][j];\n            newB[i][j] = b[i][j];\n        }\n    }\n}\n\nvoid add(vector< vector<double> > &a, vector< vector<double> > &b,\n vector< vector<double> > &resultMatrix, int n)\n{\n    for(int i=0; i<n; i++){\n        for(int j=0; j<n; j++){\n            resultMatrix[i][j] = a[i][j] + b[i][j];\n        }\n    }\n}\n\nvoid subtract(vector< vector<double> > &a, vector< vector<double> > &b,\n      vector< vector<double> > &resultMatrix, int n)\n{\n    for(int i=0; i<n; i++){\n        for(int j=0; j<n; j++){\n            resultMatrix[i][j] = a[i][j] - b[i][j];\n        }\n    }\n}\n\nvoid multiplyStrassen(vector< vector<double> > &a,\n    vector< vector<double> > &b, vector< vector<double> > &c, int n)\n{\t\n    if(n<=thresholdSize) {\n        multiplyMatStandard(a, b, c, n);\n    }\n    else {\n        //expand and fill with zeros if matrix size is not a power of two\n        int newSize = getNextPowerOfTwo(n);\n        vector< vector<double> > \n            newA(newSize, vector<double>(newSize)), \n            newB(newSize, vector<double>(newSize)), \n            newC(newSize, vector<double>(newSize));\n        if(n==newSize) {   \n            //matrix size is already a power of two\n            newA = a;\n            newB = b;\n        }\n        else {\n            fillZeros(newA, newB, a, b, n);\n        }\n        \n        //initialize submatrices\n        int blockSize = newSize/2;  //size for a partition matrix\n        vector<double> block (blockSize);\n        vector< vector<double> > \n            /*partitions of newA*/\n            a11(blockSize, block), a12(blockSize, block), \n            a21(blockSize, block), a22(blockSize, block), \n            /*partitions of newB*/\n            b11(blockSize, block), b12(blockSize, block), \n            b21(blockSize, block), b22(blockSize, block), \n            /*partitions of newC*/\n            c11(blockSize, block), c12(blockSize, block), \n            c21(blockSize, block), c22(blockSize, block), \n            /*matrices storing intermediate results*/\n            aBlock(blockSize, block), bBlock(blockSize, block),\n            /*set of submatrices derived from partitions*/\n            m1(blockSize, block), m2(blockSize, block), \n            m3(blockSize, block), m4(blockSize, block),  \n            m5(blockSize, block), m6(blockSize, block), \n            m7(blockSize, block);  \n        \n        //partition matrices\n        for (int i=0; i<blockSize; i++){\n            for (int j=0; j<blockSize; j++){\n                a11[i][j] = newA[i][j];\n                a12[i][j] = newA[i][j+blockSize];\n                a21[i][j] = newA[i+blockSize][j];\n                a22[i][j] = newA[i+blockSize][j+blockSize];\n                b11[i][j] = newB[i][j];\n                b12[i][j] = newB[i][j+blockSize];\n                b21[i][j] = newB[i+blockSize][j];\n                b22[i][j] = newB[i+blockSize][j+blockSize];\n            }\n        }\n    \n        //compute submatrices\n        //m1 = (a11+a22)(b11+b22)\n        add(a11, a22, aBlock, blockSize);\n        add(b11, b22, bBlock, blockSize);\n        multiplyStrassen(aBlock, bBlock, m1, blockSize);\n    \n        //m2 = (a21+a22)b11\n        add(a21, a22, aBlock, blockSize);\n        multiplyStrassen(aBlock, b11, m2, blockSize);\n        \n        //m3 = a11(b12-b22)\n        subtract(b12, b22, bBlock, blockSize);\n        multiplyStrassen(a11, bBlock, m3, blockSize);\n        \n        //m4 = a22(b21-b11)\n        subtract(b21, b11, bBlock, blockSize);\n        multiplyStrassen(a22, bBlock, m4, blockSize);\n        \n        //m5 = (a11+a12)b22\n        add(a11, a12, aBlock, blockSize);\n        multiplyStrassen(aBlock, b22, m5, blockSize);\n        \n        //m6 = (a21-a11)(b11+b12)\n        subtract(a21, a11, aBlock, blockSize);\n        add(b11, b12, bBlock, blockSize);\n        multiplyStrassen(aBlock, bBlock, m6, blockSize);\n    \n        //m7 = (a12-a22)(b12+b22)\n        subtract(a12, a22, aBlock, blockSize);\n        add(b12, b22, bBlock, blockSize);\n        multiplyStrassen(aBlock, bBlock, m7, blockSize);\n        \n        //calculate result submatrices\n        //c11 = m1+m4-m5+m7\n        add(m1, m4, aBlock, blockSize);\n        subtract(aBlock, m5, bBlock, blockSize);\n        add(bBlock, m7, c11, blockSize);\n        \n        //c12 = m3+m5\n        add(m3, m5, c12, blockSize);\n        \n        //c21 = m2+m4\n        add(m2, m4, c12, blockSize);\n    \n        //c22 = m1-m2+m3+m6\n        subtract(m1, m2, aBlock, blockSize);\n        add(aBlock, m3, bBlock, blockSize);\n        add(bBlock, m6, c22, blockSize);\n        \n        //calculate final result matrix\n        for(int i=0; i<blockSize; i++){\n            for(int j=0; j<blockSize; j++){\n                newC[i][j] = c11[i][j];\n                newC[i][blockSize+j] = c12[i][j];\n                newC[blockSize+i][j] = c21[i][j];\n                newC[blockSize+i][blockSize+j] = c22[i][j];\n            }\n        }\n    \n        //remove additional values from expanded matrix\n        for(int i=0; i<n; i++){\n            for(int j=0; j<n; j++){\n                c[i][j] = newC[i][j];\n            }\n        }\n    }\n}\n\ndouble calculateMean(vector<double> data, int size) \n{\n   double sum = 0.0, mean = 0.0;\n   for (int i = 0; i < size; ++i) {\n       sum += data[i];\n   }\n\n   mean = sum / size;\n   return mean;\n}\n\nint main(int argc, char *argv[])\n{\n    srand(time(0));   //seed for random number generation\n    \n    // number of sample size considered to evaluate average execution time\n    int sampleSize = 10;       \n    int matSize = 1000;       \n    double startTime;\n    double elapsedTime;\n    double standardMean;\n    double strassenMean;\n    \n    //set threshold value if given by user\n    if(argc>1) {\n        thresholdSize = atoi(argv[1]);  \n    }\n\n    //vectors storing execution time values\n    vector<double> standardTime(sampleSize);      \n    vector<double> strassenTime(sampleSize);\n    \n    for (int k = 0; k < sampleSize; k++) {\n        //initialize vectors for matrices a, b, c: a*b = c\n        vector< vector<double> > \n            a(matSize,vector<double>(matSize)),\n            b(matSize,vector<double>(matSize)),\n            c(matSize,vector<double>(matSize));\t\n        \n        initMat(a,b,matSize);\n        \n        //standard execution\t\t\n        startTime = time(0);\n        multiplyMatStandard(a,b,c,matSize);\n        elapsedTime = time(0) - startTime;\n        standardTime[k] = elapsedTime;\n        \n        //multiplication using Strassen's \n        startTime = 0;\t\t\t\n        startTime = time(0);\n        multiplyStrassen(a,b,c,matSize);\n        elapsedTime = time(0) - startTime;\n        strassenTime[k] = elapsedTime;\n    }\n    cout << \"Standard multiplication\"<< endl;\n    standardMean = calculateMean(standardTime, sampleSize);\n    cout << \"Average time taken to execute matrices of size - \" \n        << matSize << \" : \" << standardMean << endl;\n    cout << endl;\n    \n    cout << \"Multiplication using Strassen's\"<< endl;\n    cout << \"Threshold used: \"<< thresholdSize <<endl;\n    strassenMean = calculateMean(strassenTime, sampleSize);\n    cout << \"Average time taken to execute matrices of size - \" \n        << matSize << \" : \" << strassenMean << endl;\n    cout << endl;\n    \n    cout << \"Speed up gained by using Strassen's-\" << matSize \n        << \" : \" << standardMean/strassenMean << endl;\n    cout << endl;\n    \n    return 0;\n}
\n

Results

\n

Above execution gave a speed increase of 2.

\n

\"str1\"

","frontmatter":{"date":"July 06, 2017","path":"/matrix-multiplication-strassen","title":"Optimising Matrix Multiplication using Strassen's Algorithm","tags":["How to","C++"]}}},"pageContext":{}},"staticQueryHashes":["3649515864"]}