Python is convenient, but it can also be slow. However, it does allow you to access libraries that execute faster code written in languages like C. NumPy is one such library: it provides fast alternatives to math operations in Python and is designed to work efficiently with groups of numbers - like matrices.

NumPy is a large library and we are only going to scratch the surface of it here. If you plan on doing much math with Python, you should definitely spend some time exploring its documentation to learn more.

Importing NumPy

When importing the NumPy library, the convention you'll see used most often – including here – is to name it np, like so:

import numpy as np

Now you can use the library by prefixing the names of functions and types with np., which you'll see in the following examples

Data Types and Shapes

The most common way to work with numbers in NumPy is through ndarray objects. They are similar to Python lists, but can have any number of dimensions. Also, ndarray supports fast math operations, which is just what we want.

Since it can store any number of dimensions, you can use ndarrays to represent any of the data types we covered before: scalars, vectors, matrices, or tensors.

Scalars

Scalars in NumPy are a bit more involved than in Python. Instead of Python’s basic types like int, float, etc., NumPy lets you specify signed and unsigned types, as well as different sizes. So instead of Python’s int, you have access to types like uint8, int8, uint16, int16, and so on.

These types are important because every object you make (vectors, matrices, tensors) eventually stores scalars. And when you create a NumPy array, you can specify the type - but every item in the array must have the same type. In this regard, NumPy arrays are more like C arrays than Python lists.

If you want to create a NumPy array that holds a scalar, you do so by passing the value to NumPy's array function, like so:

s = np.array(5)

s.shape

()

() means it has zero dimensions.

x = s + 3
x

8

Vectors

To create a vector, you'd pass a Python list to the array function, like this:

v = np.array([1,2,3])
v.shape

(3,)
x = v[1]
x

2
x = v[1:]
x

array([2, 3])

Matrices

You create matrices using NumPy's array function, just you did for vectors. However, instead of just passing in a list, you need to supply a list of lists, where each list represents a row. So to create a 3x3 matrix containing the numbers one through nine, you could do this:

m = np.array([[1,2,3], [4,5,6], [7,8,9]])
m

array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])
m.shape

(3, 3)
m[2][2]

9

Tensors

Tensors are just like vectors and matrices, but they can have more dimensions. For example, to create a 3x3x2x1 tensor, you could do the following:

t = np.array([[[[1],[2]],[[3],[4]],[[5],[6]]],[[[7],[8]],\
    [[9],[10]],[[11],[12]]],[[[13],[14]],[[15],[16]],[[17],[17]]]])
t

array([[[[ 1],
         [ 2]],

        [[ 3],
         [ 4]],

        [[ 5],
         [ 6]]],


       [[[ 7],
         [ 8]],

        [[ 9],
         [10]],

        [[11],
         [12]]],


       [[[13],
         [14]],

        [[15],
         [16]],

        [[17],
         [17]]]])
t.shape

(3, 3, 2, 1)

Changing Shapes

Sometimes you'll need to change the shape of your data without actually changing its contents. For example, you may have a vector, which is one-dimensional, but need a matrix, which is two-dimensional. There are two ways you can do that.

Let's say you have the following vector:

v = np.array([1,2,3,4])
v.shape

(4,)
x = v.reshape(1,4)
x.shape

(1, 4)
x = v.reshape(4,1)
x.shape

(4, 1)

One more thing about reshaping NumPy arrays: if you see code from experienced NumPy users, you will often see them use a special slicing syntax instead of calling reshape. Using this syntax, the previous two examples would look like this:

x = v[None, :]
x

array([[1, 2, 3, 4]])
x = v[0:, None]
x

array([[1],
       [2],
       [3],
       [4]])

Those lines create a slice that looks at all of the items of v but asks NumPy to add a new dimension of size 1 for the associated axis. It may look strange to you now, but it's a common technique so it's good to be aware of it.

Element-wise operations

The Python way

Suppose you had a list of numbers, and you wanted to add 5 to every item in the list. Without NumPy, you might do something like this:

values = [1,2,3,4,5]
for i in range(len(values)):
    values[i] += 5
values

[6, 7, 8, 9, 10]

The NumPy way

In NumPy, we could do the following:

values = [1,2,3,4,5]
values = np.array(values) + 5
values

array([ 6,  7,  8,  9, 10])

Creating that array may seem odd, but normally you'll be storing your data in ndarrays anyway. So if you already had an ndarray named values, you could have just done:

values += 5
values

array([11, 12, 13, 14, 15])

We should point out, NumPy actually has functions for things like adding, multiplying, etc. But it also supports using the standard math operators. So the following two lines are equivalent:

x = np.multiply(values, 5)
x = values * 5
x

array([55, 60, 65, 70, 75])
a = np.array([[1,3],[5,7]])
a

array([[1, 3],
       [5, 7]])
b = np.array([[2,4],[6,8]])
b

array([[2, 4],
       [6, 8]])
a+b

array([[ 3,  7],
       [11, 15]])

Important Reminders About Matrix Multiplication

  • The number of columns in the left matrix must equal the number of rows in the right matrix.
  • The answer matrix always has the same number of rows as the left matrix and the same number of columns as the right matrix.
  • Order matters. Multiplying A•B is not the same as multiplying B•A.
  • Data in the left matrix should be arranged as rows., while data in the right matrix should be arranged as columns.

NumPy Matrix Multiplication

You've heard a lot about matrix multiplication in the last few videos – now you'll get to see how to do it with NumPy. However, it's important to know that NumPy supports several types of matrix multiplication.

Element-wise Multiplication You saw some element-wise multiplication already. You accomplish that with the multiply function or the * operator. Just to revisit, it would look like this:

m = np.array([[1,2,3],[4,5,6]])
m

array([[1, 2, 3],
       [4, 5, 6]])
n = m * 0.25
n

array([[0.25, 0.5 , 0.75],
       [1.  , 1.25, 1.5 ]])
m * n

array([[0.25, 1.  , 2.25],
       [4.  , 6.25, 9.  ]])
np.multiply(m, n)

array([[0.25, 1.  , 2.25],
       [4.  , 6.25, 9.  ]])

Matrix Product

To find the matrix product, you use NumPy's matmul function.

If you have compatible shapes, then it's as simple as this:

a = np.array([[1,2,3,4],[5,6,7,8]])
a

array([[1, 2, 3, 4],
       [5, 6, 7, 8]])
a.shape

(2, 4)
b = np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12]])
b

array([[ 1,  2,  3],
       [ 4,  5,  6],
       [ 7,  8,  9],
       [10, 11, 12]])
b.shape

(4, 3)
c = np.matmul(a, b)
c

array([[ 70,  80,  90],
       [158, 184, 210]])
c.shape

(2, 3)

If your matrices have incompatible shapes, you'll get an error, like the following:

np.matmul(b, a)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-67-af3b88aa2232> in <module>
----> 1 np.matmul(b, a)

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2 is different from 3)

NumPy's dot function

You may sometimes see NumPy's dot function in places where you would expect a matmul. It turns out that the results of dot and matmul are the same if the matrices are two dimensional.

So these two results are equivalent:

a = np.array([[1,2],[3,4]])
a

array([[ 7, 10],
       [15, 22]])
np.dot(a,a)

array([[ 7, 10],
       [15, 22]])
a.dot(a)  # you can call `dot` directly on the `ndarray`

array([[ 7, 10],
       [15, 22]])
np.matmul(a,a)

array([[ 7, 10],
       [15, 22]])

While these functions return the same results for two dimensional data, you should be careful about which you choose when working with other data shapes. You can read more about the differences, and find links to other NumPy functions, in the matmul and dot documentation.

Transpose

Getting the transpose of a matrix is really easy in NumPy. Simply access its T attribute. There is also a transpose() function which returns the same thing, but you’ll rarely see that used anywhere because typing T is so much easier. :)

For example:

m = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
m

array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])
m.T

array([[ 1,  5,  9],
       [ 2,  6, 10],
       [ 3,  7, 11],
       [ 4,  8, 12]])
m.transpose()

array([[ 1,  5,  9],
       [ 2,  6, 10],
       [ 3,  7, 11],
       [ 4,  8, 12]])
m

array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])

NumPy does this without actually moving any data in memory - it simply changes the way it indexes the original matrix - so it’s quite efficient.

However, that also means you need to be careful with how you modify objects, because they are sharing the same data. For example, with the same matrix m from above, let's make a new variable m_t that stores m's transpose. Then look what happens if we modify a value in m_t:

m_t = m.T
m_t[3][1] = 200
m_t

array([[  1,   5,   9],
       [  2,   6,  10],
       [  3,   7,  11],
       [  4, 200,  12]])
m

array([[  1,   2,   3,   4],
       [  5,   6,   7, 200],
       [  9,  10,  11,  12]])

Notice how it modified both the transpose and the original matrix, too! That's because they are sharing the same copy of data. So remember to consider the transpose just as a different view of your matrix, rather than a different matrix entirely.

A real use case

I don't want to get into too many details about neural networks because you haven't covered them yet, but there is one place you will almost certainly end up using a transpose, or at least thinking about it.

Let's say you have the following two matrices, called inputs and weights,

inputs = np.array([[-0.27,  0.45,  0.64, 0.31]])
inputs

array([[-0.27,  0.45,  0.64,  0.31]])
inputs.shape

(1, 4)
weights = np.array([[0.02, 0.001, -0.03, 0.036], \
    [0.04, -0.003, 0.025, 0.009], [0.012, -0.045, 0.28, -0.067]])

weights.shape

(3, 4)
weights

array([[ 0.02 ,  0.001, -0.03 ,  0.036],
       [ 0.04 , -0.003,  0.025,  0.009],
       [ 0.012, -0.045,  0.28 , -0.067]])

I won't go into what they're for because you'll learn about them later, but you're going to end up wanting to find the matrix product of these two matrices.

If you try it like they are now, you get an error:

np.matmul(inputs, weights)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-88-6e050fb6601d> in <module>
----> 1 np.matmul(inputs, weights)

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 3 is different from 4)

If you did the matrix multiplication lesson, then you've seen this error before. It's complaining of incompatible shapes because the number of columns in the left matrix, 4, does not equal the number of rows in the right matrix, 3.

So that doesn't work, but notice if you take the transpose of the weights matrix, it will:

np.matmul(inputs, weights.T).shape

(1, 3)
np.matmul(weights, inputs.T)

array([[-0.01299],
       [ 0.00664],
       [ 0.13494]])

The two answers are transposes of each other, so which multiplication you use really just depends on the shape you want for the output.