Broadcasting Examples

3.2. Broadcasting Examples#

General broadcasting rules (direcetly taken from NumPy website)

When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimension and works its way left. Two dimensions are compatible when

  1. they are equal, or

  2. one of them is 1.

Here is an example. Generate two arrays \(a\) and \(b\), of sizes 4 and 3, respectively, from NumPy’s random.randint() function. Subtract each element of \(a\) from each element of \(b\). The solution to this problem is to create a new “dummy axis” for each array. Here is the code using broadcasting compared to the same result using loops.

import numpy as np

a = np.random.randint(0,10, 4)
print(a) 
b = np.random.randint(0,10, 3) 
print(b) 

[9 4 8 8]
[4 0 4]

a_new = a[np.newaxis,:]
b_new = b[:, np.newaxis] 
result = a_new - b_new
result

array([[5, 0, 4, 4],
       [9, 4, 8, 8],
       [5, 0, 4, 4]])

# compare result to solution via loop
c = np.zeros((b.size, a.size))
for i in range(b.size): 
    for j in range(a.size): 
        c[i,j] = a[j] - b[i]
c

array([[5., 0., 4., 4.],
       [9., 4., 8., 8.],
       [5., 0., 4., 4.]])

Here is a broadcasting example (it comes from a common machine learning algorithm, btw).

Suppose \(X\) is an \(n \times m\) array and \(c\) is an \(l \times m\) array. I would like to subtract all \(n\) rows of \(X\) from all \(l\) rows of \(c\). How can I do that? Below I generate random data and show how to do this utilizing broadcasting.

To generate data, I will use the NumPy random.rand() function, which generates random numbers from 0 to 1.

# shapes of the arrays
n=15
l=4
m=3

# data for X

X = np.random.rand(n,m)
print(X)
print(X.shape) 

[[0.72912462 0.55286191 0.11849553]
 [0.69030164 0.66382608 0.35079224]
 [0.88606929 0.31028155 0.12060671]
 [0.69095046 0.75949483 0.61086272]
 [0.22373225 0.43869009 0.7842831 ]
 [0.53745638 0.64817916 0.36578295]
 [0.14525095 0.36039602 0.03347568]
 [0.74549505 0.02568234 0.96833873]
 [0.01272951 0.26436737 0.88082223]
 [0.5089394  0.92154922 0.07458645]
 [0.63826294 0.82527568 0.92961715]
 [0.02515156 0.88645077 0.39173641]
 [0.15941439 0.37870577 0.35056122]
 [0.2082888  0.05167067 0.81250795]
 [0.90131518 0.37090267 0.69905071]]
(15, 3)

# data for c

c = np.random.rand(l,m)
print(c)
print(c.shape) 

[[0.36268575 0.93696608 0.64044899]
 [0.82408395 0.58231496 0.33772182]
 [0.06346968 0.04380561 0.76927634]
 [0.53738355 0.47515018 0.35641984]]
(4, 3)

To subtract the two, I need to create a new axis for each array. You want to subtract every row of \(X\) from every row of \(c\), creating all possible pairwise subtractions. The solution is to use NumPy broadcasting by adding extra dimensions to each array:

  • Reshape \(X\) from (n,m) to (n,1,m) - adds a “dummy” dimension

  • Reshape \(c\) from (l,m) to (1,l,m) - adds a “dummy” dimension

NumPy will automatically “broadcasts” (repeats) the arrays along the dummy dimensions:

X gets repeated l times along the middle dimension c gets repeated n times along the first dimension Result is (n,l,m) where entry [i,j,:] = c[j,:] - X[i,:]

X_new = X[:, np.newaxis, :]
X_new.shape

(15, 1, 3)

c_new = c[np.newaxis, :,:]
c_new.shape

(1, 4, 3)

result= X_new - c_new

result.shape

(15, 4, 3)