AssertionError when normalizing matrix in Python

Question

I have defined the following function in Python to return a normalized version of the matrix M.

def get_normalized_matrix(M, O):
    M_copy = np.copy(M)
    M_copy[O == 0] = 0
    a = np.sum(M_copy) / np.sum(O)
    s = np.sqrt(np.sum((M_copy - a)**2) / np.sum(O))
    M_normalized = (M_copy - a) / s
    return M_normalized

After running the following check, I get an AssertionError for assert(abs(np.sum(Mn**2 * O) / np.sum(O) - 1) < 1e-6).

def check_normalization():
    M = np.random.rand(10, 3)
    O = (np.random.rand(10, 3) > 0.5) + 0
    Mn = get_normalized_matrix(M, O)
    assert(abs(np.sum(M * O)) > 1e-6)
    assert(abs(np.sum(Mn * O)) < 1e-6)
    assert(abs(np.sum(Mn**2 * O) / np.sum(O) - 1) < 1e-6)
    print("Function {} is working fine!".format('get_normalized_matrix'))
    
check_normalization()

Can you help me figure out where I went wrong and how I can fix this?

What are the actual results that cause the comparison to fail? — chepner
– chepner, Commented Mar 28, 2023 at 18:44
Did you mean to do assert(abs(np.sum(Mn**2 * O) / (np.sum(O) - 1)) < 1e-6) instead of assert(abs(np.sum(Mn**2 * O) / np.sum(O) - 1) < 1e-6)? Moreover, the assertion error occurs because the value returned from abs(np.sum(Mn**2 * O) / np.sum(O) - 1) is less than 1e-6. — Marcelo Paco
– Marcelo Paco, Commented Mar 28, 2023 at 18:46

Harshit chaturvedi · Accepted Answer · 2023-03-28 18:50:24Z

0

The AssertionError indicates that there is a problem with the normalization of the matrix, specifically that the squared sum of the normalized matrix Mn weighted by the O matrix is not equal to 1.

One possible issue in the code could be the order of operations in the normalization step. Instead of first subtracting the mean a and then dividing by the standard deviation s, it might be more accurate to first divide by the standard deviation and then subtract the mean. This is because the order of operations can affect the numerical precision of the calculations.

To fix this, you can modify the function as follows:

def get_normalized_matrix(M, O):
M_copy = np.copy(M)
M_copy[O == 0] = 0
s = np.sqrt(np.sum((M_copy - np.mean(M_copy[O > 0]))**2) / np.sum(O))
M_normalized = (M_copy - np.mean(M_copy[O > 0])) / s
return M_normalized

answered Mar 28, 2023 at 18:50

Harshit chaturvedi

845 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

amnpawar · Accepted Answer · 2023-03-28 18:52:06Z

0

Replace

assert(abs(np.sum(Mn**2 * O) / np.sum(O) - 1) < 1e-6)

With

assert(abs(np.sum(Mn**2 * O) / np.sum((O) - 1) < 1e-6))

It's a missing parenthesis

answered Mar 28, 2023 at 18:52

amnpawar

1087 bronze badges

Collectives™ on Stack Overflow

AssertionError when normalizing matrix in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related