Question 1: Short answer
hardAI-checkedLogistic Regression & SoftmaxOne batch gradient descent step of softmax regression in NumPy starts like this. What does the last line print?
import numpy as np
X = np.array([[1.0, 2.0], [1.0, -1.0]]) # column 0 is the bias feature x0 = 1
y = np.array([0, 1]) # class indices
Theta = np.zeros((2, 2)) # one column of parameters per class
logits = X @ Theta
P = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
Y = np.diag(np.ones(2))[y] # one-hot targets
grad = X.T @ (P - Y) / len(X)
print(grad[1, 0])