Parent: Variations of gradient descent

Source: google-ml-course

Stochastic Gradient Descent (SGD)

• Performs gradient calculation on only a single data point
• Gradient is very noisy
• ‘Stochastic’: random data point is chosen at every iteration