See also: Batching

Source: google-ml-course

Variations of gradient descent

• The two extremes:
  • Gradient descent Batch = whole dataset
  • Stochastic Gradient Descent (SGD) Batch = one data point
• The compromise: Mini-Batch Gradient Descent

For neural networks : backpropagation