embedding-dimensions

Search IconIcon to open search

Parent: hyperparameters

Source: google-ml-course

Embedding dimensions

Good starting value: $$D \approx \sqrt[4]{\text{total values}}$$

Higher dimensions

  • allow more accurate representation of the relationships between the input data
  • can lead to overfitting, slower training