How Learning Differs From Pure Optimization

Challenges In Neural Network Optimization

Basic Algorithms

Parameter Initialization Strategies

Algorithm With Adaptive Learning Rates

Approximate Second Order Methods

Optimization Strategies And Meta Algorithms