Optimization

#Optimization is the core of #machinelearning. All GPU acceleration, cloud systems, and transfer learning in ML are for better optimization. There are many attempts to make it faster. However, there are no widely adopted second-order methods in the field despite some momentum methods depicting them. I believe there is still potential in this area. Having a robust 10% better algorithm could have billion of dollars of impact on the industry. Of course, it is very hard to achieve.