Jeff Dean ML trends

Jeff Dean ML trends Much larger and sparser models     Mixture of experts layer

Automate architecture search: bottleneck to solving problems is ML expertise     Reinforcement learning

    Discover new LSTM cell structures

    Evolutionary algorithms

Need more compute     New computer architectures (TPU; TPU pods with 64 TPUs, 11.5 petaflops)

        Reduced precision is fine (floating point to 1 value…)

        Hardwired for specific operations (matrix math)

    Reinforcement learning to learn how to optimally place model components on GPUs

g.co/tpusignup