Incorporating Nesterov Momentum into

Timothy Dozat
2015
1 reference

Abstract

Cognition and behavior exhibit biases consistent with future expectations, and some of these biases result in momentum-like effects and have been linked with the idea of momentum. These momentum-like effects include representational momentum, operational momentum, attentional momentum, behavioral momentum, and psychological momentum. Effects of numerous variables involving characteristics of the target, display, context, or observer on each momentum-like effect are considered, and similarities of different momentum-like effects are considered. It is suggested that representational momentum, operational momentum, and attentional momentum reflect similar or overlapping mechanisms based on a perceptual time-scale and extrapolation primarily across space, and that behavioral momentum and psychological momentum reflect similar or overlapping mechanisms based on a longer time-scale and extrapolation primarily across time. It is further suggested that all 5 forms of momentum-like effect could reflect a more general extrapolation mechanism that anticipates the future action, behavior, or outcome of a given target, person, or process. A list of properties characterizing momentum-like effects is proposed, and constraints and issues relevant to future models of momentum-like effects are discussed. (PsycINFO Database Record

1 repository
1 reference

Code References

â–¶ tensorflow/tensorflow
1 file
â–¶ tensorflow/python/keras/optimizer_v2/nadam.py
1
- [Dozat, 2015](http://cs229.stanford.edu/proj2015/054_report.pdf).
Link copied to clipboard!