What does the “updates” argument do when called this way?
f_grad_shared = theano.function([x, mask, y], cost, updates=zgup + rg2up,
All the documentation I have seen about the “updates” argument in theano functions talk about pairs of the form (shared variables, expression used to update the shared variable). However, here there is only an expression so how to I know which shared variable is updated?
I guess the shared variable is somehow implicit but zgup and rg2up both depends on different shared variables:
zipped_grads = [theano.shared(p.get_value() * numpy_floatX(0.), name='%s_grad' % k) for k, p in tparams.iteritems()] running_grads2 = [theano.shared(p.get_value() * numpy_floatX(0.), name='%s_rgrad2' % k) for k, p in tparams.iteritems()] zgup = [(zg, g) for zg, g in zip(zipped_grads, grads)] rg2up = [(rg2, 0.95 * rg2 + 0.05 * (g ** 2)) for rg2, g in zip(running_grads2, grads)]
This code comes from lstm.py in http://deeplearning.net/tutorial/lstm.html