This is more of a reminder for myself, but I’m sure it will be useful for someone else.
What do you think the output of this should be?
a = tf.random.shuffle([1,2,3,4,5,6], seed=567436).numpy()
b = tf.random.shuffle([1,2,3,4,5,6], seed=567436).numpy()
a,b
If you were expecting a and b to be the same, you’re in for a nasty surprise.
What you should be doing is:
tf.random.set_seed(123456)
a = tf.random.shuffle([1,2,3,4,5,6], seed=567436).numpy()
tf.random.set_seed(123456)
b = tf.random.shuffle([1,2,3,4,5,6], seed=567436).numpy()
a,b
(array([6, 4, 2, 1, 5, 3], dtype=int32),
array([6, 4, 2, 1, 5, 3], dtype=int32))
Note the call to tf.random.set_seed() - make sure you do this before each operation.
TensorFlow has two random seeds - of course, this is actually well documented and I should have read the docs first before diving in:
Operations that rely on a random seed actually derive it from two seeds: the global and operation-level seeds. This sets the global seed.
Its interactions with operation-level seeds is as follows:
- If neither the global seed nor the operation seed is set: A randomly picked seed is used for this op.
- If the graph-level seed is set, but the operation seed is not: The system deterministically picks an operation seed in conjunction with the graph-level seed so that it gets a unique random sequence. Within the same version of TensorFlow and user code, this sequence is deterministic. However across different versions, this sequence might change. If the code depends on particular seeds to work, specify both graph-level and operation-level seeds explicitly.
- If the operation seed is set, but the global seed is not set: A default global seed and the specified operation seed are used to determine the random sequence.
- If both the global and the operation seed are set: Both seeds are used in conjunction to determine the random sequence.
Now to work out why despite feeding it complete garbage my neural network appeared to train really well…