Every gate computed live with real numbers — edit the sequence, tune the biases, and step through the cell.
Companion to Module 9: Recurrent Neural Networks · Dr. Abdulkarim Albanna
Drag the forget bias to +3: the forget gate saturates near 1 and the cell state accumulates — long memory. Drag it to −3: the gate closes near 0 and the cell forgets everything each step — short-term memory, exactly the failure plain RNNs suffer. The forget bias is why LSTMs are often initialized with bƒ > 0.
The update gate z is the GRU's traffic controller: with bₖ at −3, z ≈ 0 and h₄ = (1−z)·h₄₋₁ keeps the old state almost unchanged (long memory). With bₖ at +3, z ≈ 1 and the state is overwritten by the candidate every step. The reset gate r decides how much of the past the candidate is even allowed to see. Two gates, no separate cell state — that is why GRU is faster than LSTM.
Watch the play order: the forward LSTM reads x₁→xₙ, the backward LSTM reads xₙ→x₁, and only then is each output y₄ formed from both directions. Notice that y₁ already contains information about the end of the sequence — something a one-directional LSTM can never do. That is the whole idea of Bi-LSTM: context from left to right and right to left.
