Commit 0081c587 authored by Dan Povey's avatar Dan Povey
Browse files

Further edits to paper.

git-svn-id: https://svn.code.sf.net/p/kaldi/code/sandbox/discrim@534 5e6a8d80-dfce-4ca6-a32a-6e07a63d50c8
parent 25c19fa5
...@@ -324,6 +324,13 @@ $\Theta^{(i+1)}$ with the ones from the prior iteration using an interpolation w ...@@ -324,6 +324,13 @@ $\Theta^{(i+1)}$ with the ones from the prior iteration using an interpolation w
\hat\Theta^{(i)} \leftarrow (1 - \rho) \, \Theta^{(i)} + \rho \, \Theta^{(i-1)} % \quad . \hat\Theta^{(i)} \leftarrow (1 - \rho) \, \Theta^{(i)} + \rho \, \Theta^{(i-1)} % \quad .
\end{equation} \end{equation}
which leads to an exponentially decreasing impact of the initial parameter set. which leads to an exponentially decreasing impact of the initial parameter set.
We implemented this for all the parameters but found it only helped when
applied to just the weights,
which is consistent with our interpretation that stopping the weights from overfitting
too fast allows the Gaussians to ``move farther'' than they otherwise would from
the initial estimate. We note that other practitioners have also used mechanisms that
prevent the weights from changing too fast~\footnote{James Droppo, personal communication} and
have found that this improved results.
%----%<------------------------------------------------------------------------ %----%<------------------------------------------------------------------------
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment