tutorial_code.dox 15 KB
 Dan Povey committed Aug 05, 2011 1 2 3 4 // doc/tutorial_code.dox // Copyright 2009-2011 Microsoft Corporation  Dan Povey committed Sep 24, 2013 5 6 // See ../../COPYING for clarification regarding multiple authors //  Dan Povey committed Aug 05, 2011 7 8 9 10 11 12 13 14 15 16 17 18 19 20 // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // http://www.apache.org/licenses/LICENSE-2.0 // THIS CODE IS PROVIDED *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY // KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED // WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE, // MERCHANTABLITY OR NON-INFRINGEMENT. // See the Apache 2 License for the specific language governing permissions and // limitations under the License. /**  Dan Povey committed Aug 05, 2011 21  \page tutorial_code Reading and modifying the code (1/2 hour)  Dan Povey committed Aug 05, 2011 22 23 24 25 26 27 28 29  \ref tutorial "Up: Kaldi tutorial"
While the triphone system build is running, we will take a little while to glance at some parts of the code. The main thing you will get out of this section of the tutorial is some idea of how the code is organized and what the  Dan Povey committed Aug 05, 2011 30 31  dependency structure is; and some experience with modifying and debugging the code. If you want to understand it code in more depth, we advise you to follow the links  Dan Povey committed Aug 08, 2011 32  on the \ref index "main documentation page", where we have more detailed documentation  Dan Povey committed Aug 05, 2011 33  organized by topic.  Dan Povey committed Aug 05, 2011 34   Dan Povey committed Aug 05, 2011 35  \section tutorial_code_utils Common utilities  Dan Povey committed Aug 05, 2011 36 37 38  Go to the top-level directory (we called it kaldi-1) and then into  kkm committed Aug 03, 2015 39  src/.  Dan Povey committed Aug 08, 2011 40 41  First look at the file base/kaldi-common.h (don't follow the links within this document; view it from the shell or from an editor). This \#includes a number of  Dan Povey committed Aug 05, 2011 42 43 44 45 46 47 48 49 50  things from the base/ directory that are used by almost every Kaldi program. You can mostly guess from the filenames the types of things that are provided: things like error-logging macros, typedefs, math utility functions such as random number generation, and miscellaneous \#defines. But this is a stripped-down set of utilities; in util/common-utils.h there is a more complete set, including command-line parsing and I/O functions that handle extended filenames such as pipes. Take a few seconds to glance over util/common-utils.h and see what it \#includes. The reason why we segregated a subset of utilities into the base/ directory is so that we could  Dan Povey committed Sep 06, 2011 51  minimize the dependencies of the matrix/ directory (which is useful in  Dan Povey committed Aug 05, 2011 52 53 54 55 56 57  itself); the matrix/ directory only depends on the base/ directory. Look at matrix/Makefile and search for base/ to see how this is specified. Looking at this type of rule in the Makefiles can give you some insight into the structure of the toolkit.  Dan Povey committed Aug 05, 2011 58  \section tutorial_code_matrix Matrix library (and modifying and debugging code)  kkm committed Aug 03, 2015 59   Dan Povey committed Aug 05, 2011 60 61 62  Now look at the file matrix/matrix-lib.h. See what files it includes. This provides an overview of the kinds of things that are in the matrix library. This library is basically a C++ wrapper for BLAS and LAPACK, if that means anything to you (if not,  Dan Povey committed Aug 05, 2011 63  don't worry). The files sp-matrix.h and tp-matrix.h relate to symmetric packed matrices and  Dan Povey committed Aug 05, 2011 64 65 66  triangular packed matrices, respectively. Quickly scan the file matrix/kaldi-matrix.h. This will give you some idea what the matrix code looks like. It consists of a C++ class representing a matrix. We provide a mini-tutorial on the matrix  Dan Povey committed Aug 08, 2011 67 68 69 70 71  library \ref matrix "here", if you are interested. You might notice what seems like a strange comment style in the code, with comments started by three slashes (///). These types of commends, and block comments that begin with /**, are interpreted by the Doxygen software that automatically generates documentation. It also generates the page you are reading right now (the source for this type of documentation  kkm committed Aug 03, 2015 72  is in src/doc/).  Dan Povey committed Aug 05, 2011 73   Dan Povey committed Aug 08, 2011 74 75 76 77  At this point we would like you to modify the code and compile it. We will be adding a test function to the file matrix/matrix-lib-test.cc. As mentioned before, the test programs are designed to abort or exit with nonzero status if something is wrong.  Dan Povey committed Aug 05, 2011 78   Dan Povey committed Aug 05, 2011 79 80  We will be adding a test routine for the function Vector::AddVec. This function adds some constant times one vector, to another vector. Read through the code  Dan Povey committed Aug 08, 2011 81  below  Dan Povey committed Aug 05, 2011 82  and try to understand as much of it as you can (be careful: we have deliberately  Dan Povey committed Aug 08, 2011 83  inserted two errors into the code). If you are not familiar with templates,  Dan Povey committed Aug 05, 2011 84 85 86 87 88 89  understanding it may be difficult. We have tried to avoid the use of templates as much as possible, so large parts of Kaldi are still understandable without knowing template progamming. \verbatim template void UnitTestAddVec() { // note: Real will be float or double when instantiated.  Augusto Henrique Hentz committed Aug 12, 2014 90  int32 dim = 1 + Rand() % 10;  Dan Povey committed Aug 08, 2011 91  Vector v(dim); w(dim); // two vectors the same size.  Dan Povey committed Aug 05, 2011 92 93 94  InitRand(&v); InitRand(&w); Vector w2(w); // w2 is a copy of w.  kkm committed Aug 03, 2015 95  Real f = RandGauss();  Dan Povey committed Aug 05, 2011 96 97 98 99 100  w.AddVec(f, v); // w <-- w + f v for (int32 i = 0; i < dim; i++) { Real a = w(i), b = f * w2(i) + v(i); AssertEqual(a, b); // will crash if not equal to within // a tolerance.  kkm committed Aug 03, 2015 101  }  Dan Povey committed Aug 05, 2011 102 103 } \endverbatim  Dan Povey committed Aug 08, 2011 104 Add this code to the file matrix-lib-test.cc, just above the function  Dan Povey committed Aug 05, 2011 105 106 107 108 109 MatrixUnitTest(). Then, inside MatrixUnitTest(), add the line: \verbatim UnitTestAddVec(); \endverbatim It doesn't matter where in the function you add this.  Ho Yin Chan committed Sep 26, 2013 110 Then type "make test". There should be an error (a semicolon that should be  Dan Povey committed Aug 08, 2011 111 a comma); fix it and try again.  kkm committed Aug 03, 2015 112 Now type "./matrix-lib-test". This should crash with an assertion failure,  Dan Povey committed Aug 08, 2011 113 because there was another mistake in the unit-test code. Next we will debug it.  Dan Povey committed Aug 05, 2011 114 115 116 117 118 119 120 121 122 123 124 125 Type \verbatim gdb ./matrix-lib-test \endverbatim (if you are on cygwin, you should now type into the gdb prompt, "break __assert_func"). Type "r". When it crashes, it calls abort(), which gets caught by the debugger. Type "bt" to see the stack trace. Nagivate up the stack by typing "up" until you are inside the test function. When you are at the right place you should see output like: \verbatim #5 0x080943cf in kaldi::UnitTestAddVec () at matrix-lib-test.cc:2568 2568 AssertEqual(a, b); // will crash if not equal to within \endverbatim  Dan Povey committed Aug 08, 2011 126 If you go too far you can type "down". Then type "p a" and "p b" to see the  Dan Povey committed Aug 05, 2011 127 128 129 130 131 132 values of a and b ("p" is short for "print"). Your screen should look someting like this: \verbatim (gdb) p a $5 = -0.931363404 (gdb) p b$6 = -0.270584524  kkm committed Aug 03, 2015 133 (gdb)  Dan Povey committed Aug 05, 2011 134 135 136 137 138 139 140 141 142 143 144 145 146 147 \endverbatim The exact values are, of course, random, and may be different for you. Since the numbers are considerably different, it's clear that it's not just a question of the tolerances being wrong. In general you can access any kind of expression from the debugger using the "print" expression, but the parenthesis operator (expressions like "v(i)") doesn't work, so to see the values inside the vectors you have to enter expressions like the following: \verbatim (gdb) p v.data_[0] $8 = 0.281656802 (gdb) p w.data_[0]$9 = -0.931363404 (gdb) p w2.data_[0] \$10 = -1.07592916  kkm committed Aug 03, 2015 148 (gdb)  Dan Povey committed Aug 05, 2011 149 150 151 152 \endverbatim This may help you work out that the expression for "b" is wrong. Fix it in the code, recompile, and run again (you can just type "r" in the gdb prompt to rerun). It should now run OK. Force gdb to break into the code at the point where it was previously failing, so you can check the values of the expressions again  Dan Povey committed Aug 08, 2011 153 and see that things are now working OK. To get the debugger to break there you have to set a  Dan Povey committed Aug 05, 2011 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 breakpoint. Work out the line number that the assertion was failing (somewhere in UnitTestAddVec()), and type into gdb something like the following: \verbatim (gdb) b matrix-lib-test.cc:2568 Breakpoint 1 at 0x80943b4: file matrix-lib-test.cc, line 2568. (4 locations) \endverbatim Then run the program (type "r"), and when it breaks there, look at the values of the expressions using "p" commands. To continue, type "c". It will keep stopping there since it was inside a loop. Type "d 1" to delete the breakpoint (assuming it was breakpoint number one), and type "c" to continue. The program should run to the end. Type "q" to quit the debugger. If you need to debug a program that takes command-line arguments, you can do it like: \verbatim gdb --args kaldi-program arg1 arg2 ... (gdb) r ... \endverbatim or you can invoke gdb without arguments and then type "r arg1 arg2..." at the prompt.  kkm committed Aug 03, 2015 172 \todo This paragraph is full of lies!  Dan Povey committed Aug 05, 2011 173 174 175 176 When you are done, and it compiles, type \verbatim svn diff \endverbatim  Dan Povey committed Aug 08, 2011 177 178 to see what changes you made. If you are contributing to the Kaldi project and you are planning to commit code in the near future, you  Dan Povey committed Aug 05, 2011 179 may want to revert the changes you made so you don't accidentally commit them. The following  kkm committed Aug 03, 2015 180 commands will save the file you modified in case you need it later, and will revert to  Dan Povey committed Aug 08, 2011 181 the original version:  Dan Povey committed Aug 05, 2011 182 183 184 185 186 187 188 189 190 191 192 \verbatim cp matrix-lib-test.cc matrix-lib-test.cc.tmp svn revert matrix-lib-test.cc \endverbatim If you actually wanted to commit the changes, and you had an account on Sourceforge, you would have to ask us to add you to the Kaldi project, and you would type something like \verbatim svn commit --username=your_sourceforge_username -m "Added a unit-test in matrix/ directory." \endverbatim \section tutorial_code_acoustic Acoustic modeling code  Dan Povey committed Aug 05, 2011 193   kkm committed Aug 03, 2015 194 Next look at gmm/diag-gmm.h (this class stores a Gaussian Mixture Model).  Dan Povey committed Aug 05, 2011 195 196 197 198 The class DiagGmm may look a bit confusing as it has many different accessor functions. Search for "private" and look at the class member variables (they always end with an underscore, as per the Kaldi style). This should make it clear how we store the GMM.  kkm committed Aug 03, 2015 199 This is just a single GMM, not a whole collection of GMMs.  Dan Povey committed Aug 05, 2011 200 201 202 203 204 205 206 207 208 Look at gmm/am-diag-gmm.h; this class stores a collection of GMMs. Notice that it does not inherit from anything. Search for "private" and you can see the member variables (there are only two of them). You can understand from this how simple the class is (everything else consists of various accessors and convenience functions). A natural question to ask is: where are the transitions, where is the decision tree, and where is the HMM topology? All of these things are kept separate from the acoustic model, because it's likely that researchers might want to replace the acoustic likelihoods while  Dan Povey committed Aug 08, 2011 209 keeping the rest of the system the same. We'll come to this other stuff later.  Dan Povey committed Aug 05, 2011 210   Dan Povey committed Aug 05, 2011 211 212 \section tutorial_code_feat Feature extraction code  Dan Povey committed Aug 05, 2011 213 214 Next look at feat/feature-mfcc.h. Focus on the MfccOptions struct. The struct members give you some idea what kind of options are supported  kkm committed Aug 03, 2015 215 in MFCC feature extraction.  Dan Povey committed Aug 05, 2011 216 217 Notice that some struct members are options structs themselves. Look at the Register function. This is standard in Kaldi options classes.  Dan Povey committed Aug 08, 2011 218 219 220 Then look at featbin/compute-mfcc-feats.cc (this is a command-line program) and search for Register. You can see where the Register function of the options struct is called.  Dan Povey committed Aug 05, 2011 221 222 To see a complete list of the options supported for MFCC feature extraction, execute the program featbin/compute-mfcc-feats with no arguments.  kkm committed Aug 03, 2015 223 224 Recall that you saw some of these options being registered in the MfccOptions class, and others being registered in  Dan Povey committed Aug 05, 2011 225 featbin/compute-mfcc-feats.cc. The way to specify options is --option=value.  Dan Povey committed Aug 08, 2011 226 227 228 229 230 231 232 233 234 235 236 237 Type \verbatim featbin/compute-mfcc-feats ark:/dev/null ark:/dev/null \endverbatim This should run successfuly, as it interprets /dev/null as an empty archive. You can try setting the options using this example. Try, for example, \verbatim featbin/compute-mfcc-feats --raw-energy=false ark:/dev/null ark:/dev/null \endverbatim The only useful information you get from this is that it doesn't crash; try removing the "=" sign or abbreviating the option name or changing the number of arguments, and see that it fails and prints a usage message.  Dan Povey committed Aug 05, 2011 238   Dan Povey committed Aug 05, 2011 239 240 \section tutorial_code_acoustic Acoustic decision-tree and HMM topology code  Dan Povey committed Aug 05, 2011 241 242 243 244 245 246 Next look at tree/build-tree.h. Find the BuildTree function. This is the main top-level function for building the decision tree. Notice that it returns a pointer the type EventMap. This is a type that stores a function from a set of (key, value) pairs to an integer. It's defined in tree/event-map.h. The keys and values are both integers, but the keys represent phonetic-context positions (typically 0, 1 or 2) and the values represent phones. There is also a special  Dan Povey committed Aug 08, 2011 247 key, -1, that roughly represents the position in the HMM. Go to the experimental  Dan Povey committed Feb 28, 2015 248 directory (../egs/rm/s5), and we are going to look at how the tree is built.  Dan Povey committed Aug 08, 2011 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 The main input to the BuildTree function is of type BuildTreeStatsType, which is a typedef as follows: \verbatim typedef vector > BuildTreeStatsType; \endverbatim Here, EvenType is the following typedef: \verbatim typedef vector > EventType; \endverbatim The EventType represents a set of (key,value) pairs, e.g. a typical one would be { {-1, 1}, {0, 15}, {1, 21}, {2, 38} } which represents phone 21 with a left-context of phone 15, a right-context of phone 38, and "pdf-class" 1 (which in the normal case means it's in state number 1, which is the middle of three states). The Clusterable* pointer is a pointer to a virtual class which has a generic interface that supports operations like adding statistics together and evaluating some kind of objective function (e.g. a likelihood). In the normal recipe, it actually points to a class that contains sufficient statistics for estimating a diagonal Gaussian p.d.f..  kkm committed Aug 03, 2015 268 Do  Dan Povey committed Aug 08, 2011 269 \verbatim  Jan Trmal committed Mar 02, 2015 270 less exp/tri1/log/acc_tree.log  Dan Povey committed Aug 08, 2011 271 \endverbatim  kkm committed Aug 03, 2015 272 There won't be much information in this file, but you can see the command  Dan Povey committed Aug 08, 2011 273 274 line. This program accumulates the single-Gaussian statistics for each HMM-state (actually, pdf-class) of each seen triphone context.  Dan Povey committed Sep 22, 2012 275 The --ci-phones options is so that it knows to avoid accumulating separate  Dan Povey committed Aug 08, 2011 276 277 278 279 280 281 282 statistics for distinct context of phones like silence that we don't want to be context dependent (this is an optimization; it would work without this option). The output of this program can be thought of as being of the type BuildTreeStatsType discussed above, although in order to read it we have to know what concrete type it is. Do \verbatim  Jan Trmal committed Mar 02, 2015 283 less exp/tri1/log/train_tree.log  Dan Povey committed Aug 08, 2011 284 285 286 287 288 \endverbatim This program does the decision-tree clustering; it reads in the statistics that were output by. It is basically a wrapper for the BuildTree function discussed above. The questions that it asks in the decision-tree clustering are automatically generated, as you can see in the script steps/train_tri1.sh (look for the programs cluster-phones  kkm committed Aug 03, 2015 289 and compile-questions).  Dan Povey committed Aug 08, 2011 290 291 292 293   Dan Povey committed Aug 05, 2011 294 295 296 297 298  Next look at hmm/hmm-topology.h. The class HmmTopology defines a set of HMM topologies for a number of phones. In general each phone can have a different topology. The topology includes "default" transitions, used for initialization. Look at the example topology in the extended comment at the top of the header.  Dan Povey committed Aug 05, 2011 299 There is a tag (note: as with HTK text formats,  kkm committed Aug 03, 2015 300 this file looks vaguely XML-like, but it is not really XML).  Dan Povey committed Aug 05, 2011 301 302 The is always the same as the HMM-state () here; in general, it doesn't have to be. This is a mechanism to enforce tying of  Dan Povey committed Aug 05, 2011 303 distributions between distinct HMM states; it's possibly useful if you want to  Dan Povey committed Aug 05, 2011 304 5~create more interesting transition models.  Dan Povey committed Aug 05, 2011 305 306 307 308 309  \ref tutorial "Up: Kaldi tutorial"
*/