This evening I was reading in Pattern Recognition and Machine Learning, the book we use in our machine learning class. We only use the first half of the book, but we are thinking about extending the class to cover two terms and then cover the entire book (or most of it, anyway) so I figured this was a good excuse to actually read the whole book. So far, I’ve only read the chapters we actually use, plus a few pages here and there.
It is supposed to illustrate kernel functions k(x,x’) as linear combinations of feature functions: k(x,x’)=Σφi(x)φi(x’). The top row shows the feature functions, φi(x), and the bottom row the kernel function, as a function of x with x’ fixed at 0.
That doesn’t make any sense at all to me.
On the left-most figure, the feature functions are all 0 for x’=0, so the kernel function is a sum of zeroes. It should be constant zero, not the curvy blue line.
For the other two, the feature functions are all non-negative, so how can the kernel function ever be negative? A product of non-negative values cannot be negative, and neither can the sum of non-negative numbers.
In short, the figure is all wrong. There isn’t a single thing right about it.
That was my reasoning, in any case, but I wasn’t completely sure. I could be missing something.
So I googled for the book, but then I found powerpoint presentations including the figure, with no mentioning of any errors. Clearly someone was using the figure in their teaching, so maybe it wasn’t wrong after all.
It got me nervous. I feel that I really need to understand something to teach it, so I expect other people to feel the same way, and someone had used this figure.
I am not mentioning names here, ’cause as you have probably guessed the figure is wrong. There is nothing wrong with my reasoning above.
Well, another minutes Googling found me the errata list, and sure enough, the figure is fixed there.
I’m happy to find that I hadn’t completely misunderstood the topic and that I was right about the figure.
I am a little disappointed that a teacher would use the figure without at least checking that the figure actually makes sense. Showing an example that makes no sense at all is doing a lot of harm to the students…