Vamp plugins need to be installed in a specific location -- you can't just add them to wherever you installed Sonic Visualiser.
The install location depends on what operating system you are using. You can find details at the bottom of the Vamp plugins download page at
http://vamp-plugins.org/download.html .
Chris