README

Vampy is a wrapper for the Vamp audio analysis plugin API.
(http://www.vamp-plugins.org/) It allows for writing Vamp 
plugins in Python.

WHAT IS IT FOR?

Vamp is an audio analysis and feature extraction plugin system 
with a C/C++ Application Programming Interface (API).

Typical applications of Vamp plugins include visualisation, using 
a host such as Sonic Visualiser (https://www.sonicvisualiser.org/),
or batch feature extraction from audio, using Sonic Annotator 
(https://vamp-plugins.org/sonic-annotator).

Vamp plugins are typically written in C++. Although currently
available plugin hosts are valuable tools in audio research,
the long and tedious development cycle of plugins does not 
support quick prototyping of feature extraction algorithms.
Learning the extra skills needed for plugin development or using 
scientific libraries available for C and C++ is often outside 
the interest of audio researches typically using MATLAB or other
high-level development environments.

This package aims at easing Vamp plugin development, prototyping 
or deployment by using the high-level Python scripting language.

WHY PYTHON?

The Python programming language is extremely popular in the
scientific community. Besides being a high-productivity
interpreted language, it has extensions for scientific
computing such as Numpy, an efficient numerical library and
SciPy, a collection of Python modules for signal processing,
linear algebra, statistics and machine learning ...
(www.SciPy.org). These packages together with matplotlib
(http://matplotlib.sourceforge.net/) provide similar
capabilities to most commercial modelling environments. As a
further advantage, Python is a general purpose language which
also supports the functional programming style.

HOW DOES IT WORK?

Vampy acts like a bridge between a Vamp plugin host application
and Python scripts. It translates host function calls to Python
interpreter calls and converts Python data types to C++ and Vamp
defined data structures. 

Vampy is distributed and can be installed like any other ordinary 
Vamp plugin. When it is installed, any appropriately structured 
Python script in its script directory will be presented to 
host programs as if they were native Vamp plugins written in C++.

Vampy embeds the Python interpreter dynamically, and also extends 
it with data types defined by the Vamp C++ API, all within a 
single shared library.

WHAT'S NEW IN THIS RELEASE?

    See the file CHANGELOG for details of changes in this release
    (and previous releases) of Vampy.

OBTAINING VAMPY:

Vampy is a free, cross platform, open source package. The
source code is available from its repository at
https://code.soundsoftware.ac.uk/projects/vampy.

* Binary distributions are available for Windows, macOS, and
Linux.

* The source code can be obtained using the Mercurial command:
hg clone https://code.soundsoftware.ac.uk/hg/vampy

DEPENDENCIES:

* Vampy requires Python 2.7.

Note that Vampy does not support Python 3 at all at this
point.  Note also that on a Mac in normal circumstances Vampy
expects to use the system installation of Python, so plugins
that you write should be tested using the system Python.

* Vampy supports Numpy 1.1 or greater. 

Using Numpy is optional, however writing plugins in pure
Python typically results in significantly slower processing.

BUILDING VAMPY:

It is advised to use a binary distribution if available for
your platform and Python/Numpy versions before attempting to 
compile it from source. If you decide to do so, please use the 
make files provided. Make sure the correct include locations 
are set for Python, Numpy, and the Vamp plugin SDK.

COMPILER OPTIONS:

HAVE_NUMPY : compile with Numpy array interface support

NUMPY_SHORTVERSION : set to the minimum version of Numpy you have, 
	as a floating-point value; the default is 1.1, which should be 
	OK for using the plugin with Numpy 1.1, 1.2 and 1.3

simple debugging (for developers):
	_DEBUG : print more detailed messages while Vampy is in use
	_DEBUG_VALUES : print all converted values to stderr
	
(But note that basic debug messages are compiled in already, and
will be displayed if the environment variable VAMPY_VERBOSE is set.)

USING VAMPY:

(1) Make sure you have Python 2.7 installed and you
	have a recent Vamp plugin host application. 
	(e.g. Sonic Visualier)

(2) Download a version of Vampy compatible with your
	operating system and Python distribution.

(3) Unzip the package and copy the shared library 
	(Windows: vampy.dll, Linux: vampy.so, MacOS: vampy.dylib)
	to your Vamp plugin path.
	
(4) Copy the example plugins (.py files) from the 
	'Example VamPy plugins' directory to the same place.
	(without the example directory itself)

(5) If you are familiar with Python, it is straightforward 
	to start writing your own plugins by following these examples.
	
Note: The interpreter automatically generates a compiled version
of each plugin when their source file is first imported. This 
file can be distributed alone is so desired. Compiled or compiled 
and optimised versions of a plugin can also be obtained using the 
'py_compile' standard library module. (Note that Python byte 
compiled binaries are easier to reverse than C++ binaries.)

Some familiarity with the Vamp plugin SDK and Vamp Plugin 
documentation is assumed before one would start writing a plugin
using Vampy. Only the particularities of Vampy plugins are 
covered here. The Vamp plugin documentation is available at:
* http://www.vamp-plugins.org/code-doc/index.html
* http://www.vamp-plugins.org/guide.pdf

BASIC RULES:

Only the Python scripts that follow some basic rules qualify as
Vampy plugins:

(1) Each plugin must contain a single class with the 
	same name as the script file name.
	
	e.g. PyZeroCrossing.py -> class PyZeroCrossing
	
(2) Vampy plugins have to be in a specific directory designated
	to Vamp plugins. The exact location is platform specific.
	Additionally, you can use the VAMPY_EXTPATH environment
	variable to specify a separate path for Vampy plugins.
	
(3) Vampy plugins can be used and distributed as Python scripts 
	(.py) or byte compiled Python binaries (.pyc / .pyo).
	
	When a script is present with the same name as a compiled
	file on any of the valid paths, the script will be preferred.
	
(4) Vampy may decide to reject some scripts after some basic
	validation is performed:

	* Scripts with syntax errors in them are ignored.
	
	* Scripts not containing a class with the exact same name 
	as the file name are ignored. (Python is case sensitive!)
	
	* Scripts with the wrong number of arguments to the plugin 
	class's __init__() function will be avoided.

	* Scripts that redefine any of Vampy's standard type names
	will be avoided.
	
(5) Unknown scripts may cause undesired behaviour.
	Don't put arbitrary Python scripts in your Vamp directory,
	you may use a subdirectory for that.

PLUGIN ERRORS:

Script validation is performed by the interpreter itself
using the same rules as module compilation. This means that
while most syntax errors will be noted when Vampy is first
used by a host, runtime errors can still occur during
execution. For example, a plugin calculating the dot product
of two vectors with different sizes will produce a runtime error.

Error messages from Vampy are printed on the standard error
channel.
If you're using a graphical host (such as Sonic Visualiser)
you may start the application from a command line terminal
in order to see these messages, or they may be forwarded by
the host to its own debug log file.
	
Exceptions:
	
* Runtime errors occurring in the plugin's __init__() function
will prevent the host from loading the plugin.  

* Runtime errors in the plugin's initialise() function will
prevent the host from using the plugin.
	
* Module level errors resulting from importing a non-existent
module or source file or an error occurring on an imported
module's source tree will prevent the plugin from loading.

Any other error, including those during the process will
only be noted on the terminal output. Processing errors will
generally result in a blank screen or no results displayed by
graphical hosts.

EXTENSION MODULE:

Vampy extends Python with some useful data types defined
by the Vamp plugin API. This extension module is embedded
into the Vampy shared library, therefore it doesn't need 
to be installed separately. However, it works very similarly 
to any third party Python extension within a Vampy plugin.

You may import the extension in the usual manner using
" import vampy " and " from vampy import * ". (Note that
currently the extension module is not available as a 
separate package, therefore this will only work if the 
plugin is executed by Vampy within a usual host context.)

You can use any standard Python statement involving 
modules such as " dir(vampy) " to print the names exported
by the module. The use of the extension in entirely optional,
however its use is strongly advised for the following reasons:

* Using the module hides the mapping between Python and
C++ data types and provides improved plugin portability.
	
* Returning types exported by the module is often faster.
	
* In future releases its use may become mandatory.

PROCESS INTERFACES:

Most computationally intensive processing takes place in
the plugin's process() method. This method has two arguments,
(besides the 'self' argument mandatory in all Python class methods).

* The fist argument is used to pass audio samples (in time 
domain plugins) or frequency samples (complex FFT output)
in frequency domain plugins. This argument is always a
Python list object where each element of the list corresponds
to an audio channel. (The length of this list can not be zero.)
The actual element types contained in this list depends
on the domain type of the plugin (time/frequency domain) and
the selected process interface. (explained below)

* The second argument is the time stamp of the processing 
block passed to the plugin. This time stamp is either
a long integer corresponding to a sample number, or a
RealTime data type exposed by the vampy module.
The use of the time stamp is different in time and frequency
domain plugins. Please refer to the Vamp plugin documentation
for more details.

Vampy supports three interfaces to process() function. 
The interface type can be selected using the flags indicated 
next to the process name below. The detailed use of these 
flags will be explained later.

INTERFACE TYPES:

(1) Legacy interface (default, slowest):

Vampy passes a Python List of List of values to the 
plugin corresponding to each audio channel, and the 
time or frequency domain samples of each channel:

* Audio samples are passed as an N element list 
of floating point values in time domain plugins,
(where N equals to the block size parameter of the plugin).

* Frequency Domain plugins are passed an N element list 
of complex numbers, where N = (blockSize/2) + 1. This list 
includes the DC and the Nyquist frequency FFT oputputs.

Note: This is the only available interface which can be used 
without Numpy or a compatible numerical library.

(2) Buffer interface (vf_BUFFER, fast):

* Both time and frequency domain plugins are passed a list 
of shared memory buffer objects where each buffer corresponds
to an audio channel. The length of these buffers is blockSize
in time domain plugins and blockSize+2 in frequency domain
plugins. The easiest way to access the data in the buffers
is the use of Numpy's frombuffer() command. See the Numpy
documentation or the Vampy example plugins for more details.

Note that this interface is very similar to how the data is
passed to Vamp plugins in C++.

(3) Numpy Array interface (vf_ARRAY, fast):

Vampy passes a list of Numpy arrays to the process() 
corresponding to each audio channel.

* Time Domain plugins are passed an array of numpy.float32
values where the array size is N = blockSize.

* Frequency Domain plugins are passed an array of 
numpy.complex64 values where the size N = (blockSize/2) + 1.

RETURNING VALUES:

Python is a dynamically typed language, which means
that the programmer is not forced to declare variable 
types strictly and specifically, they can be decided 
or changed at runtime. This leads to different programming 
styles compared to using statically typed languages such 
as C++. The Vamp API is declared using C++ and expects 
statically declared types returned by the plugin.
This leads to difficulties to the Python programmer, and
requires a detailed knowledge of the API which otherwise
would be unnecessary. Vampy relaxes this requirement by
using a runtime type inference mechanism.

Vampy can convert just about any suitable Python data
object to the appropriate C++ data type expected by a 
Vamp plugin host. This includes Numpy data types such as 
numpy.float32 or a Numpy array. The type conversion is 
dynamic and it is decided based on the plugin context and 
the expected data type defined by the Vamp plugin API 
in that context. This mechanism also takes advantage of the 
higher level Python number, sequence and mapping protocols.

For example if the Vamp API expects a floating point value,
any returned Python object will be attempted to cast
to a floating point value first and returned to the host.
If the value can not be converted, an error message is 
displayed.

Similarly, any returned value will be converted to a vector of 
the appropriate element type when the expected return type is 
a sequence of values. This allows the programmer to omit 
unnecessary conversions, when, for example, a one element 
list (vector) would be returned.

The type conversion can be controlled specifically for
each plugin. Vampy supports the use case of prototyping
C++ Vamp plugins in Python by using a more strict type
conversion mechanism which would issue an error message
if the Python object does not correspond to a C++ type
according to a strict one-to-one mapping. This mapping
can be briefly outlined as follows:

* numerical types require direct correspondence 
between Python and C++ types when available
e.g. C++ float -> Python float

* Data structures defined in the Vamp Plugin API require
a type exported be the vampy extension module.
Vamp::FeatureSet() -> vampy.FeatureSet()
Vamp::RealTime() -> vampy.RealTime()

The strict type conversion method can be selected using
the Vampy flag: vf_STRICT (explained in the FLAGS section).

TIME STAMPS :

Vamp uses RealTime time stamps to indicate the position of
a processing block passed to the plugin, or the position of
any returned features relative to the start of the audio.
RealTime uses two integer values to represent time values
to nanosecond precision. Vampy provides a Python compatible
representation of this this type which can be imported and 
used in any Vampy plugin. 

* Vampy RealTime objects can be initialised using integers 
corresponding to second and nanosecond values, or seconds (floats).
e.g.: 
	timestamp1 = RealTime(2,0) 
	timestamp2 = RealTime('seconds',2.123)

Please note that only the following methods are available:

* values() : returns a tuple of integers (sec,nsec) 
* toFloat() : return a floating point representation (in seconds)
* toFrame(samplerate) : convert to frame 
	(sample number) given the audio sample rate
* toString() : human readable string representation
* a limited set of arithmetic operators (+,-)

Additionally Vampy provides a function to convert frame
counts (in audio samples) to RealTime:

	timestamp = frame2RealTime(frameCount,inputSampleRate)
	
For the detailed use of time stamps, refer to the Vamp plugin
documentation. i.e. Section 5, "Sample Types and Timestamps"
in the Vamp plugin guide, and the Vamp SDK documentation:
http://vamp-plugins.org/code-doc/classVamp_1_1Plugin.html
on how time stamps are used in process calls.

Note: The support for RealTime time stamps is new in this 
version of Vampy. Vampy 1 used long integer sample counts
instead. This is still accepted for backward compatibility,
but the use of RealTime is encouraged whenever possible.
By default sample counts are used, please set the falg:
vf_REALTIME to obtain RealTime time stamps in process calls. 

	

VAMPY FLAGS :

The execution of Vampy plugins can be controlled using a set
of flags. (Each control flag is prefixed by vf_)

vf_NULL : zero value, default for Vampy version 1 behaviour
vf_DEBUG : print debug messages to standard error
vf_STRICT : strict type conversion (follows the C++ API more closely)
vf_QUIT : quit the host process on hard errors
vf_REALTIME : use RealTime time stamps
vf_BUFFER : use the Numpy Buffer interface 
vf_ARRAY : use the numpy Array interface
vf_DEFAULT_V2 : default Vampy version 2 behaviour 
	(equals to setting: vf_ARRAY | vf_REALTIME)
				
The use of flags is optional. The default behaviour is that 
of Vampy version 1.

To set the flags, place a variable called 'vampy_flags' in
your plugin class's __init__() function.

Example:

class PyMFCC(melScaling): 
	def __init__(self,inputSampleRate):
		self.vampy_flags = vf_DEBUG | vf_ARRAY | vf_REALTIME

ENVIRONMENT VARIABLES:

Vampy recognises these optional environment variables:

VAMPY_VERBOSE if set at all, print out debug info to stderr

VAMPY_COMPILED=1 recognise byte compiled python plugins (default)
VAMPY_COMPILED=0 ignore them 

VAMPY_EXTPATH: if given, searches this path for vampy plugins.
	This is useful if you want to keep your python plugins 
	separate. Only a single absolute path name is recognised.
	
	Example:
  	export VAMPY_EXTPATH="/Users/Shared/Development/vampy-path"

VAMPY_PYLIB: path to the Python shared library to be preloaded
    before scripts are run.  The preload is necessary on some
    systems to support plugins that load additional Python modules.
    Vampy will attempt to preload the right library by default, but
    it sometimes fails; if so, set this variable to override it.

HISTORY:

v1:
* added support for Numpy arrays in processN()
* framecount is now passed also to legacy process() 
	and fixed resulting bugs in the PyZeroCrossing plugin
* added two examples which use Frequency Domain input in processN()

v2.0:
* complete rewrite using generic functions for 
	implementing full error checking on Python/C API calls
* added extension module; 
	supports RealTime and other Vamp type wrappers
	enables a much more readable syntax
* added Numpy Array interface
* added flags
* added environment variables
* recognise byte compiled python scripts 
* new example plugin PyMFCC
* modified all examples for the new syntax
* bug fix: Nyquist frequency FFT output is now passed correctly

TODO: * Vamp 'programs' not implemented * support multiple classes per script in scanner * implement missing methods of vampy.RealTime type

LICENCE:

VamPy is distributed under a "new-style BSD" license; see the
file COPYING for details.  You may modify and redistribute it
within any commercial or non-commercial, proprietary or
open-source context.  VamPy imposes no limitation on how you
may choose to license your own plugin scripts.  Note that
these happen to be the same terms as the Vamp SDK itself.

VamPy was written by Gyorgy Fazekas at the Centre for Digital
  Music, Queen Mary University of London.
Copyright 2008-2009 Gyorgy Fazekas.
Copyright 2008-2019 Queen Mary University of London.