Previous Section | Next Section | Table of Contents | Index | Title Page

## Interactive Nyquist

Nyquist is not intended for real-time performance, but it has some features that allow you to adjust parameters interactively. The basic idea is that there is an array of 1000 floating point values, called sliders, that can be accessed while synthesizing sounds in Nyquist. The `slider` unit generator returns a signal that copies the current value of a slider value. You can change the slider value while playing sounds using either Open Sound Control or the NyquistIDE.

Sounds are normally computed on demand. So the result returned by `slider` does not immediately compute any samples. Samples are only computed when something tries to use this signal. At that time, the slider value is read. Normally, if the slider is used to control a sound, you will hear changes in the sound pretty soon after the slider value changes. However, one thing that can interfere with this is that `SOUND` samples are computed in blocks of about 1000 samples. When the slider value is read, the same value is used to fill a block of 1000 samples, so even if the sample rate is 44,100 Hz, the effective slider sample rate is 44,100/1000, or 44.1 Hz. If you give the slider a very low sample rate, say 1000, then slider value changes will only be noticed by Nyquist approximately once per second. For this reason, you should normally use the audio sample rate (typically 44,100 Hz) for the rate of the `snd-slider` output `SOUND`. (Yes, this is terribly wasteful to represent each slider value with 1000 samples, but Nyquist was not designed for low-latency computation, and this is an expedient work-around.)

When you load `sliders.lsp`, which defines a number of slider functions, two important settings may be changed. First `autonorm-off` is called. The problem with auto-normalization is that it works by computing 1 million samples ahead of real time to determine a normalization factor. If Nyquist computes ahead, it will be unable to respond to control changes until the million samples (about 20 seconds) have been played. Secondly, `snd-set-latency` is used to set the audio latency to 0.02s (20 milliseconds). Normally, Nyquist uses a generous 0.3s latency which allows Nyquist to stop computing audio and run garbage collection without breaks in the audio output. At 20ms, interactivity is greatly enhanced because changes do not sit in audio buffers for 300ms, but you may notices some break-up in the audio, especially when garbage collection takes place. The latency can be changed to any value you like after you load `sliders.lsp`.

In addition to reading sliders as continually changing `SOUND`s, you can get the slider value as a Lisp `FLONUM` (a floating point number) using `get-slider-value`. This might be useful if you are computing a sequence of many notes (or other sound events) and want to apply the current slider value to the whole note or sound event.

Other unit generators exist to instantiate behaviors and to stop sounds according to slider values. These will be described below.

### Interactive Control with the NyquistIDE

To control sounds interactively using the NyquistIDE, you first create a control panel, then populate the panel with sliders and buttons. These will send values into the sliders array in the Nyquist process. To control a sound, you use built-in functions to retrieve real-time values from the sliders array as sound is being played. Further discussion and examples can be found in `nyquist/lib/sliders/slider-demos.sal`.

#### Creating a Control Panel

A control panel is created with `make-slider-panel`, which takes a panel name and color as parameters. Control panels can only be created by executing code in Nyquist. There is no way to configure control panels directly using the NyquistIDE. Control panels can be deleted interactively using the close button on the panel or through code by calling `close-slider-panel`.

`make-slider-panel(name, color)` [SAL]
`(make-slider-panel name color)` [LISP]
  Create a control panel in the IDE. The title of the control panel window is given by name, a STRING. The color of the panel is given by color, a FIXNUM from 0 through 12. The color 0 is gray. Other colors are implementation-dependent, but different numbers give distinguishable colors. You must load `sliders.lsp` to access this function.
`close-slider-panel(name)` [SAL]
`(close-slider-panel name)` [LISP]
  Close a control panel in the NyquistIDE named by name, a STRING. Any embedded controls (sliders or buttons) are also destroyed. You must load `sliders.lsp` to access this function.

#### Creating Controls

You can create slider and button controls. A slider control adjusts a floating point value in the sliders array and accessible as a time-varying signal (a `SOUND`) or as a `FLONUM`. A button control sets a floating point value in the sliders array to zero (0.0) or one (1.0). The value changes when the left mouse button is pressed over the button control and changes back when the mouse button is released.

`make-slider(name, init, low, high)` [SAL]
`(make-slider name init low high)` [LISP]
  Create a slider in the most recently created control panel. (Thus, you should populate a control panel with sliders and buttons before creating another control panel.) Sliders have a label specified by name, a STRING, an initial value specified by init, a FLONUM, a minimum value specified by low, a FLONUM, and a maximum value specified by high, a FLONUM. Sliders are added to the current panel in order from top to bottom. You must load `sliders.lsp` to access this function.
`make-button(name [, normal])` [SAL]
`(make-button name [normal])` [LISP]
  Create a button in the most recently created control panel. Buttons have a label specified by name, a STRING, and a "normal value" specified by 0 or 1 (a FIXNUM). If 0 is specified, the value controlled by the button is 0 when the button is released and 1 when the button is pressed. If 1 is specified, the normal value is 1 changing to 0 when the button is pressed. Buttons are added to the control panel in order from top to bottom. You must load `sliders.lsp` to access this function.

#### Accessing Control Values

Each control created by `make-slider` or `make-button` is assigned a slider index from 10 to 999. Control changes are passed via hidden text input from the NyquistIDE to the nyquist process, where the values are converted to floats and stored in the slider array. You can then access these values with either `slider`, `lpslider`, or `get-slider-value`.

`slider(number [, dur])` [SAL]
`slider(name [, dur])` [SAL]
`slider(panel, name [, dur])` [SAL]
`(slider number [dur])` [LISP]
`(slider name [dur])` [LISP]
`(slider panel name [dur])` [LISP]
  Create a `SOUND` that reads signal values from the slider array. In the first form, the first parameter is the index (a FIXNUM) of the value in the slider array. In the second form, the slider value will be controlled by the NyquistIDE control created by `make-slider` or `make-button` using the same name, a STRING. The control must be in the most recently created panel. In the third form, the panel named panel is searched for the control named name to determine the value. In all cases, the optional dur, a FLONUM, is used to determine the duration of the sound. This duration is scaled by the environment in the usual way. You must load `sliders.lsp` to access this function.
`lpslider(number [, dur])` [SAL]
`lpslider(name [, dur])` [SAL]
`lpslider(panel, name [, dur])` [SAL]
`(lpslider number [dur])` [LISP]
`(lpslider name [dur])` [LISP]
`(lpslider panel name [dur])` [LISP]
  Create a `SOUND` based on the value of an interactive control. This function is exactly like `slider`, except the sound is low-pass filtered to avoid sudden jumps when the control value is adjusted. The low-pass filter cutoff is determined by `*lpslider-cutoff*`, which is initialized to 20Hz when `slider.lsp` is loaded. You must load `sliders.lsp` to access this function.
`get-slider-value(number)` [SAL]
`get-slider-value(name)` [SAL]
`get-slider-value(panel, name)` [SAL]
`(get-slider-value number)` [LISP]
`(get-slider-value name)` [LISP]
`(get-slider-value panel name)` [LISP]
  Get a value (a FLONUM) stored in the slider array. The array is accessed directly if the parameter is number, a FIXNUM. The index can be determined by searching the most recently created control panel by name, a STRING, if only name is given. If both panel and name are given (STRINGS), the named panel is searched for the named control. The result is a FLONUM. You must load `sliders.lsp` to access this function.
`snd-slider(index, t0, srate, duration)` [SAL]
`(snd-slider index t0 srate duration)` [LISP]
  Create a sound controlled by the slider named by index (an integer index into the array of sliders). The function returns a sound. Since Nyquist sounds are computed in blocks of samples, and each block is computed at once, each block will contain copies of the current slider value. Normally, you would call `slider` (see above) rather than this low-level function.

#### Starting and Stopping Sounds

All Nyquist sounds have a duration. Even the `slider` unit generator has a duration and terminates at the end of that duration. In most cases, you will instead want interactive functions to run until you interactively ask them to stop. The `stop-on-zero` function terminates when an input signal (typically a slider) goes to zero. This can be used to terminate a complex interatively controlled sound. For example, here is a tone that terminates when the Stop button is pressed:

    exec make-button("Stop", 1)

    function can-stop()
      play (hzosc(1000) * stop-on-zero(slider("Stop"))) ~ 100

The stretch factor of 100 will cause this tone to play for 100 seconds. However, if the button named "Stop" is pressed, it will change value from the "normal" value 1 to 0. When the signal goes to zero, `stop-on-zero` will terminate. The multiplication then immediately terminates (because anything multiplied by zero will be zero; termination is just an efficient way to return zeros from now on). Since the multiplication is the top-level sound being played, the play stops.

Another thing you might want to do with interactive control is start some sound. The `trigger` function computes an instance of a behavior each time an input `SOUND` goes from zero to greater-than-zero. This can be used, for example, to create a sequence of sound events interactively. For example:

    exec make-button("Trigger", 0)

    function trigger-me()
      play trigger(slider("Trigger", 100), pluck(c3))

Here, a button control changes value from 0 to 1 when the button is pressed. This value is retrieved by the `slider` function, which runs for 100 seconds. When the button and hence the `slider` goes from 0 to 1, the behavior, `pluck(c3)` is instantiated. Many instances can be triggered by the button.

`stop-on-zero(s)` [SAL]
`(stop-on-zero s)` [LISP]
  Return a `SOUND` that is identical to s, a `SOUND`, except the returned sound terminates when s first goes to zero. When a sound terminates, it remains at zero. A `SOUND` multiplication terminates when either parameter terminates, so multiplying by `stop-on-zero` is a way to terminate a sound interactively. (See the example above.) You must load `sliders.lsp` to access this function.
`trigger(s, beh)` [SAL]
`(trigger s beh)` [LISP]
  Returns a sound which is the sum of zero or more possibly overlapping instances of the behavior beh. One instance is created each time `SOUND` s makes a transition from less than or equal to zero to greater than zero. (If the first sample of s is greater than zero, an instance is created immediately.) The start time of the result is the start time of s, and zero samples will be generated until the first instance of beh. The sample rate of the result is `*sound-srate*`, and all instances of beh must have the same `*sound-srate*` sample rate. The behaviors (instances of beh) must be (monophonic) `SOUND`s. The stop time of the result is the maximum stop time of s and all sounds returned by instances of the behavior. This function is particularly designed to allow behaviors to be invoked in real time. See the `trigger-me` function definition shown above. An implementation note: There is no way to have `trigger` return a multichannel sound. An alternative implementation would be a built-in function to scan ahead in a sound to find the time of the next zero crossing. This could be combined with some LISP code similar to `seq` to sum up instances of the closure. However, this would force arbitrary look-ahead and therefore would not work with real-time inputs, which was the motivation for `trigger` in the first place. Warning: The beh argument of `trigger` is converted to a closure that captures the current environment, including any variables in scope. The following example illustrates a problem where s is a local variable:

      (defun example (snd) (trigger snd (pluck c4)))

  or in SAL:

      function example(snd) return trigger(snd, pluck(c4))

  The problem here is that snd will be captured by the closure built from `pluck(c4)`. As `trigger` begins to evaluate snd looking for zero crossings, the samples from snd will be retained in memory because snd is retained in the closure. A solution is the following:

      (defmacro unbind (sym) `(let ((x ,sym)) (setf ,sym nil) x))
      (defun example (snd) (trigger (unbind snd) (pluck c4)))

  or in SAL:

      ;; unbind must be defined as above and loaded from a .lsp file
      function example(snd) return trigger(unbind(snd), pluck(c4))

  In this code, snd is still retained by the constructed closure, but it is set to `nil` by `unbind`, which returns the value of snd. Thus, the only surviving reference to the original value of snd is held by `trigger`, which frees samples immediately after computing them. No samples are retained in memory.

`snd-stoponzero(s)` [SAL]
`(snd-stoponzero s)` [LISP]
  This function is identical to `stop-on-zero`. You should use `stop-on-zero` instead.
`snd-trigger(s, closure)` [SAL]
`(snd-trigger s closure)` [LISP]
  This is a low-level support function for `trigger`. The closure takes a starting time and returns a `SOUND`. See `trigger` above for more details. Use `trigger` as described above and do not call this function directly.

### Using Open Sound Control

Open Sound Control (OSC) is a simple protocol for communicating music control parameters between software applications and across networks. For more information, see `http://www.cnmat.berkeley.edu/OpenSoundControl/`. The Nyquist implementation of Open Sound Control is simple: an array of floats can be set by OSC messages and read by Nyquist functions. That is about all there is to it.

The `slider` and `get-slider-value` functions, described above, can be used to access these values within Nyquist. Each of these functions can take a slider array index to specify which value to use. Since `make-slider` allocates slider indices starting at 10, it is recommended that you control sliders 0 through 9 via OSC. If you change a slider array value via OSC that is already controlled by a graphical slider in the NyquistIDE, the graphical slider will not be updated or synchronized to the OSC value. (And there is no current way to send OSC command to the NyquistIDE.)

Note: Open Sound Control must be enabled by calling `osc-enable(t)`. If this fails under Windows, see the installation instructions in `sys/win/README.txt` regarding `SystemRoot`.

`osc-enable(flag)` [SAL]
`(osc-enable flag)` [LISP]
  Enable or disable Open Sound Control. (See Section Using Open Sound Control.) Enabling creates a socket and a service that listens for UDP packets on port 7770. Currently, only two messages are accepted by Nyquist. The first is of the form `/slider` with an integer index and a floating point value. These set internal slider values accessed by the `slider` and `get-slider-value` functions. The second is of the form `/wii/orientation` with two floating point values. This message is a special case to support the DarwiinRemoteOsc program which can relay data from a Nintendo WiiMote device to Nyquist via OSC. The two orientation values control sliders 0 and 1. Disabling terminates the service (polling for messages) and closes the socket. The previous state of enablement is returned, e.g. if OSC is enabled and flag is nil, OSC is disabled and `T` (true) is returned because OSC was enabled at the time of the call. This function only exists if Nyquist is compiled with the compiler flag `OSC`. Otherwise, the function exists but always returns the symbol `DISABLED`. Warning: there is the potential for network-based attacks using OSC. It is tempting to add the ability to evaluate XLISP expressions sent via OSC, but this would create unlimited and unprotected access to OSC clients. For now, it is unlikely that an attacker could do more than manipulate slider values.

#### Sending Open Sound Control Messages

A variety of programs support OSC. The only OSC message interpreted by Nyquist has an address of `/slider`, and two parameters: an integer slider number and a float value, nominally from 0.0 to 1.0.

#### Python3 OSC Interface Demo

In `nyquist/demos/osc` are two programs that demonstrate controlling Nyquist sounds with Python in real time. Open `nyquist/demos/osc/getosc.sal` in the NyquistIDE and read the comments. There are several functions you can run or play while executing the Python program `nyquist/demos/osc/nyquistosc.py` from a command line (terminal) window.

#### Test Programs in C

Two small programs are included in the Nyquist distribution for sending OSC messages. (Both can be found in the same directory as the nyquist executable.) The first one, `osc-test-client` sends a sequence of messages that just cause slider 0 to ramp slowly up and down. If you run this on a command line, you can use "?" or "h" to get help information. There is an interactive mode that lets you send each OSC message by typing RETURN.

#### The ser-to-osc Program

The second program is `ser-to-osc`, a program that reads serial input (for example from a PIC-based microcontroller) and sends OSC messages. Run this command-line program from a shell (a terminal window under OS X or Linux; use the CMD program under Windows). You must name the serial input device on the command line, e.g. under OS X, you might run:

    ./ser-to-osc /dev/tty.usbserial-0000103D

(Note that the program name is preceded by “`./`". This tells the shell exactly where to find the executable program in case the current directory is not on the search path for executable programs.) Under Windows, you might run:

    ser-to-osc com4

(Note that you do not type “`./`” in front of a windows program.)

To use `ser-to-osc`, you will have to find the serial device. On the Macintosh and Linux, try the following:

    ls /dev/*usb*

This will list all serial devices with “usb” in their names. Probably, one will be a name similar to `/dev/tty.usbserial-0000103D`. The `ser-to-osc` program will echo data that it receives, so you should know if things are working correctly.

Under Windows, open Control Panel from the Start menu, and open the System control panel. Select the Hardware tab and click the Device Manager button. Look in the device list under Ports (COM & LPT). When you plug in your serial or USB device, you should see a new entry appear, e.g. `COM4`. This is the device name you need.

The format for the serial input is: any non-whitespace character(s), a slider number, a slider value, and a newline (control-j or ASCII 0x0A). These fields need to be separated by tabs or spaces. An optional carriage return (control-m or ASCII 0x0D) preceding the ASCII 0x0A is ignored. The slider number should be in decimal, and theh slider value is a decimal number from 0 to 255. This is scaled to the range 0.0 to 1.0 (so an input of 255 translates to 1.0).

There is a simple test program in `nyquist/lib/osc/osc-test.lsp` you can run to try out control with Open Sound Control. There are two examples in that file. One uses `snd-slider` to control the frequency of an oscillator. The other uses `get-slider-value` to control the pitch of grains in a granular synthesis process.

Previous Section | Next Section | Table of Contents | Index | Title Page