Add Harp SoundCard tutorial

docfx.json

@@ -106,6 +106,7 @@
     "xref": [
       "https://bonsai-rx.org/docs/xrefmap.yml",
       "https://bonsai-rx.org/gui/xrefmap.yml",
+      "https://bonsai-rx.org/numerics/xrefmap.yml",
       "https://horizongir.github.io/opencv.net/xrefmap.yml",
       "https://horizongir.github.io/ZedGraph/xrefmap.yml",
       "https://horizongir.github.io/opentk/xrefmap.yml",

tutorials/soundcard-playsound.md

@@ -0,0 +1,146 @@
+# Play Sound
+
+The [Harp SoundCard](https://github.com/harp-tech/device.soundcard) supports playback of waveforms stored in its onboard memory. It also includes an internal sine wave generator for pure tones. The following exercises demonstrate how to play these sounds in Bonsai.
+
+> [!WARNING]
+> When adding these operators to the workflow, make sure to use the device-specific versions, e.g. `Device (Harp.SoundCard)` instead of `Device (Harp)`. If correctly selected, the names of these operators in the workflow panel will change to reflect either the name of the device or the selected register/payload.
+
+## Prerequisites
+
+- Install the `Bonsai.Windows.Input` package from the Bonsai [package manager](https://bonsai-rx.org/docs/articles/packages.html).
+
+## Device pattern
+
+Set up the standard Harp [device pattern](../articles/operators.md#device-pattern) to initialize the device, log data, broadcast events, and send commands to the `SoundCard`.
+
+:::workflow
+![SoundCard Device Pattern](../workflows/soundcard-playsound-devicepattern.bonsai)
+:::
+
+- Insert a [`Device`] operator, set the `PortName` property to the communications port for the device.
+- Insert a [`DeviceDataWriter`] sink and set the `Path` property (e.g. `SoundCard.harp`). 
+   - This will save the data in the standard Harp logging format, which can be loaded with [`harp-python`](../articles/python.md).
+- Insert a [`PublishSubject`] operator and name it `SoundCard Events`.
+- Right-click the [`Device`] operator, select "Create Source (Bonsai.Harp.HarpMessage)" > "BehaviorSubject". 
+   - Name the generated [``BehaviourSubject`1``] [source subject](https://bonsai-rx.org/docs/articles/subjects.html#source-subjects) `SoundCard Commands`. 
+   - Connect it as input to the [`Device`] operator.
+
+## Exercise 1 - Play sound index
+
+Sounds can be played from the `SoundCard` onboard memory by using the [`PlaySoundOrFrequency`] register.
+
+:::workflow
+![Play Sound Index Keydown](../workflows/soundcard-playsound-indexkeydown.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`. 
+- Insert a [`CreateMessage`] operator to construct a [`HarpMessage`] command to send to the device and configure these properties:
+    - `Payload` - Select [`PlaySoundOrFrequencyPayload`] from the property dropdown menu.
+    - `PlaySoundOrFrequency` - Set this to the index of the sound you want to play from the `SoundCard` onboard memory (2-31).
+- Insert a [`MulticastSubject`] operator to send [`HarpMessage`] commands to named subjects, and configure the `Name` property to `SoundCard Commands`.
+
+Run the workflow and press the <kbd>A</kbd> key to play the sound. Sound duration is determined by the length of the stored waveform.
+
+You can replace [`KeyDown`] with other operators to trigger sound playback on other events in Bonsai.
+
+:::workflow
+![Play Sound Index Timer](../workflows/soundcard-playsound-indextimer.bonsai)
+:::
+
+- Replace the [`KeyDown`] source with a [`Timer`] source and set the `DueTime` property to 0.
+- Insert a [`SubscribeWhen`] operator after `SoundCard Commands`.
+- Insert a [`SubscribeSubject`] operator, configure the `Name` property to `SoundCard Events`, and connect it to [`SubscribeWhen`].
+
+> [!TIP]
+> The `SubscribeWhen` > `SoundCard Events` pattern is useful for ensuring that [`HarpMessage`] commands are only sent after the [`Device`] has been initialized. It relies on the `DumpRegisters` property being set to `True` in [`Device`]. Use it when needed, for instance, if sounds are being played at the start of the workflow.
+
+## Exercise 2 - Play pure tone
+
+The [`PlaySoundOrFrequency`] register can also be used to play pure tones using the internal sine wave generator.
+
+:::workflow
+![Play Sound Frequency](../workflows/soundcard-playsound-frequency.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`. 
+- Insert a [`CreateMessage`] operator, select [`PlaySoundOrFrequencyPayload`] for the `Payload` property, and set the `PlaySoundOrFrequency` property to the desired frequency in Hz (e.g. 1000).
+- Insert a [`MulticastSubject`] operator named `SoundCard Commands`.
+
+Unlike playback of sounds from the onboard memory, the pure tone will continue playing until it is terminated via the [`Stop`] register.
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `S`.
+- Insert a [`CreateMessage`] operator, select [`StopPayload`] for the `Payload` property, and set the `Stop` property to 1 (or any other value than 0).
+- Insert a [`MulticastSubject`] operator named `SoundCard Commands`.
+
+Run the workflow, press the <kbd>A</kbd> key to play the sound, and press the <kbd>S</kbd> key to stop playback.
+
+> [!WARNING]
+> The [`Stop`] register can only be used to stop playback from the internal sine wave generator, not sounds from the onboard memory.
+
+## Exercise 3 - Lower sound playback volume
+
+The [`PlaySoundOrFrequency`] register plays the sound at the amplitude of the stored waveform or at maximum amplitude for pure tones. To lower the volume, use the [`AttenuationAndPlaySoundOrFreq`] register instead to set the attenuation in 0.1 dB steps.
+
+:::workflow
+![Play Sound Attenuation](../workflows/soundcard-playsound-attenuation.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`. 
+- Insert a [`CreateMessage`] operator and select [`AttenuationAndPlaySoundOrFreqPayload`] for the `Payload` property.
+   - Click on the dialog button for the `AttenuationAndPlaySoundOrFreq` field in the property grid to open the member collection editor. Add three members:
+      - The sound index or pure tone frequency to be played (e.g. 2).
+      - The attenuation of the left channel (e.g. 200 = -20 dB).
+      - The attenuation of the right channel (e.g. 200 = -20 dB).
+- Insert a [`MulticastSubject`] operator named `SoundCard Commands`.
+
+Run the workflow and press the <kbd>A</kbd> key to play the sound at reduced volume.
+
+> [!TIP]
+> Pure tone playback must be stopped explicitly via the [`Stop`] register.
+
+## Exercise 4 - Trigger sound index playback with digital inputs
+
+The `SoundCard` also features digital input channels that can be configured to trigger sound index playback.
+
+:::workflow
+![Play Sound Digital Input](../workflows/soundcard-playsound-configureDI.bonsai)
+:::
+
+- Connect a TTL signal from another device to the digital input channel `DI0` (5 V tolerant) and `GND` on the `SoundCard`.
+- Insert a [`SubscribeSubject`] operator and configure the `Name` property to `SoundCard Events`.
+- Insert a [`Take`] combinator and set the `Count` property to 1.
+- Insert a [`CreateMessage`] operator, select [`ConfigureDI0Payload`] for the `Payload` property, and set the `ConfigureDI0` property to `StartSound`.
+- Insert a second [`CreateMessage`] operator on a new branch, select [`SoundIndexDI0Payload`] for the `Payload` property, and set the `SoundIndexDI0` property to the sound index for playback.
+- Combine both messages with a [`Merge`] combinator.
+- Insert a [`MulticastSubject`] operator named `SoundCard Commands`.
+
+Run the workflow and send the TTL signal from the other device to trigger sound playback.
+
+> [!WARNING]
+> Only sound index playback is supported currently.
+
+> [!TIP]
+> The `SoundCard Events` > `Take(1)` is another useful pattern for ensuring that configuration commands are sent as soon as the `SoundCard` has initialized. It also relies on the `DumpRegisters` property being set to `True` in the [`Device`] operator.
+
+<!--Reference Style Links -->
+[`AttenuationAndPlaySoundOrFreq`]: xref:Harp.SoundCard.AttenuationAndPlaySoundOrFreq
+[`AttenuationAndPlaySoundOrFreqPayload`]: xref:Harp.SoundCard.CreateAttenuationAndPlaySoundOrFreqPayload
+[``BehaviourSubject`1``]: xref:Bonsai.Reactive.BehaviorSubject
+[`ConfigureDI0Payload`]: xref:Harp.SoundCard.CreateConfigureDI0Payload
+[`SoundIndexDI0Payload`]: xref:Harp.SoundCard.CreateSoundIndexDI0Payload
+[`CreateMessage`]: xref:Harp.SoundCard.CreateMessage
+[`Device`]: xref:Harp.SoundCard.Device
+[`DeviceDataWriter`]: xref:Harp.SoundCard.DeviceDataWriter
+[`HarpMessage`]: xref:Bonsai.Harp.HarpMessage
+[`KeyDown`]: xref:Bonsai.Windows.Input.KeyDown
+[`Merge`]: xref:Bonsai.Reactive.Merge
+[`MulticastSubject`]: xref:Bonsai.Expressions.MulticastSubject
+[`PlaySoundOrFrequency`]: xref:Harp.SoundCard.PlaySoundOrFrequency
+[`PlaySoundOrFrequencyPayload`]: xref:Harp.SoundCard.CreatePlaySoundOrFrequencyPayload
+[`PublishSubject`]: xref:Bonsai.Reactive.PublishSubject
+[`Stop`]: xref:Harp.SoundCard.Stop
+[`StopPayload`]: xref:Harp.SoundCard.CreateStopPayload
+[`SubscribeSubject`]: xref:Bonsai.Expressions.SubscribeSubject
+[`SubscribeWhen`]: xref:Bonsai.Reactive.SubscribeWhen
+[`Take`]: xref:Bonsai.Reactive.Take
+[`Timer`]: xref:Bonsai.Reactive.Timer

tutorials/soundcard-setup.md

@@ -0,0 +1,38 @@
+# Getting Started
+
+The [Harp SoundCard](https://github.com/harp-tech/device.soundcard) is a high-performance audio device with two output channels using 24-bit DACs and a maximum sampling rate of 192 kHz.
+
+![Harp SoundCard](../images/device-soundcard-pcb.png){width=300}
+
+## Installation
+
+- Install the [FTDI D2XX driver](https://ftdichip.com/drivers/d2xx-drivers/).
+- Connect both `SoundCard` USB cables to the computer.
+- Install the WinUSB driver:
+   - Download and launch [Zadig](https://zadig.akeo.ie/).
+   - Select the "Harp Sound Card" from the list. If the device is not available, go to "Options" > "List All Devices".
+   - Select the "WinUSB" driver and click "Install Driver".
+- Install [Bonsai](https://bonsai-rx.org/docs/articles/installation.html).
+- Install the `Harp.SoundCard` package by searching for it in the [Bonsai package manager](https://bonsai-rx.org/docs/articles/packages.html).
+
+Waveforms can be generated and uploaded to the `SoundCard` in Bonsai. Optionally, you can use the [Harp SoundCard GUI](https://bitbucket.org/fchampalimaud/downloads/downloads/Harp_Sound_Card_v1.3.2.zip) as a standalone interface for waveform management. This requires the [LabVIEW runtime](https://bitbucket.org/fchampalimaud/downloads/downloads/Runtime-1.0.zip) to be installed first.
+
+## Connections
+
+![Harp SoundCard Connections](../images/device-soundcard-connection.jpg){width=450}
+
+*<small>Single channel connection diagram with Harp Audio Amplifier and attached speaker. Reproduced from [Silva et al. (2024)](https://doi.org/10.1016/j.ohx.2024.e00555). CC BY 4.0.</small>*
+
+**Amplifier** - The `SoundCard` requires an external amplifier. For high-fidelity applications, consider using the [Harp Audio Amplifier](https://github.com/harp-tech/peripheral.audioamp).
+
+**Speaker** - The choice of speaker depends on the amplifier. For the `Harp Audio Amplifier`, any speaker with an impedance from 4 to 8 ohms can be used. The XT25SC90-04 (Peerless by Tymphany) has been tested and has a good frequency response up to 80 kHz.
+
+## Testing the device
+
+:::workflow
+![SoundCard Hello World](../workflows/soundcard-helloworld.bonsai)
+:::
+
+- Hover over the workflow cell above, click the "Copy" icon in the top right, and paste the workflow into Bonsai.
+- Set the `PortName` property of the [`SoundCard`](xref:Harp.SoundCard.Device) operator to the communications port of the `SoundCard` (e.g. COM7).
+- Run the workflow. If the `SoundCard` is properly connected, you should hear a short tone.

tutorials/soundcard-uploadwaveform.md

@@ -0,0 +1,138 @@
+# Upload Waveform
+
+The `Harp.SoundCard` Bonsai package supports preload and update of sound waveforms during runtime. The following exercises will demonstrate how to generate and load various waveforms for upload to the `SoundCard` in Bonsai.
+
+## Prerequisites
+
+- Install the following packages from the Bonsai [package manager](https://bonsai-rx.org/docs/articles/packages.html):
+   - `Bonsai.WindowsInput`
+   - `Bonsai.Dsp`
+   - `Bonsai.Dsp.Design`
+   - `Bonsai.Audio`
+   - `Bonsai.Numerics`
+
+## UpdateSoundWaveform
+
+Uploading of waveforms is done with the [`UpdateSoundWaveform`] operator, which interfaces directly with the `SoundCard` onboard memory via `libusb` (not [`HarpMessage`] commands).
+
+:::workflow
+![UpdateSoundWaveform Operator](../workflows/soundcard-uploadwaveform-updatesoundwaveform.bonsai)
+:::
+
+For each of the following exercises, configure these properties for the [`UpdateSoundWaveform`] operator:
+- `DeviceIndex` - Set the index of the `SoundCard` device to update. If no index is specified, the first `SoundCard` will be used.
+- `SampleRate` - Set the sample rate used for playback. Only 96 kHz or 192 kHz are supported.
+- `SoundIndex` - Set the index of the sound to update. The `SoundCard` onboard memory is partitioned into 32 indices, of which index 2-31 can be used for storing waveforms.
+- `SoundName` - Set the name of the sound to be stored in the device. This field is optional.
+
+Continuous streaming of sample buffers is not supported, so the entire waveform must be sent during upload. Dynamic update of waveforms when another sound is playing is only possible for waveforms with a 96 kHz sample rate.
+
+## Waveform Requirements
+
+- `Type` - The [`UpdateSoundWaveform`] operator accepts `byte[]` and `Mat` type inputs. For these exercises, we will use `Mat`, where rows represent channels and columns represent samples.
+- `Bit Depth` - Waveforms must be encoded as 32-bit signed integers (`S32`).
+- `Channels` - Both mono and stereo waveforms are supported.
+- `Size` - Each sound index on the `SoundCard` memory can store an 8 MB file, corresponding to ~2 million samples. Mono waveforms are duplicated for dual-channel playback and occupy the same amount of storage space as stereo waveforms.
+
+Given these parameters, sounds are limited to a length of 10.922 s at a 96 kHz sample rate, or 5.461 s at 192 kHz. 
+
+## Exercise 1: Generate periodic waveform
+
+Pure tones and other periodic waveforms can be generated using a function generator.
+
+:::workflow
+![Upload Waveform Pure Tone](../workflows/soundcard-uploadwaveform-puretone.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`.
+- Insert a [`FunctionGenerator`] source and configure the following properties:
+   - `Amplitude` - Set this to the volume in 32-bit range (e.g. 2147483647 for max value).
+   - `BufferLength` - Set this to the number of samples, which will determine the sound length in combination with the sample rate (e.g. 192000 for a 2 s sound at 96 kHz sample rate).
+   - `Depth` - Set this to `S32` for 32-bit signed integers.
+   - `Frequency` - Set this to the frequency of the pure tone, in Hz (e.g. 2000).
+   - `SampleRate` - Set this to the sample rate, in Hz (e.g. 96000).
+   - `Waveform` - Set this to `Sine` for pure tones.
+- Insert a [`UpdateSoundWaveform`] operator and configure the relevant properties.
+
+Run the workflow and press the <kbd>A</kbd> key to upload the waveform. Test it out by playing the [sound index](soundcard-playsound.md#exercise-1---play-sound-index).
+
+> [!WARNING]
+> [`FunctionGenerator`] should always be connected to another trigger source to prevent continuous buffer generation. [`KeyDown`] can be replaced with other operators, such as a one-shot [`Timer`].
+
+## Exercise 2: Generate white noise waveform
+
+White noise can be generated by randomly sampling from a discrete uniform distribution.
+
+:::workflow
+![Upload Waveform White Noise](../workflows/soundcard-uploadwaveform-whitenoise.bonsai)
+:::
+
+- Insert a [`CreateDiscreteUniform`] source and set the `Lower` and `Upper` properties to the volume in 32-bit range.
+- Insert a [`SampleArray`] operator and set the `Count` property to the number of samples to draw.
+- Insert a [`Sample`] operator.
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`.
+- Insert a [`ConvertFromArray`] transform and set the `Depth` property to `S32`.
+- Insert a [`UpdateSoundWaveform`] operator and configure the relevant properties.
+
+Run the workflow and press the <kbd>A</kbd> key to upload the waveform. Test it out by playing the [sound index](soundcard-playsound.md#exercise-1---play-sound-index).
+
+## Exercise 3: Load waveform from WAV file
+
+Waveforms can be loaded from uncompressed WAV files, but will require bit depth conversion and scaling.
+
+:::workflow
+![Upload Waveform WAV File](../workflows/soundcard-uploadwaveform-wavfile.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`.
+- Insert an [`AudioReader`] source and configure the following properties:
+   - `BufferLength` - Set this to 0 to load the entire file into a single buffer (the `SampleRate` property is ignored in this instance).
+   - `FileName` - Set this to the file path.
+- Insert a [`ConvertScale`] transform and configure the following properties:
+   - `Depth` - Set this to `S32`.
+   - `Scale` - Set this to the scaling factor to fit the `S32` range. For a 1:1 conversion from a 16-bit WAV file, the scaling factor will be 65536.
+- Insert a [`UpdateSoundWaveform`] operator and configure the relevant properties.
+
+Run the workflow and press the <kbd>A</kbd> key to upload the waveform. Test it out by playing the [sound index](soundcard-playsound.md#exercise-1---play-sound-index).
+
+> [!WARNING]
+> 24-bit (or higher) and `WAVE_FORMAT_EXTENSIBLE` WAV files are not currently supported by [`AudioReader`]. 
+
+## Exercise 4: Load waveform from raw binary matrix file
+
+Waveforms can also be loaded from raw binary matrix files (`*.mat`).
+
+:::workflow
+![Upload Waveform MAT File](../workflows/soundcard-uploadwaveform-matfile.bonsai)
+:::
+
+- Insert a [`KeyDown`] source and set the `Filter` property to `A`.
+- Insert a [`MatrixReader`] source and configure the following properties:
+   - `BufferLength` - Set this to 0 to load the entire file into a single buffer (the `SampleRate` property is ignored in this instance).
+   - `ChannelCount` - Set this to the number of channels in the file (1 for mono or 2 for stereo).
+   - `Depth` - Set this to the bit depth of the input file (e.g. `S32`). 
+   - `Layout` - Set this to `RowMajor`.
+   - `Path` - Set this to the file path.
+- Insert a [`UpdateSoundWaveform`] operator and configure the relevant properties.
+
+Run the workflow and press the <kbd>A</kbd> key to upload the waveform. Test it out by playing the [sound index](soundcard-playsound.md#exercise-1---play-sound-index).
+
+> [!TIP]
+> Add a [`ConvertScale`] after [`MatrixReader`] if the `*.mat` bit depth is not `S32`.
+
+> [!TIP]
+> Stereo channel samples should be stored in the `*.mat` file in sequential order (not interleaved).
+
+<!--Reference Style Links -->
+[`AudioReader`]: xref:Bonsai.Audio.AudioReader
+[`ConvertScale`]: xref:Bonsai.Dsp.ConvertScale
+[`ConvertFromArray`]: xref:Bonsai.Dsp.ConvertFromArray
+[`CreateDiscreteUniform`]: xref:Bonsai.Numerics.Distributions.CreateDiscreteUniform
+[`HarpMessage`]: xref:Bonsai.Harp.HarpMessage
+[`FunctionGenerator`]: xref:Bonsai.Dsp.FunctionGenerator
+[`KeyDown`]: xref:Bonsai.Windows.Input.KeyDown
+[`MatrixReader`]: xref:Bonsai.Dsp.MatrixReader
+[`Sample`]: xref:Bonsai.Reactive.Sample
+[`SampleArray`]: xref:Bonsai.Numerics.Distributions.SampleArray
+[`Timer`]: xref:Bonsai.Reactive.Timer
+[`UpdateSoundWaveform`]: xref:Harp.SoundCard.UpdateSoundWaveform

tutorials/toc.yml

@@ -1,4 +1,8 @@
 - name: Harp Hobgoblin
 - href: hobgoblin-setup.md
 - href: hobgoblin-acquisition.md
-- href: hobgoblin-reaction.md
+- href: hobgoblin-reaction.md
+- name: Harp SoundCard
+- href: soundcard-setup.md
+- href: soundcard-playsound.md
+- href: soundcard-uploadwaveform.md