HexoSynth 2022 - Devlog #7 - The DSP JIT Compiler
The HexoSynth modular synthesizer (programmed in Rust) was extended with a proof of concept JIT compiler for DSP.
If you don't want to dig through the detailed log:
TLDR: Skip to the Devlog 7 Conclusion Section
Detailed Log
This is the day by day log of my progress. If this is too verbose for your interest, please skip down to the Devlog 7 Conclusion Section.
2022-07-27
On this day I published the Devlog #6 article and refined the oscilloscope more.
HexoSynth: I finished work on the Scope today. The offset and gain parameters are all wired up now too:
Also the trigger threshold is now displayed in the oscilloscope:
HexoDSP: At the end of the day the Scope node was officially finished and fully covered by automated tests in the test suite. This meant for me, that I could start focusing on the next big feature.
As I got a few days of vacation ahead and a few quiet evenings for myself, I decided to look into WBlockDSP integration into HexoSynth. After seeing another DSP JIT project with
2022-07-28
WBlockDSP: I've invested this day into WBlockDSP (which I later partially refactored into synfx-dsp-jit ) and made a lot of progress in the DSP node JIT API. The API allows constructing an AST (Abstract Syntax Tree) as intermediate DSP code representation. That AST can then be JIT compiled into one function that can be executed at runtime. The API also allows specifying a library of DSP node types that the AST can refer to and construct. The API also takes care of persisting state properly across multiple recompilations of code.
Those features will all allow you to edit the DSP function at runtime in HexoSynth and immediately see and hear the result.
2022-07-29
WBlockDSP: Finished the JIT API today. Still some automated tests to implement and some macro magic for defining DSP nodes. And lots of little things like documentation to do. But the core is implemented now.
HexoSynth: A few hours later the integration of WBlockDSP with HexoSynth/HexoDSP was done:
2022-07-30
HexoSynth: I've integrated the block code editor veeeeery rudimentary. That is not where it will stay, it will be toggle-able. And the context menus don't have a proper layout/styling yet.
But that is roughly the direction it is going to right now.
I'll keep the feature set minimal for now, and also put primary focus back to the hexagonal DSP nodes. This little JIT DSP thing is really just a little feature in the whole thing. The point is to combine low level DSP elements for little experiments for now. Of course there are tons of features you could add. like more feedback of values in the DSP code, and all that is not impossible to add. also more convenient editing or making "subroutines" are all nice features, but will not be included for now. eventually they might, depends on user interest (eg. mine).
I'll however factor out the whole JIT stuff into the synfx-dsp-jit crate,
and I will factor out some of the generic DSP utilities I've accumulated in HexoDSP into synfx-dsp.
For now I really just want to establish this feature. Maybe I extend it in future, maybe someone else
wants to extend it. Or maybe takes the building blocks in synfx-dsp-jit and builds an alternative
tool with it.
2022-07-31
synfx-dsp-jit: I've spent some time on factoring out the JIT into it's own crate. I've then documented that crate and published it: synfx-dsp-jit on crates.io. Here is an example from it's documentation. You can formulate simple little DSP code (mostly arithmetics) and call out to functions defined in Rust (not shown in the example though).
use synfx_dsp_jit::build::*;
use synfx_dsp_jit::instant_compile_ast;
let (ctx, mut fun) = instant_compile_ast(
op_add(literal(11.0), var("in1"))
).expect("No compile error");
fun.init(44100.0, None); // Sample rate and optional previous DSPFunction
let (sig1, sig2, res) = fun.exec_2in_2out(31.0, 10.0);
// The result should be 11.0 + 31.0 == 42.0
assert!((res - 42.0).abs() < 0.0001);
// Yes, unfortunately you need to explicitly free this.
// Because DSPFunction might be handed around to other threads.
ctx.borrow_mut().free();
I hope someone else finds it useful. May it be at least as example or starting point for their own projects.
synfx-dsp-jit: I've published a new version of the synfx-dsp-jit crate (version 0.5.2), I discovered that I need to handle multiple return values somehow. And this is what I came up with:
HexoDSP: After the work on synfx-dsp-jit I've started to work on the block language compiler.
There is not much to show, I'm still right in the middle, but the first lowering step from
the block language towards the JIT AST has been done. Next up I'm trying to write the last
compilation step and output JIT code finally. Afterwards I'll start and refine the language
itself and nail down the syntax and semantics more precisely.
2022-08-01
HexoDSP: Today I started the final translation step of the BlockCode to JIT compiler. Already got the first values being generated in the DSP backend of HexoDSP. Unfortunately there is still a bug somewhere in the test itself.
synfx-dsp-jit: Worked a bit on the DSPNodeType trait. Because I prioritize documentation
of things quite high, I added a few methods to that trait to help building a documented
standard library of DSP nodes. The port names are also very helpful for any compiler
that wants to target synfx-dsp-jit, because referring to ports by their index is a
maintenance horror. If you have names, you can grep the source code easier.
Before it was this:
stateful_dsp_node_type! {
AccumNodeType, AccumNodeState => process_accum_nod "accum" "vvSr"
}
It will not tell you anything about how exactly to use it.
Compare it with this, which should be immediately more clear what to do
with the "accum" node:
stateful_dsp_node_type! {
AccumNodeType, AccumNodeState => process_accum_nod "accum" "vvSr"
doc
"This is a simple accumulator. It sums up it's input and returns it. \
You can reset it's state if you pass a value >= 0.5 into 'reset'."
inputs
0 "input"
1 "reset"
outputs
0 "sum"
}
2022-08-02
HexoDSP/synfx-dsp-jit: More work on the BlockCode to JIT compiler in HexoDSP.
Also added a few minor features to synfx-dsp-jit. Such as access to the persistent
variables in the DSPFunction.
HexoSynth: Finished the very bare minimum of the JIT compiler now, and wanted to finally see that HexoSynth can generate audio data from the JIT compiled function. I still had to finish the text entry widget WLambda API for HexoSynth. Which was quickly done. Not soon after I was able to finally go through the complete technology stack and generate audio samples from the BlockCode visual programming language in the HexoSynth frontend. It does not look spectacular, but if you look closely, you can see the signals in the oscilloscope change.
YouTube Sound Demos
More playing around with the sample player and the new oscilloscope:
Devlog 7 Conclusion
The oscilloscope was finished and I love it. I'm glad I invested time into developing it. I will also benefit from that a lot once I get back to developing more DSP nodes/modules for HexoDSP/HexoSynth.
With the displayed time range shortened:
And in it's minimized form:
I could not resist the temptation that I've been pondering over the last week: The visual JIT compiled DSP programming language I named WBlockDSP. The feature set will be limited. I don't want to develop a full blown Pure Data or Super Collider alternative. I limit the scope to be a supplemental programmable DSP node for HexoSynth. A little escape hatch to implement your own formulas, to write interesting control signal modulation for instance. Or try out some simple wave shaping algorithms.
I put up that scope limitation purely because I don't have the time or resources to further explore the possibility of a more general purpose (visual) audio programming language developed in Rust. Feel free to contact me if you feel inspired. If you are interested and want to contribute or try to develop this idea in a different direction (maybe a more classical visual programming language with wires? Maybe a text based audio programming language like SuperCollider or Faust?).
The inspiration for the WBlockDSP programmable DSP nodes for HexoSynth came from the DROID - Universal CV Processor Eurorack synthesizer module. Those are basically programmable Eurorack modules. You could customize their functions using files on an SD card in some INI file format.
My estimations are, that it will take me another 2-3 weeks to stabilize WBlockDSP this feature so far that I can leave it alone for a while. It probably won't be finished anytime, but I hope this lays a foundation for further experimentation. Maybe it's a dead end, or maybe I will expand on that. In any case, it lead to the release of a new crate: synfx-dsp-jit - a specialized JIT compiler for digital (audio) signal processing for Rust. Maybe it helps, maybe at least as example on how to use the Cranelift JIT compiler.
All that means that the roadmap of last week is still mostly unchanged:
- Stabilize WBlockDSP.
- Add more automated test cases for the UI workflow.
- Rewrite the online help.
- Add inserting DSP chains that are to be pre-defined. And also inserting random DSP chains for a more explorative/experimental workflow.
- Finish the nih-plug integration. That means a better integration of HexoSynth as VST3/CLAP plugin into your favorite DAW.
- Add back editing CV widgets.
Contact
In case you find this project interesting or have questions,
you can reach me via Discord these days, check out the #hexosynth channel
in the Rust Audio Discord. Optionally I'm also online
on IRC (via Matrix) in the Linux Audio Developer channel #lad (nickname 'wct')
on Libera.Chat: irc:#lad@irc.libera.chat.
