Long time no see!

So it's been a while since I've posted a new build. But I assure you, I haven't been sitting on my thumbs! So I'm gonna talk a bit about what I've been doing and the challenges involved.

Onset detection

"Onset refers to the beginning of a musical note or other sound." - Wikipedia

I started by falling further down the rabbithole that is audio analysis. Although it's hard to tell whether the end result yielded any improvement. Detecting onsets as a gameplay mechanic is a bit risky. Finding a "one size fits all"-solution is very hard and time-consuming, as music of different genres have wildly different attributes. Electronic music is arguably the easiest to cater to, as sounds are usually pretty distinctive and easier to isolate than eg. analog music.

Detecting onsets is just one part of the equation, however. It has to be tuned to whatever you want to use it for.  When you use it as an input for a core game mechanic, it's obviously important to get it to feel right. In Audio.Flow, onsets are used to generate blocks, which you pick up as you fly through the level. Making just a beat detector for that wouldn't suffice, as there are many songs with parts completely void of beats. As the game is fully procedural, I have no control over what kind of songs the player would like to play.  But it's a fair guess that any song would have some sort of instrument and/or a beat. 

My current solution is to divide the frequency spectrum into 3 bins - Low, Mid and High. A combination of Low+High is used for beats. Low will pick up kickdrums, while high (very high part of the spectrum) will pick up snares/hi-hats/etc. Mid is used to pick up instruments. That's the idea, anyway. Separating instruments is a whole other can of worms which I haven't yet attempted, nor do I plan to do so in the near future. 

I also made myself a tool to visualize the results:

God knows how many hours I've spent looking at these graphs. I have a set of songs I use to test it out with. Everytime I get something working in one song, another song gets broken. And then, when I think I've figured out something that works for most songs, I go to test it in the actual game and it doesn't feel right. Then its back to the drawing board. It's a painstaking process, but I learn something new everytime I go through it.

I could (or maybe should) have opted for a simpler/more consistent way of generating blocks. But I really do want this to work, and besides there's a lot more I could use onsets for if I can get acceptable results.

For anyone interested in the topic of onset / transient detection, here are some resources:

- https://www.badlogicgames.com/wordpress/?cat=18
- https://www.parallelcube.com/2018/03/30/beat-detection-algorithm/
- http://archive.gamedev.net/archive/reference/programming/features/beatdetection/
- https://medium.com/giant-scam/algorithmic-beat-mapping-in-unity-preprocessed-audio-analysis-d41c339c135a

The second part of the challenge is deciding which lane to put a block. I have another analyzer which detects changes in a song and attempts to split songs into sections. Each section will be given a set of attributes, such as whether it's a melodic section or a rhythmic section. 

This is again used as input for proc gen. It can be used to decide whether I should use Mid frequencies or Low+High to generate blocks. For melodic parts, I use a Spectral Centroid to decide whether or not the change the lane (Thanks to Evan for the tip). At some point I hope to use actual pitch detection, but that's just too out of scope for the early access release. For beats, I place the Low-onsets in the left/right lane, while the High-onsets goes in the middle. The effect is that you have to press a button on the kicks, which feels very right. The problem is, as always, that occasionally there will be onsets picked up that doesn't feel like they belong.

Tunnels and new gameplay

I also went ahead and completely redesigned the procedural generation of the road mesh. Previously it was just a simple quad strip (x3) which used the smoothed magnitude of a frequency band as a "heightmap". After watching this excellent GDC talk by @FreyaHolmér, I was inspired to make a Spline-based solution instead. With this new power, I opted to make tunnels by extruding a circle shape along a spline instead of just placing rings at certain positions.

As I was already experimenting with new gameplay mechanics, I tried to use this for something useful. At first I started making a prototype inspired by this really cool game: Aaero. I made a path that rotates around the tunnel, where you get points for hitting it. The rotation of the path would be based on variation in magnitude of the song. Left and right keys would rotate the player around the tunnel wall. While it worked pretty well, it didn't combine very well with the block-based gameplay. Changing from road to tunnel and back felt very jarring. But I might play around with it more for a new game mode, further down the line.

Instead I let the roads flow through the tunnels, and worked in some of the previous attempts at new gameplay mechanics. In this mode you alternate between collecting blocks on the roads, while dodging lasers and shooting red thingies in the tunnels. The thingies can drop shields, which will protect you against lasers (once). The gameplay felt a lot more coherent, while it adds a much needed gameplay variation to the game. Instead of using onsets here, I used the BPM of the song to place objects. You can see the result here:

It's not finished by any means, and I won't be releasing a new build before it is. I'm still playing around with ideas and seeing how they feel. If you have any ideas, feel free to let me know!