First turn space into musical parameters
MelodyCanvas interprets the horizontal axis as time, the vertical axis as pitch, and line color as instrument. The canvas is sampled on a 4/4, sixteenth-note grid, so users do not need to hit an exact note position for the system to produce ordered events. Adding measures preserves the absolute timing of existing lines; expanding the pitch range preserves their original pitches.
Quantization makes freehand drawing play back reliably, but it also loses some gesture detail. The product does not pretend that every pixel maps to infinitely precise music; it is positioned as an approachable creative tool instead.
A continuous line should not become a string of abrupt short notes
If every time cell plays a sample independently, diagonal lines can sound like rapid retriggering and long horizontal lines may keep restarting. The implementation fills the time cells crossed by the same continuous line and applies short crossfades between adjacent sounds, keeping pitch changes continuous without cutting the previous note off too early.
Recorded samples cannot cover every pitch or every device format. The system applies small pitch shifts from nearby samples and provides both Ogg Opus and AAC/M4A so the browser can choose a playable format. If an individual sample fails, it falls back to locally synthesized audio instead of stopping the whole piece.
Why page timers alone are not enough
Page animation usually follows display refresh, while background tabs, mobile power saving, and a busy main thread can all delay timers. The audio system has its own continuous clock, which is better suited to scheduling when a sound should begin in the future. MelodyCanvas uses rolling look-ahead scheduling: it schedules the next few seconds of audio, then periodically adds later events.
The playback cursor also reads audio output time instead of running a separate page stopwatch. This keeps the display aligned with what the user actually hears and reduces visual lead caused by mobile audio-device buffering.
For dense works, protect the audio first and reduce decorative cost second
Many lines can trigger many sounds at the same time. The mix bus must limit peaks and compress excessive loudness to prevent summed clipping. When a piece exceeds 400 lines, the page reduces nonessential animation and visual refresh frequency, but it does not delete notes or cut sustained sounds: visual performance may degrade, but the work itself must not change silently.
| Sources of load | Processing strategy |
|---|---|
| Unstable page timing | Use the audio clock and schedule ahead |
| Repeated onsets along a continuous line | Fill time cells and use short crossfades |
| Many sounds overlapping at once | Peak protection and bus compression |
| Lower drawing frame rate on mobile | Reduce nonessential visual updates without dropping musical events |