For a complete description of the project, click here.
From the thesis "Hardware/Software Co-design of an Optical Music Recognition System":
The objective of the project presented herein is to provide a co-design approach to the general optical music recognition (OMR) framework. The proposed system aims to be the first step towards a full-fledged hardware implementation of a music recognizer.
The OMR system is partitioned into two subsystems: software and hardware. Utilizing a combination of morphological and template matching techniques, the HOMeR software subsystem processes and converts simple monophonic, homophonic, and piano sheet music images to two numeric arrays corresponding to notes and beats through a graphical user interface. Subsequently, the hardware subsystem applies these array outputs into a custom synthesizer core written for the Zybo Z7 development board, which produces an organ-like audible equivalent of the initial input images through the onboard audio codec.
HOMeR was able to process sheet music images of different types and sizes with high accuracy and fast run times, both for pre- and post- correction.
- Software
- Hardware
- Zybo Z7: Zynq-7000 ARM/FPGA SoC Development Board (Z7-20 variant)
- Speaker with an AUX connection
- Download the required software.
- Clone the repository.
- Open the
HOMeR.mlappwithin theHOMeRfolder. Make sure that the current MATLAB workspace is the same folder. - Select sheet music image(s) to be read, then click on the Read Score button.
- Once the processing is complete, open Xilinx SDK. When prompted for the workspace, select the
OrganNotes.sdkfolder. - Connect the Zybo board to the computer using a micro-USB cable. Insert one end of an AUX cord to the HPH OUT port of the board, and the other to the speaker.
- Program the FPGA.
- Click on Run As > Launch On Hardware (GDB).
- Push the BTN1 button on the board to listen to the audio version of the images.
If the Synthesizer IP core is to be modified, follow these instructions:
- Delete the contents of the
OrganNotes.sdkfolder underHOMeR. - Open Vivado 2019.1.
- Using the tcl console, type the following:
cd <change to extracted_folder>/Synthesizer/
source ./OrganNotes.tcl- Once the project is created, modify the IP through the Block Design section.
- Create a new HDL wrapper for the block design, then generate the bitstream.
- Go to File > Export > Export Hardware. Make sure that the "Include Bitstream" box is marked, and the "Export to" location is the
OrganNotes.sdkfolder. - Go to File > Launch SDK. Change both "Exported location" and "Workspace" to
OrganNotes.sdk, then click OK. - Once the SDK is launched, go to File > New > Application Project.
- Fill up the form with the following details:
| Field | Value |
|---|---|
| Project Name | Synthesizer |
| OS Platform | Standalone |
| Hardware Platform | test_wrapper_hw_platform_0 |
| Processor | ps7_cortex a9_0 |
| Language | C |
| Board Support Package | Create New |
- Click on Next, select the Empty Application template, then click Finish.
- Copy the contents of
HOMeR/srcintoOrganNotes.sdk/Synthesizer/src. - Run the HOMeR app as described in the Setup section.
- Launch On Hardware must be performed after every image read.
- The board need not to be re-programmed every read, unless the audio produces undesirable results.
- Synthesizer: The tune files being read by the synthesizer were hardcoded to test its capabilities before final integration.
- HOMeR App: The following videos demonstrate various test cases for the app.
- Single image:
- Multiple images:
System Setup
Block Diagram
HOMeR Sample Output
