First pass parsing MIDI

livearmada,
Sample MIDI
Sample MIDI

So. What is this MIDI stuff anyways? I touched on it while I said specifications can be fun and it’s about time I dive further into this rather old technology.

A communications protocol for musical instruments

At the core the title says it all. It’s a serial protocol that can be driven by a UART at 31250 baud.

For a remote controller, such as a simple instrument; you’d probably need only NOTE_ON and NOTE_OFF. These two commands happen to take two additional parameters. Those are in order to determine which note to play, and at what velocity.

If you want to make your own little MIDI controller, there are plenty of things you can google but I think “Arduino Midi Controller” will get you very far! Good luck, there’s a lot of fun stuff ahead.

A file format used to store sequential events for triggering

Believe me. I had a couple of hours online looking at various attempts to explain the MIDI file format. While it really is simple, there seems to be no good explanations for it. So in order to understand it, I made a well known test MIDI file. You can see this file in the topmost image, and I got it done with the help of three or four resources. Linked here for your reference:

Now, let’s have a look at my red marker handywork:

Hand parsed MIDI
Hand parsing MIDI

Nothing makes much sense from that image, I get that. Though, selfish as I am this post is mainly intended for myself in one or two months time. As I’m probably dumber then, than you are now – let me try to make sense of all this!

File format; MIDI 0

Header chunk

The header is always the first 14 bytes.

[M T r d L1 L2 L3 L4 F1 F2 T1 T2]

  • Lx is length, always 0 0 0 6
  • Fx is format, one of 00, 01 or 02
  • Tx is time division, midi ticks per quarter note.
    • division = (T1 << 8) | T2

After the header comes the track chunk (for MIDI 0 this is always 1 track)

The track chunk and its data

[M T r k  { variable_length, event }, … ]

The variable length is a way to pack integers that are more than 127 into several bytes. There is just a few things to note. All the bytes except the least significant one, must have its most significant bit set. The least significant byte cannot have the most significant bit set. This means, that a time of variable_length = 0x83 0x8c 0x08 will become:

time = (0x83 & 0x7F) << 14 | (0x8c & 0x7F) << 7 | 0x08

We use to logical and with 0x7F or 0b01111111 to get rid of that most significant bit. Then we must shift by 7 because there is not 8th bit of information in these variable length values.

(currentByte & 0x80) == 0x80 means we need to truncate away the highest bit, store this value shifted by 7 and continue on until we reach a byte which does not satisfy this test.

0 is a perfectly valid length as a variable length entry.

These events can be divided into midi events, meta events and sysex events. Let’s assume you’ve read one byte at the beginning of a track chunk data entry. You already know the length of the entire data string

I will sort them by their identifier, then tell you how many bytes of information they are made of.

  • Midi Event (<= 0xEF)
    • Beginning with  0xC0, 0xD0 -> read one more byte of data
    • Beginning with 0x80, 0x90, 0xA0, 0xB0, 0xE0 -> read two more bytes of data
  • Meta Event (0xFF)
    • Parse through the variable length -> read in length bytes
  • Sysex Event (0xF0 || 0xF7)
    • Parse through the variable length -> read in length bytes

MIDI ticks and MIDI clock

Let’s say we have a midi file that’s made with 60 BMP and we get the following events:

00 ff 51 03 0f 42 40

  • This message has a delta tick == 00
  • We know it’s a meta event because of the ff
  • The next byte 51 indicates a set tempo event
  • It has a payload length of 03
  • 0x0f4240 = 1000000 microseconds per quarter note

00 ff 58 04 04 02 18 04

  • This message has a delta tick == 00
  • We know it’s a meta event because of the ff
  • The next byte 58 indicates a time signature event
  • It has a payload length of 04
  • It’s time signature numerator is 04
  • It’s time signature denominator is 2 ^ 02
  • It has 0x18 = 24 MIDI clocks per metronome tick
  • It has 04 1/32 notes per 24 MIDI clocks

MIDI tick calculation

From the MThd we get duration. Ticks per quarter note. For the 60 BPM example, you’d typically get 0x03c0 which is 960 decimal.

The midi ticks occur every MicrosecondsPerQuarterNote / TicksPerQuarterNote. For us, this would mean:

tickEveryMicroseconds = 1000000 / 960;

Approximately 1041.7 microseconds per MIDI tick

BPM calculation

BPM = (60000000.0f / (float)MicrosecondsPerQuarterNote) * ((float)TimeSignatureDenominator / (float)TimeSignatureNumerator);
BPM = (60000000.0f / 1000000.0f) * (4.0f / 4.0f);

BPM = 60

 

C++ code example

This should get you somewhere. If not I’ll try to keep my Gist updated:

#include <iostream> // std::cout
#include <fstream> // std::ifstream
#include <chrono>
#include <thread>
#define MIDI_DEBUG(s) (s);
using namespace std;
//expect total size and current size to be available
class fileinfo {
public:
uint32_t currentPosition;
uint32_t fileSize;
};
namespace MidiHelper {
typedef uint8_t(*getchar)(fileinfo*);
};
class MidiEvent {
public:
static const uint8_t META_PREFIX = 0xFF;
static const uint8_t SYSEX_PREFIX = 0xF0;
static const uint8_t SYSEX_SUFFIX = 0xF7;
void reset() {
deltaTick = 0;
eventType = EventType::NONE_EVENT;
dataLength = 0;
for (int i = 0; i < MAX_DATA_LENGTH; i++) { data[i] = 0; }
isConsumed = false;
isDone = false;
hasDeltaTick = false;
messageLength = 0;
metaQualifier = 0;
}
uint8_t wantChar() {
return !isDone;
}
void parse(MidiHelper::getchar getChar, fileinfo *info) {
MIDI_DEBUG(cout << "PARSE MESSAGE" << endl);
uint8_t ch = 0;
deltaTick = getVarLen(getChar, info);
midiId = getChar(info);
if (midiId == SYSEX_PREFIX || midiId == SYSEX_SUFFIX) {
eventType = EventType::SYSEX_EVENT;
dataLength = getVarLen(getChar, info);
for (int i = 0; i < dataLength; i++) {
data[i] = getChar(info);
}
MIDI_DEBUG(cout << "\t[^sysex message]" << endl);
}
else if (midiId == META_PREFIX) {
eventType = EventType::META_EVENT;
ch = getChar(info); //purge meta qualifier
for (int i = 0; i < META_TYPES_LENGTH; i++) {
if (ch == META_TYPES[i]) {
metaQualifier = i;
}
}
data[0] = ch;
dataLength = getVarLen(getChar, info);
for (int i = 0; i < dataLength; i++) {
data[1 + i] = getChar(info);
}
dataLength++; //account for meta type
MIDI_DEBUG(cout << "\t[^meta message]" << endl);
}
else {
eventType = EventType::MIDI_EVENT;
data[0] = midiId;
if (isEventType(MidiEventType::PC))
{
dataLength = 2;
data[1] = getChar(info);
MIDI_DEBUG(cout << "\t[^midi program change message]" << endl);
}
else if (isEventType(MidiEventType::AFTERTOUCH_PRESSURE))
{
dataLength = 2;
data[1] = getChar(info);
MIDI_DEBUG(cout << "\t[^midi pressure aftertouch message]" << endl);
}
else if (isEventType(MidiEventType::CC))
{
dataLength = 3;
data[1] = getChar(info);
data[2] = getChar(info);
MIDI_DEBUG(cout << "\t[^midi control change message]" << endl);
}
else {
dataLength = 2;
data[1] = getChar(info);
if (isEventType(MidiEventType::NOTE_OFF)
|| isEventType(MidiEventType::NOTE_ON)
|| isEventType(MidiEventType::AFTERTOUCH)) {
dataLength = 3;
data[2] = getChar(info);
MIDI_DEBUG(cout << "\t[^midi message 3]" << endl);
}
else {
MIDI_DEBUG(cout << "\t[^midi message 2]" << endl);
}
}
}
isDone = true;
}
/// utility for clients
static uint32_t getVariableLength(uint8_t *arr, uint8_t idx) {
uint8_t ch = 0;
uint32_t len = 0;
while (true) {
ch = arr[idx++];
if (ch & 0x80) {
len = (len << 7) + (ch & 0x7F);
}
else {
if (len > 0) { len = len << 7; }
len += ch;
break;
}
}
return len;
}
bool isEventType(uint8_t eventTypeSpecifier) {
return (midiId & eventTypeSpecifier) == eventTypeSpecifier;
}
enum EventType {
NONE_EVENT,
MIDI_EVENT,
META_EVENT,
SYSEX_EVENT
};
enum MidiEventType {
NOTE_OFF = 0x80,
NOTE_ON = 0x90,
AFTERTOUCH = 0xA0,
CC = 0xB0,
PC = 0xC0,
AFTERTOUCH_PRESSURE = 0xD0,
PITCH_CHANGE = 0xE0
};
enum MetaType {
SEQUENCE = 0x00,
TEXT = 0x01,
COPYRIGHT = 0x02,
NAME = 0x03,
LYRIC = 0x05,
MARKER = 0x06,
CUE = 0x07,
CHANNEL_PREFIX = 0x20,
EOT = 0x2F,
SET_TEMPO = 0x51,
SMTPE = 0x54,
TIME_SIGNATURE = 0x58,
KEY_SIGNATURE = 0x59,
SEQ_SPEC_META_EVENT = 0x7F
};
static const uint8_t META_TYPES_LENGTH = 15;
static const uint8_t META_TYPES[META_TYPES_LENGTH];
static const uint8_t MAX_DATA_LENGTH = 128;
uint8_t midiId;
uint8_t metaQualifier;
uint8_t data[MAX_DATA_LENGTH]; //figure out a max data length
uint8_t dataLength;
uint32_t deltaTick;
EventType eventType;
uint8_t isConsumed;
uint8_t isDone;
uint8_t hasDeltaTick;
uint32_t messageLength;
uint32_t emitAtTick;
private:
uint32_t getVarLen(MidiHelper::getchar getChar, fileinfo *info) {
uint8_t ch = 0;
uint32_t len = 0;
while (true) {
ch = getChar(info);
if (ch & 0x80) {
len = (len << 7) + (ch & 0x7F);
}
else {
if (len > 0) { len = len << 7; }
len += ch;
break;
}
}
return len;
}
};
const uint8_t MidiEvent::META_TYPES[META_TYPES_LENGTH] = {
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x20, 0x2F, 0x51, 0x54, 0x58, 0x59, 0x7F
};
class MidiParser {
public:
MidiParser(fileinfo *finfo, MidiHelper::getchar getter) : fileInfo(finfo), getChar(getter) {
lastEventTick = 0;
totalTicks = 0;
firstParse = true;
BPM = 0;
microsecondsPerQuarterNote = 0;
ticksPerQuarterNote = 0;
microsecondsPerTick = 0;
timeSignatureNumerator = 0;
timeSignatureDenominator = 0;
currentEvent.reset();
}
void parseMThd() {
uint8_t mthd[14];
for (int i = 0; i < 14; i++) {
mthd[i] = getChar(fileInfo);
}
format = (uint8_t)mthd[8] << 8 | mthd[9];
tracks = (uint8_t)mthd[10] << 8 | mthd[11];
ticksPerQuarterNote = (uint32_t)mthd[12] << 8 | mthd[13]; //8 here?
}
void parseMTrk() {
uint8_t mtrd[8];
for (int i = 0; i < 8; i++) {
mtrd[i] = getChar(fileInfo);
}
trackLength = (((uint8_t)mtrd[4] << 24)
| ((uint8_t)mtrd[5] << 16)
| ((uint8_t)mtrd[6] << 8)
| ((uint8_t)mtrd[7]));
}
MidiEvent *getNextEvent(uint32_t deltaTicks) {
totalTicks += deltaTicks;
if (firstParse) {
parseMThd();
parseMTrk();
MIDI_DEBUG(cout << "ticksPerQuarterNote: " << ticksPerQuarterNote << endl);
MIDI_DEBUG(cout << "tracklen: " << (int)trackLength << endl);
firstParse = false;
currentEvent.isConsumed = true; //force first parse
}
if (currentEvent.isConsumed) {
currentEvent.reset();
/// stream in a frame
while (currentEvent.wantChar()) {
currentEvent.parse(getChar, fileInfo);
}
if (currentEvent.eventType == MidiEvent::EventType::META_EVENT && currentEvent.data[0] == 0x51) {
hasTempo = true;
microsecondsPerQuarterNote = (uint32_t)((uint32_t)currentEvent.data[1] << 16) | ((uint32_t)currentEvent.data[2] << 8) | ((uint32_t)currentEvent.data[3]);
calculateTime();
}
if (currentEvent.eventType == MidiEvent::EventType::META_EVENT && currentEvent.data[0] == 0x58) {
hasTimeSign = true;
timeSignatureNumerator = currentEvent.data[1];
timeSignatureDenominator = pow(2, currentEvent.data[2]);
calculateTime();
}
currentEvent.emitAtTick = totalTicks + currentEvent.deltaTick;
//return &currentEvent;
}
if (currentEvent.emitAtTick <= totalTicks) {
return &currentEvent;
}
return 0;
}
uint8_t format;
uint8_t tracks;
uint32_t trackLength;
uint32_t totalTicks;
uint8_t timeSignatureNumerator;
uint8_t timeSignatureDenominator;
uint32_t microsecondsPerQuarterNote;
uint32_t ticksPerQuarterNote;
float BPM;
float microsecondsPerTick;
private:
void calculateTime() {
if (hasTimeSign && hasTempo) {
microsecondsPerTick = (float)microsecondsPerQuarterNote / ticksPerQuarterNote;
const float kOneMinuteInMicroseconds = 60000000;
BPM = (kOneMinuteInMicroseconds / (float)microsecondsPerQuarterNote) * ((float)timeSignatureDenominator / (float)timeSignatureNumerator);
MIDI_DEBUG(cout << "BPM: " << BPM << " muS/tick " << microsecondsPerTick << endl);
}
}
MidiEvent currentEvent;
fileinfo *fileInfo;
MidiHelper::getchar getChar;
uint32_t lastEventTick;
uint8_t frame[4];
uint8_t firstParse;
bool hasTimeSign = false;
bool hasTempo = false;
};
/////
char * buffer = 0;
fileinfo info;
uint8_t getCharCallback(fileinfo* info) {
if (info->currentPosition < info->fileSize) {
uint8_t ch = buffer[info->currentPosition++];
cout << "\t" << std::hex << (int)ch << endl;
return ch;
}
else 0; //should never happen, don't use readlength 0
}
void printEventType(MidiEvent *midiEvent) {
switch (midiEvent->eventType) {
case MidiEvent::EventType::META_EVENT:
cout << "META_EVENT";
if (midiEvent->data[0] == MidiEvent::MetaType::MARKER) {
cout << ": ";
for (int i = 1; i < midiEvent->dataLength; i++) {
cout << (char)midiEvent->data[i];
}
}
cout << endl;
break;
case MidiEvent::EventType::MIDI_EVENT:
cout << "MIDI_EVENT" << endl;
cout << '\a';
break;
case MidiEvent::EventType::SYSEX_EVENT:
cout << "SYSEX_EVENT" << endl;
break;
}
}
int main() {
Beep(440, 330);
int length = 0;
std::ifstream is("testmidi0.mid", std::ifstream::binary);
if (is) {
is.seekg(0, is.end);
length = is.tellg();
is.seekg(0, is.beg);
buffer = new char[length];
for (int i = 0; i < length; i++) {
buffer[i] = 0;
}
is.read(buffer, length);
is.close();
}
info.currentPosition = 0;
info.fileSize = length;
MidiParser parser = MidiParser(&info, getCharCallback);
MidiEvent *midiEvent;
/// start by purging out every 0
int i = 0;
while (midiEvent = parser.getNextEvent(0)) {
if (midiEvent->deltaTick > 0) { break; }
cout << "EVENT: @" << (int)midiEvent->deltaTick << " type: ";
printEventType(midiEvent);
cout << endl << "\t";
for (int i = 0; i < midiEvent->dataLength; i++) {
cout << std::hex << (int)midiEvent->data[i] << " ";
}
cout << endl;
midiEvent->isConsumed = true;
i++;
if (i > 10) {
break;
}
}
//let's simulate some ticks!
for (int i = 0; i < parser.ticksPerQuarterNote * 100; i++) {
if (midiEvent = parser.getNextEvent(1)) {
if (midiEvent->emitAtTick <= i) {
cout << "EVENT: @" << (int)midiEvent->deltaTick << " type: ";
printEventType(midiEvent);
cout << endl << "\t [";
for (int i = 0; i < midiEvent->dataLength; i++) {
cout << std::hex << (int)midiEvent->data[i] << " ";
}
cout << "]" << endl;
midiEvent->isConsumed = true;
}
}
std::this_thread::sleep_for(std::chrono::microseconds((uint32_t)parser.microsecondsPerTick));
}
delete[] buffer;
system("pause");
return 0;
}
view raw LightMIDI.cpp hosted with ❤ by GitHub

Good luck and I’ll catch you later!

First pass parsing MIDI

Leave a comment