The idea is to use CPP preprocessing instructions along with standard Linux CLI tools(bash,sed,grep) to generate FIX API for a given venue.
Mindset:
- No third party dependencies. All the generated code is yours and can go straight into your lib or app.
- Repeating groups are supported.
- You can strip off all useless standard FIX tags to fit your venue specs.
- FIX tags and enums are treated as integers and not strings thus removing transformations.
Receiving (read more):
- Lightweight and low latency FIX parsing with reusable memory and objects.
- Optional message sanity checks are available.
- Flexible formatting with available predefined TTY color styles.
- Pretty printing with GDB and VSCode.
Sending (read more):
- Low latency FIX message building with reusable memory and objects.
- For a given message type, only fields changing between two sends are to be updated.
- BodyLength and MsgSeqNum fields can be zero padded to reduce memory copies.
You have to know:
fixppis not a FIX engine (yet)- it performs better in an optimistic mode and relies on the venue FIX conformance (this saves quite a few CPU cycles...)
- a bit slower but safer parsing is available as well (it detects bad fields, groups and enums during message scan)
Tags and enums are not converted. For instance the sequence like "|49=" will be inserted as a single integer assignment with value 0x3d'39'34'01. On reception, the string "49" will not be translated to decimal value 49 but rather reinterpreted as 0x39'34.
Results obtained in a tight loop on an i7-1360P @ 5GHz
| action | message type | length | time, ns | CPU cycles | HW instructions |
|---|---|---|---|---|---|
| trusted decode | ExecReport | 170 | 74 | 370 | 912 |
| safe decode | ExecReport | 170 | 104 | 518 | 1694 |
| trusted decode | MarketDataSnapshotFullRefresh* | 330 | 116 | 580 | 1797 |
| safe decode | MarketDataSnapshotFullRefresh* | 330 | 122 | 610 | 2708 |
| encode | NewOrderSingle | 170 | 55 | 273 | 897 |
| update | NewOrderSingle | 170 | 25 | 112 | 376 |
* 6 repeating groups
Based on your venue FIX API prepare files Fields.def, Groups.def and Messages.def. One can pick up ready to use files from examples/spec and adjust Groups.def and Messages.def. Fields.def will be cleaned automatically during generation.
If you have already a Quickfix XML file you can use the xml2cpp converter to generate the definitions.
src/spec/Fields.def
FIX_FIELD_BEGIN_STRING( FIX.4.4 )
FIX_FIELD_DECL( Account , 1, STRING )
FIX_FIELD_DECL( AvgPx , 6, PRICE )
FIX_FIELD_DECL( BeginSeqNo , 7, SEQNUM )
FIX_FIELD_DECL( BeginString , 8, STRING )
FIX_FIELD_DECL( BodyLength , 9, LENGTH )
FIX_FIELD_DECL( CheckSum , 10, STRING )
FIX_FIELD_DECL( ClOrdID , 11, STRING )
...
FIX_ENUM_BEGIN( MsgType )
FIX_ENUM_DECL( MsgType, HEARTBEAT , SOHSTR(0) )
FIX_ENUM_DECL( MsgType, TEST_REQUEST , SOHSTR(1) )
FIX_ENUM_DECL( MsgType, RESEND_REQUEST , SOHSTR(2) )
FIX_ENUM_DECL( MsgType, REJECT , SOHSTR(3) )
FIX_ENUM_DECL( MsgType, SEQUENCE_RESET , SOHSTR(4) )
FIX_ENUM_DECL( MsgType, LOGOUT , SOHSTR(5) )
FIX_ENUM_DECL( MsgType, EXECUTION_REPORT , SOHSTR(8) )
FIX_ENUM_DECL( MsgType, LOGON , SOHSTR(A) )
FIX_ENUM_DECL( MsgType, ORDER_SINGLE , SOHSTR(D) )
FIX_ENUM_DECL( MsgType, MARKET_DATA_REQUEST , SOHSTR(V) )
FIX_ENUM_DECL( MsgType, MARKET_DATA_SNAPSHOT_FULL_REFRESH, SOHSTR(W) )
FIX_ENUM_DECL( MsgType, MARKET_DATA_INCREMENTAL_REFRESH , SOHSTR(X) )
FIX_ENUM_DECL( MsgType, MARKET_DATA_REQUEST_REJECT , SOHSTR(Y) )
FIX_ENUM_END
...
src/spec/Groups.def
FIX_MSG_GROUP_BEGIN( MDEntries, MDEntryType )
FIX_MSG_FIELD( MDEntryPx )
FIX_MSG_FIELD( Currency )
FIX_MSG_FIELD( MDEntryPositionNo )
FIX_MSG_FIELD( MDEntrySize )
FIX_MSG_FIELD( MDEntryDate )
FIX_MSG_FIELD( MDEntryTime )
FIX_MSG_GROUP_END
...
src/spec/Messages.def
FIX_MSG_BEGIN( Header )
FIX_MSG_FIELD( BeginString )
FIX_MSG_FIELD( BodyLength )
FIX_MSG_FIELD( MsgType )
FIX_MSG_FIELD( SenderCompID )
FIX_MSG_FIELD( TargetCompID )
FIX_MSG_FIELD( MsgSeqNum )
FIX_MSG_FIELD( SendingTime )
FIX_MSG_END
FIX_MSG_BEGIN( Heartbeat )
FIX_MSG_FIELD( TestReqID )
FIX_MSG_END
FIX_MSG_BEGIN( Logon )
FIX_MSG_FIELD( EncryptMethod )
FIX_MSG_FIELD( HeartBtInt )
FIX_MSG_FIELD( ResetSeqNumFlag )
FIX_MSG_FIELD( Username )
FIX_MSG_FIELD( Password )
FIX_MSG_END
FIX_MSG_BEGIN( Logout )
FIX_MSG_FIELD( Text )
FIX_MSG_END
...
See more in examples.
/path/to/fixpp/generate.sh -d MYPRJ/src/myprj/fix -s MYPRJ/src/spec -i myprj/fix -n venue::fix -p MYPRJ/src/gdb
Where the options are:
-ddestination directory-sspecification directory with .def files-iprefix to use in include statements (ex:#include <prefix/Field.h>)-nnamespace to use-ppretty printers destination dir
It will create the following tree:
MYPRJ
├── Makefile
|
└── src
|
├── spec // your input
| ├── Fields.def
| ├── Groups.def
| └── Messages.def
│
├── gdb // generated files
│ └── printers.py
|
└── myprj
│
└── fix // generated files
├── Fields.cpp
├── Fields.h
├── FixApi.h
├── Groups.cpp
├── Groups.h
├── Messages.cpp
├── Messages.h
└── SenderApi.h
A fully generated and committed primitive project is available in examples/order.
class BusinessLogic
{
void onMessage( const MessageExecutionReport & msg )
{
// for mandatory fields it is safe to get the value:
Quantity qty = msg.getOrderQty();
// for optional fields:
double px = msg.isSetPrice() ? msg.getPrice() : 0;
...
}
void onMessage( const MessageMarketDataSnapshotFullRefresh & msg )
{
...
}
};In most cases a latency sensitive implementation will tend to reuse objects and avoid allocations on the critical path. Here we suggest keeping a Header object and messages separately. The parsing is carried out in two steps:
- scanning the header first,
- then, after identifying the message type, scanning the relevant message body.
// my class attribute:
Header _header;
MessageExecutionReport _execReport;
MessageMarketDataSnapshotFullRefresh _mdsfr;
BusinessLogic _businessLogic;
...
// after reading the message into fixString
offset_t pos = _header.scan( fixString.data(), fixString.size() );
switch( _header_.getRawMsgType() )
{
case MsgTypeRaw_EXECUTION_REPORT:
_msgExecutionReport.scan( fixString.data() + pos, fixString.size() - pos );
_businessLogic.onMessage( _execReport );
break;
case MsgTypeRaw_MARKET_DATA_SNAPSHOT_FULL_REFRESH:
_mdsfr.scan( fixString.data() + pos, fixString.size() - pos );
_businessLogic.onMessage( _mdsfr );
break;
...
}fixpp generates a class ParserDispatcher implementing the above switch. You have just to override onMessage(...) methods.
#include <tiny/Messages.h>
#include <cstring>
const char * buffer = "8=FIX.4.4" I "9=156" I "35=8" I ... I "10=075" I;
using namespace venue::fix;
class MyProcessor: public ParserDispatcher
{
protected:
virtual void unprocessedMessage( raw_enum_t msgType, MessageBase & msg ) override
{
auto it = MsgTypeEnums::itemByRaw.find( msgType );
const char * typeName = it != MsgTypeEnums::itemByRaw.end() ? it->second->name : "unknown";
std::cout << "unprocessed " << typeName << std::endl;
}
virtual void onMessage( MessageMarketDataSnapshotFullRefresh & msg ) override
{
std::cout << "processed " << getCurrentHeader().getMsgType() << " " << msg.getMessageName() << std::endl;
}
};
int main( int args, const char ** argv )
{
size_t len = strlen( buffer );
const char * cursor = buffer;
MyProcessor mp;
while( cursor = mp.parseAndDipatch( cursor, len - ( cursor - buffer ) ) )
{
}
return 0;
} constexpr unsigned begStrAndBodyLenBytes = MESSAGE_BEGIN_MIN_BYTES_TO_READ;
std::vector<char> recvbuffer( 4096 );
while( source.read( &recvbuffer[0], begStrAndBodyLenBytes ) )
{
unsigned msgTypeOffset;
len = parseMessageLength( &recvbuffer[0], msgTypeOffset ) + CHECKSUM_FIELD_LENGTH;
if( recvbuffer.size() < len + msgTypeOffset )
{
recvbuffer.insert( recvbuffer.end(), len + msgTypeOffset - recvbuffer.size() + 100 , 0 );
}
// read the remaining bytes
if( source.read( & recvbuffer[begStrAndBodyLenBytes], len - begStrAndBodyLenBytes + msgTypeOffset ) )
{
mp.parseAndDipatch( &recvbuffer[0], len + msgTypeOffset );
}
else
{
break;
}
}#include <myprj/fix/Messages.h>
const char * execReport = "8=FIX.4.4" I "9=332" I "35=8" I "49=foo" I "56=bar" I "52=20071123-05:30:00.000" I
"11=OID123456" I "150=E" I "39=A" I "55=XYZ" I "167=CS" I "54=1" I "38=15" I "40=2" I "44=15.001" I "58=EQUITYTESTING" I "59=0" I "32=0" I "31=0" I "151=15" I "14=0" I "6=0" I
"555=2" I "600=SYM1" I "624=0" I "687=10" I "683=1" I
"688=A" I "689=a" I
"564=1" I
"539=2" I "524=PARTY1" I "525=S" I
"524=PARTY2" I "525=S" I
"804=2" I "545=S1" I "805=1" I "545=S2" I "805=2" I
"600=SYM2" I "624=1" I "687=20" I "683=2" I
"688=A" I "689=a" I
"688=B" I "689=b" I
"10=027" I;
using namespace venue::fix;
...
MessageHeader header;
// pos = offset from message start
offset_t pos = header.scan( execReport, strlen( execReport ) );
MessageExecutionReport er;
pos = er.scan( execReport + pos, strlen( execReport ) - pos );
// print single field value
std::cout << ' ' << FixOrdStatus << " = " << er.getOrdStatus() << std::endl;
// print entire message
std::cout << "\n\n -- Pretty Printing --" << std::endl;
// use operator <<
std::cout << fixstr( execReport, ttyRgbStyle ) << std::endl;
// print flat and advance pos
pos = 0;
fixToHuman( execReport, pos, std::cout, ttyRgbStyle ) << std::endl;
// indent groups
pos = 0;
fixToHuman( execReport, pos, std::cout, ttyRgbStyle, MessageExecutionReport::getFieldDepth ) << std::endl;
// iterate over groups
if( er.isSetNoLegs() )
{
unsigned noLegs = er.getNoLegs();
for( unsigned legIdx = 0 ; legIdx < noLegs; ++legIdx )
{
const GroupLegs & leg = er.getGroupLegs( legIdx );
std::cout << legIdx << ": side:" << leg.getLegSide() << " qty: " << leg.getLegQty() << std::endl;
}
}
You will have to include SenderApi.h to build FIX messages with fixpp. It offers both
- low level buffer construction with FixBufferStream
- and reusable memory approach with ReusableMessageBuilder
This structure has two attributes: begin and end. Respectively pointing to the message's first and past last byte.
Each time a new field is inserted end will shift forward accordingly. In most cases the fields will be appended as tag-value pairs:
execReport.append<ClOrdID>("OID4567");
execReport.append<QtyType>( QtyTypeEnums::UNITS );
execReport.append<Price>( 21123.04567, 2 );It is also possible to push tags and values separately:
execReport.pushTag<ClOrdID>().pushValue("OID4567");The idea behind is for a given FIX session:
- to reuse the header since most of it's fields will not change,
- to pre-compute the checksum for non-changing header's fields,
- to update only changing time within timestamps since the date does not change intra day.
This structure inherits the begin and end pointers from FixBufferStream. But begin refers to the first changing field like SendingTime for instance. The very first byte of the sending buffer will be pointed to by start. The latter will move each time the header's width changes. For example when the sequence number or body length change their widths.
buffer start msgType sendingTime body
| | | | |
"..." "8=FIX.4.4" I "9=315" I "35=W" I "49=foo" I "56=bar" I "34=1234" I "52=20190101-01:01:01.000" I "..."
| |
begin end
A typical scenario will be
using namespace fix;
using namespace fix::field;
using namespace fix::message;
...
/// before we send it
// prepare
ReusableMessageBuilder order( NewOrderSingle::getMessageType(), 512, 128 );
order.header.append<SenderCompID>("ASENDER");
order.header.append<TargetCompID>("ATARGET");
order.header.pushTag<FieldMsgSeqNum>();
order.header.finalize();
// append SendingTime to the header
auto constexpr tsLen = TimestampKeeper::DATE_TIME_MILLIS_LENGTH;
auto constexpr tsFrac = TimestampKeeper::Precision::MILLISECONDS;
order.append<SendingTime>( TimestampKeeper::PLACE_HOLDER, tsLen );
// initialize the timestamp keeper
order.sendingTime.setup( order.end - tsLen, tsFrac );
order.sendingTime.update();
const unsigned sendingTimeLength = order.end - order.begin;
...
/// sending it
void sendOrder( const OrderFields & of )
{
// move end past SendingTime
order.rewind( sendingTimeLength );
// update changing fields in SendingTime
order.sendingTime.update();
// append order specific fields
order.append<Account>( of.account, of.accountLen );
order.append<ClOrdID>( of.orderId, of.orderIdLen );
order.append<Symbol>( of.symbol, of.symbolLen );
order.append<Side>( of.side );
order.append<Price>( of.price, 6 );
order.append<OrderQty>( of.qty );
// copy SendingTime into TransactTime
order.append<TransactTime>( order.sendingTime.begin, tsLen );
order.append<OrdType>( of.type );
// finalize
order.setSeqnumAndUpdateHeaderAndChecksum(++seqnum);
// send it
socket.send( order.start, order.end - order.start );
}
To compile the examples you wil have to clone the makefile project next to fixpp:
$> git clone https://github.com/sashamakarenko/makefile.git makefile
$> git clone https://github.com/sashamakarenko/fixpp.git fixpp
$> cd fixpp/examples/tiny
$> make
$> make check
- fix44 all committed complete lib with all FIX4.4 messages
- fixdump tool to decode FIX messages
- odd lib with unit tests with irregular messages
- tiny venue specific example with unit tests
- order all committed primitive project only with NewOrderSingle and ExecutionReport
- spec input files for different FIX standards
const FixFormatStyle htmlRgbStyle =
{
.messageBegin = "<pre>",
.messageEnd = "</pre>",
.indent = " ",
.groupFirstField = " •",
.fieldBegin = " ",
.fieldEnd = "\n",
.headerTagNameStart = "<font color=\"#444444\">",
.headerTagNameStop = "</font>",
.tagNameStart = "<font color=\"black\"><b>",
.tagNameStop = "</b></font>",
.tagValueStart = "<font color=\"grey\">(",
.tagValueStop = ")</font>",
.equal = " = ",
.valueStart = "<font color=\"darkblue\">",
.valueStop = "</font>",
.enumStart = " <font color=\"darkgreen\">",
.enumStop = "</font>",
.unknownStart = "<font color=\"red\">",
.unknownStop = "</font>"
};
...
std::ofstream html;
html.open( "mdfr.html" );
pos = 0;
fixToHuman( mdFullRefresh, pos, html, htmlRgbStyle, autoIndentFields );
html.close();
BeginString(8) = FIX.4.4 BodyLength(9) = 315 MsgType(35) = W MARKET_DATA_SNAPSHOT_FULL_REFRESH SenderCompID(49) = foo TargetCompID(56) = bar MsgSeqNum(34) = 1234 SendingTime(52) = 20190101-01:01:01.000 Symbol(55) = EUR/USD NoMDEntries(268) = 6 MDEntryType(269) = 1 OFFER MDEntryPositionNo(290) = 1 MDEntryPx(270) = 1.21 Currency(15) = USD MDEntrySize(271) = 1000000 MDEntryType(269) = 1 OFFER MDEntryPositionNo(290) = 2 MDEntryPx(270) = 1.211 Currency(15) = USD MDEntrySize(271) = 2000000 MDEntryType(269) = 1 OFFER MDEntryPositionNo(290) = 3 MDEntryPx(270) = 1.221 Currency(15) = USD MDEntrySize(271) = 3000000 MDEntryType(269) = 1 OFFER MDEntryPositionNo(290) = 4 MDEntryPx(270) = 1.2315 Currency(15) = USD MDEntrySize(271) = 4000000 MDEntryType(269) = 0 BID MDEntryPositionNo(290) = 5 MDEntryPx(270) = 1.201 Currency(15) = USD MDEntrySize(271) = 1000000 MDEntryType(269) = 0 BID MDEntryPositionNo(290) = 6 MDEntryPx(270) = 1.205 Currency(15) = USD MDEntrySize(271) = 2000000 CheckSum(10) = 075
Show message as HTML table:
const FixFormatStyle htmlTableRgbStyle =
{
.messageBegin = "<pre><table>",
.messageEnd = "</table></pre>",
.indent = " ",
.groupFirstField = " •",
.fieldBegin = "<tr><td>",
.fieldEnd = "</td></tr>\n",
.headerTagNameStart = "<font color=\"#444444\">",
.headerTagNameStop = "</font>",
.tagNameStart = "<font color=\"black\"><b>",
.tagNameStop = "</b></font>",
.tagValueStart = "<font color=\"grey\">(",
.tagValueStop = ")</font>",
.equal = " </td><td> ",
.valueStart = "<font color=\"darkblue\">",
.valueStop = "</font>",
.enumStart = " <font color=\"darkgreen\">" ,
.enumStop = "</font>",
.unknownStart = "<font color=\"red\">",
.unknownStop = "</font>"
};| BeginString(8) | FIX.4.4 |
| BodyLength(9) | 332 |
| MsgType(35) | 8 EXECUTION_REPORT |
| SenderCompID(49) | foo |
| TargetCompID(56) | bar |
| SendingTime(52) | 20071123-05:30:00.000 |
| ClOrdID(11) | OID123456 |
| ExecType(150) | E |
| OrdStatus(39) | A PENDING_NEW |
| Symbol(55) | XYZ |
| SecurityType(167) | CS COMMON_STOCK |
| Side(54) | 1 BUY |
| OrderQty(38) | 15 |
| OrdType(40) | 2 LIMIT |
| Price(44) | 15.001 |
| Text(58) | EQUITYTESTING |
| TimeInForce(59) | 0 DAY |
| LastQty(32) | 0 |
| LastPx(31) | 0 |
| LeavesQty(151) | 15 |
| CumQty(14) | 0 |
| AvgPx(6) | 0 |
| NoLegs(555) | 2 |
| •LegSymbol(600) | SYM1 |
| LegSide(624) | 0 |
| LegQty(687) | 10 |
| NoLegStipulations(683) | 1 |
| •LegStipulationType(688) | A |
| LegStipulationValue(689) | a |
| LegPositionEffect(564) | 1 |
| NoNestedPartyIDs(539) | 2 |
| •NestedPartyID(524) | PARTY1 |
| NestedPartyIDSource(525) | S |
| •NestedPartyID(524) | PARTY2 |
| NestedPartyIDSource(525) | S |
| NoNestedPartySubIDs(804) | 2 |
| •NestedPartySubID(545) | S1 |
| NestedPartySubIDType(805) | 1 |
| •NestedPartySubID(545) | S2 |
| NestedPartySubIDType(805) | 2 |
| •LegSymbol(600) | SYM2 |
| LegSide(624) | 1 |
| LegQty(687) | 20 |
| NoLegStipulations(683) | 2 |
| •LegStipulationType(688) | A |
| LegStipulationValue(689) | a |
| •LegStipulationType(688) | B |
| LegStipulationValue(689) | b |
| CheckSum(10) | 027 |
