FIX with QuickFIX/J and
EPAM FIX
Balázs Kollár
APRIL 25, 2015
CONFIDENTIAL
1
About EPAM Systems, Inc
EPAM 1993 founded, US-based
Systems Public (NYSE: EPAM)
Products:
B2BITS© EPAM's
Capital
Markets
Competency Center delivering a
broad range of solutions and
consulting services in Capital
Markets
 FIX servers
Development Belarus
Centers Ukraine
Russia
Hungary
Kazakhstan
Poland, Bulgaria
Headcount
CONFIDENTIAL
>10000 (engineers)
 FIX engines
 Solutions for Direct Exchange Access
 OMS & Matching engines
 Market data feeds adaptors
 FIX Testing & Operational Tools
Services:

Consulting
 Integration
 FIX on-boarding services
 Cloud services
2
2
Table of contents
1
What is FIX protocol
2
Systems utilizing FIX
3
FIX messages, types
4
Transports, threading
5
Advantages & drawbacks of QFJ
5
Benefits of EPAM FIX
CONFIDENTIAL
3
What is FIX protocol
•
Financial Information Exchange, in use since 1992
•
Covers pre-trade, trade, post-trade communication
•
Much more than another markup language:
•
It’s types, value sets give a comprehensive business dictionary
•
Standard names make understanding between parties easier
CONFIDENTIAL
4
SYSTEMS UTILIZING FIX
CONFIDENTIAL
5
OMS: Order Lifecycle Management
1.
Enter, change, cancel orders
2.
Order event processing
3.
Provide order state and history
4.
Get a list of orders (open, done)
CONFIDENTIAL
6
MarketData Streaming
1.
Clients subscribe to Market Data updates
–
–
2.
Clients need to know prices to submit orders
Machines vs humans
Quote engines
–
Clients request a “price offer” before submitting quoted orders
CONFIDENTIAL
7
Post-trade
1.
Reconciliation modules get a notification of each trade. Later they check if all trades
arrived properly to the middle office.
2.
Post-trade enrichment modules
3.
4.
– Client Identifying Data
– fees
Booking, allocation: splitting trades
Position and risk management
– How much USD I owe to the market? What amount I need to hedge?
CONFIDENTIAL
8
Questions?
CONFIDENTIAL
9
FIX MESSAGE TYPES
CONFIDENTIAL
10
Pre-trade: Market Data
•
A client would like to see the actual prices
•
Market Data Request ->
•
– initiate subscription for stream of market data of an instrument
<- Market Data (Snapshot/Incremental)
– Bid/Ask best price and amount: top-of-the book
– May be a full order book: best 5-10 prices and amounts
CONFIDENTIAL
11
Pre-trade: Quoting Orders
„A quote is an offer with validity. The sell side is obliged to hold this offer.”
1.
Quote request: before entering an order clients can request an „offer”.
2.
Quote: an offer containing price and amount, that’s valid until a certain time.
– Example: give me buy price for 2m$. Quote time is shown with progress bar.
CONFIDENTIAL
12
Order entry: Single-leg (35=D)
•
New Order Single
–
–
–
–
Side, Instrument, quantity, quantity unit: USD/EUR BUY 1m$
OrdType: market, limit, stop: Buy @1.11 or better
Price, StopPx
TimeInForce
• IOC
• resting: day, good-till-cancel, good-till-date
•
Detailed explanation: @Investopedia.com
CONFIDENTIAL
13
Order entry: Single-leg (35=D)
•
New Order Single
–
–
–
–
Effective Time
Expire Time
Settlement Type/Date: T+2, T+3 days
Parties
• Originating Firm
• Trader ID
• Account Nr
CONFIDENTIAL
14
Order event notifications: ExecutionReport (35=8)
One message type for many purposes
1.
Confirmation / Rejection
2.
Replaced
3.
Cancelled (may be unsolicited)
4.
Expired
5.
Filled: partial fill or last fill
6.
DoneForDay: daily stats at end of the trading day
CONFIDENTIAL
15
Order event notifications: ExecutionReport
Notification attributes
Filled-, Open-, Cumulative quantity
Filled price
Avarage Price
CONFIDENTIAL
16
Order amendment and cancellation
1.
OrderCancelReplaceRequest (35=G)
– Not amendable: instrument, side, quantity unit
2.
OrderCancelRequest (35=F)
– Only the open quantity can be canceled, the executed quantity can not
3.
Both can be rejected: OrderCancelReject (35=9)
– Nothing happened: the original order remains active
CONFIDENTIAL
17
Questions?
CONFIDENTIAL
19
SOCKET TRANSPORT,
REPLAYING
CONFIDENTIAL
20
Socket transport
•
Traditional Server-Client communication over sockets
•
The server is called Acceptor
•
The client is called Initiator
•
The Sessions are preconfigured on the server:
• SenderCompID, TargetCompID
•
Sequence numbers: both sides has a counter, always increasing
CONFIDENTIAL
21
Socket transport
•ASCII, separator 0x00 or SOH
CONFIDENTIAL
22
Offline messages
•
Session acts like a queue on both sides: store and forward technique
•
While offline both parties may produce messages, especially the one producing Executions
•
Orders can be rejected while offline or delivered later
•
After reconnection the offline messages will be delivered
CONFIDENTIAL
23
How message replaying works
1.
When session was lost, the last SeqNr seen from the Acceptor side was 1000
2.
While offline 500 trades are executed on the Acceptor
3.
Initiator logs in to the Acceptor -> gets a login confirmation with SeqNr=1501
4.
Now the FIX library knows that it missed 500 messages!
5.
Initiator sends ResendRequest from 1001 to 1500
6.
Messages >1500 will be queued in the FIX stack until the replay is finished.
•
You may want to reject any new, replace, cancel requests until eveything is back in sync!
CONFIDENTIAL
24
Questions?
CONFIDENTIAL
25
OTHER TRANSPORTS
CONFIDENTIAL
26
FIXML
1.
Not plain text anymore, don’t need socket connection
2.
Parsing, binding is for you to do: SAX, JAXB, JibX
3.
You can send it via any transport: message queues, HTTP
4.
QFJ transports it via 212/213 tags (XmlData) but still over Socket transport
5.
No other support in QFJ, won’t bind it to it’s model
CONFIDENTIAL
27
FIX over tunnels
1.
Not supported by QuickFIX/J
2.
You can use Apache Camel’s QuickFIX/J endpoints and route the messages to any other
transport: JMS, HTTP, file, etc.
3.
Or you can use FIXML and generate the java model from XSD
CONFIDENTIAL
28
Questions?
CONFIDENTIAL
29
THREADING
CONFIDENTIAL
30
Threading: sending
•
QuickFIX/J does transport with async networking
•
The thread you’re sending from (Session.send()) won’t deal with networking
•
Always check the return value of send(), it may return false and won’t throw an exception
if session is not logged in
CONFIDENTIAL
31
Threading: receiving
1.
You’ll receive messages on the QuickFIX/J listener thread
2.
Dispatch all work to another thread if possible
– Throwing an exception on the listener thread you can break QFJ, trigger resending, etc.
3.
Single thread or
4.
Sticky thread assignment: Given a thread pool for processing. Make sure events of an order
are always processed on the same thread.
CONFIDENTIAL
32
Questions?
CONFIDENTIAL
33
MEMORY
CONFIDENTIAL
34
QFJ model
1.
Lots of wrappers, even around primitives
2.
Produces lots of garbage
CONFIDENTIAL
35
Advanced: skipping QFJ model
1.
2.
Replace FIXMessageEncoder
– Write an adaptor to read your internal model directly
Replace FIXMessageDecoder
– Bind to your model while parsing
CONFIDENTIAL
36
Questions?
CONFIDENTIAL
37
PROS/CONS FOR QFJ
CONFIDENTIAL
38
PROs
1.
Free and open source.
2.
You can extend it.
3.
Well tested. Not too many bugs faced.
4.
We use it in prod for years now.
CONFIDENTIAL
39
CONs
1.
Memory footprint, GC overhead
2.
Over-a-day scheduling not supported
3.
4.
– Early start on Sunday
HA limitations: only file, jdbc and sleepycat
– no cache-based message store, but easy to write one
Parser: no annotation-based binding to custom model
– But extensible: you can implement your own parser
CONFIDENTIAL
40
CONs
1.
Defalt parser creates lots of objects
2.
Replay handling is hidden
3.
4.
– Hard to wait until replay is finished. You can add a listener.
No monotoring admin UI
– Just a simple JMX interface
No synchronuous sending.
CONFIDENTIAL
41
EPAM FIX
CONFIDENTIAL
42
EPAM FIX
CONFIDENTIAL
43
FIX ANTENNA JAVA ENGINE API WORKING WITH MESSAGES
FIX FLAT MESSAGE MODEL
CREATE MESSAGE
SINGLETON BY FIX
PROTOCOL
VERSION &
MESSAGE TYPE
VALIDATE
MESSAGE
API TO WORK
WITH FIX
MESSAGES
•
Generic model to work with the abstract FIXMessage
class via fields and groups getters and setters
•
Provides the highest performance
FIX OBJECT MODEL
CREATE MESSAGE
FROM RAW FIX
STRING (PARSE)
•
Each FIX message type is a class with FIX fields as
members
•
Use of intelligence to make work with FIX business
object more pleasant
PREPARED MESSAGES
•
ADD, REMOVE,
MODIFY FIX
FIELDS AND
REPEATING
GROUPS
CONFIDENTIAL
SERIALIZE FIX
OBJECT MESSAGE
TO RAW FIX
STRING
Message template to increase speed of sending
messages of the same structure but with different
values
EFFICIENT GETTERS AND SETTERS
•
Allows working with values as with primitive type to
reduce garbage production
44
FIX Antenna Java Low-Latency Design
1.
2.
3.
4.
5.
6.
7.
Zero-GC design for FIX message parse/send/receive paths
Avoid the use of garbage producing calls in standard java library
Code instrumented GC-free regression testing of new releases
Use of pre-allocated object pools
Use of Java off-heap memory for data buffers
Custom data structures packed in byte / integer arrays (no standard java collections)
Use of memory mapped files in persistence layer
CONFIDENTIAL
45
FIX ANTENNA JAVA INTEGRATION
Set of adapters extending FIX Antenna Java functionality
•Links FIX Antenna with the
Java Message Service
• Contains a server for routing
FIX messages to and from JMS
server
• Functional core to support
FAST
CONFIDENTIAL
• Set of libraries for FIX FIXML
conversion
JMS
ADAPTOR
FIXML
CONVERTOR
FAST
ENGINE
CME MDP
3.0
ADAPTOR
• Fully functional application
supporting all features of
new MDP3.0 platform
46
FIX ANTENNA JAVA THREADING MODEL
SYNCHRONOUS SENDING
•
•
ASYNCHRONOUS SENDING
Send in the same thread where FIXSession.sendMessage
was called
No additional delays for threads communication (better
latency)
• No blocking on sending (till queue has free space)
• FIX Antenna can send messages in batches (better throughput)
• Current queue size is accessible for analysis
SERIALIZE
FIX
MESSAGE
USER THREAD
USER
THREAD
SERIALIZE FIX
MESSAGE
FIX ANTENNA
SYSTEM
THREAD POOL
MESSAGE
QUEUE
OUTGOING MESSAGE
STORAGE
READING
THREAD
INCOMING MESSAGE
STORAGE
PER TYPE
HANDLERS
CONFIDENTIAL
USER
DEFINED
HANDLERS
SYSTEM
HANDLERS
READ &
PARSE FIX
MESSAGE
MESSAGE
QUEUE
FIX ANTENNA
SYSTEM
THREAD POOL
SENDING
THREAD
OUTGOING
MESSAGE
STORAGE
READING
THREAD
INCOMING MESSAGE
STORAGE
USER
DEFINED
HANDLERS
SYSTEM
HANDLERS
READ &
PARSE FIX
MESSAGE
47
FIX ANTENNA JAVA THREADING MODEL
• Input and output disruptors
FIX ANTENNA
OUTPUT
DISRUPTOR
COOPERATION WITH DISRUPTOR
SENDER
RECEIVER
• Asynchronous sending – stay not
affected by slow FIX clients
FIX SESSION
FIX MESSAGE
SENDER
FIX
FIX MESSAGE
READER
FIX SESSION
FIX MESSAGE
SENDER
FIX
FIX MESSAGE
READER
Input
Disruptor
SENDER
FIX SESSION
FIX MESSAGE
SENDER
FIX
FIX MESSAGE
READER
CONFIDENTIAL
48
FIX Antenna Java “Hello World!”
SessionParameters details = new SessionParameters();
details.setFixVersion(FIXVersion.FIX42);
details.setSenderCompId("BLP");
details.setTargetCompId("SCHB");
FIXSession fixSession =
connectionDetails.createNewFIXSession();
message.set(11, clordid);
message.set(44, 100); // Price
message.set(38, 10);
// OrderQty
fixSession.sendMessage(message);
CONFIDENTIAL
49