Quant Development

Introduction

What Is Quant Development?

Everyone seems to mean something slightly different by this. The definition I am going to use is:

The process by which mathematical models are implemented in a bank to create organisational value

The Big Question

Mathematical models have not really changed, from an IT perspective, since the early 1990s.

Given that, why are all the banks still spending millions a year on the systems around quant code?

The Problem With Quant Code

What Should Happen	What Actually Happens
Trade and market data descriptions should not be translated	Trade and market data are translated at each layer
Results (delta ladders etc) should not be translated	Results are translated at each layer
Quants should produce serialization and deserialization of their library objects	IT systems do their own serialization and deserialization of quant library objects
Quant interfaces should look as similar as possible from all languages and environments	Quant libraries have several different interfaces for different languages and environments
Quant libraries should offer only one way to do any given thing	Quant libraries may have several overlapping sets of models, all in use for live pricing
Static data should not be packaged with quant libraries	Every quant library release has its own set of static data with it

Why Is Quant Code So Poorly Implemented In the Application Stack?

Quants And IT Don't Get Along So Well

What Some Quants Think About IT People	The Truth
They are scared by maths and don't want to understand what is happening	IT People often just aren't all that interested in maths
They don't understand the maths, and so they are stupid	IT People often just aren't all that interested in maths
They pick up Quant libraries and misuse the interfaces without letting the Quants know what they are doing	A great developer will talk to Quants off their own bat. But most developers will go off the documentation they can find. And if they can't find good documentation, whose fault is that?
They don't let the Quants know what they want in good time - everything is an emergency	The IT people often don't know enough about the Quant libraries to understand how requirements arise
What Some IT People Think About Quants	The Truth
They want to look good by writing lots of models but can't be bothered to help get them implemented	Quants are often simply unaware of how complex systems in the bank are
They think you are stupid if you don't have a PhD	Quants spent years of their life getting a PhD and so are emotionally invested in their being worthwhile
They aren't capable of writing robust code	Quants are perfectly capable, but often woefully undertrained in how to produce robust libraries
They don't document how their libraries are supposed to be used well enough	There is often lots of documentation but no-one is organising it
They don't take the problems of people trying to use their libraries seriously	IT people often don't provide enough information for Quants to be able to diagnose problems

Too Many Specialists, Not Enough Generalists

If you're in banking IT, learn the maths and the business. Hardly anyone does, and that lack of a wider understanding is partly why it's so hard for people in IT to push good practice to the business. Learn the language and frames that everyone else is using rather than being stuck in a ghetto.

But conversely, if you're in banking IT, find a trading desk that has some people that understand technology. Otherwise you'll always be having to make the business case for good practices rather than meeting the business half-way. You need to be able to make the case, sure, but it's a waste of time and energy to have to do so continually. Many teams get tired of doing this, and end up hustled into too many short term fixes, as a result of which the whole development eventually needs to be restarted from scratch.

Complexity Budgets And Project Restarts

No matter how much money you have to throw at software development, there will come a point where you simply cannot make any further progress because your environment has become too complex to release updates into reliably.

A good way to think about this is as a "complexity budget" that needs to be handled like any other scarce resource. Overall complexity in practice the main constraint in most banks nowadays.

The only way to get around a complexity barrier is to re-engineer parts of your system to make them less complex.

The trajectory of a typical banking IT project is a year or two of wild enthusiasm, a few years of gradually slowing releases, one or two years of stagnation as the structure comes apart and then a restart. The whole process seems to take about seven to ten years.

Lack Of Knowledge Of Good Practice

Banking experience is overpriced by many places. Many people in banking Quant and IT become less valuable over time. They see projects buckling under the pressure from the trading desk to deliver, and think that somehow practices such as unit and other testing, daily builds, and the like don't apply to banks. They get lazy and used to delivering low quality stuff for a high reward.

People coming from outside IB have often got better quality development experience than peers who have spent all their time in the IB sector. They're the ones who are more likely to be able to make a good business case for decent development practices.

The best thing to do if you're in a bank and recruiting is to actively seek people with no banking experience. Telecommunications. Games. Image Processing. Air Traffic Control. There's plenty of people out there who are great value. Many agencies will assume you want prior banking experience unless you explictly tell them otherwise. Do so.

Bonus Cycles

The pressure is always on to deliver in time for the compensation round. Stuff is rushed out before it's ready, and without the testing in place to allow refactoring. Much of the following year is spent clearing up the resulting mess.

Turnover

People move between banks with great frequency. Average times in jobs depend on who you ask, but if you're in the same area for more than 2 years, you'll be starting to be an authority. The knowledge of a system's "theory", the metaphors and assumptions that underly its structure, degrades quickly.

This is why documentation is so important. Unfortunately the worth of documentation tends to come in over a multi-year cycle, so it's particularly badly aligned with the yearly bonus cycles that banks use.

High Entry Cost To The Sector

It takes a lot of money to set up as a bank. So banks can be inefficient everywhere in the organisation and still be OK. This is getting eroded by the hedge funds, but is still important.

Organisational Structure

I've moved employers a few times, and am always struck by the difference between old and new in what software was easy to write, and what software was hard to write.

Here's a law dating back to the 1970s:

Conway's Law: Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure. More than 25 years ago, a book called "The Psychology Of Computer Programming" talked at length about this kind of thing.

For example, a compiler's internal structure will likely be split along lines dictated by the existing expertise and political structure, rather than some structure purely dictated by technological considerations.

However, many technologists tend to frame their view of the world using technical considerations in preference to political/organisational considerations. The upshot can be that people attempt to defend the structure of a software project in technical terms because they think that other drivers of the structure aren't (or shouldn't be) relevant. If, instead, there was a wider recognition that it's perfectly OK for non-technical considerations to be the most important shaping factor on the structure of software, rational debate about the structure would be easier; for instance, people would address broken software structure by addressing the underlying organisational mismatch, rather than searching for purely technical fixes.

It's worth keeping Conway's Law in mind when a technological goal seems strangely hard to accomplish. Look at how the effort is being organised, and how well it fits with the people who are actually supposed to do the work. Is it something that requires a high degree of inter-team co-ordination in an environment where autonomy is highly prized? Or, conversely, is it something that requires initiative and a number of prototyping runs in an environment where blaming behaviour is common, and failure attracts censure? In either of those two cases, you've got a problem.

Instead of banging your head against a brick wall, look at how you can orient things to align with the setting you're working in. If autonomy is highly prized, keep the shared infrastructure as loose and minimal as possible, and bend over backwards to encourage participation (or it won't happen). If control and blaming are more the order of the day, use the hierarchy for what it's good at, and get skunk-works set up to do the more experimental stuff, providing a separate safe environment for the riskier failure-prone part.

How To Improve Things?

Bridging The Gap

A Quant Developer needs to straddle the IT and Quant worlds:

• Helping Quants to structure their documentation so it is actually useful to IT
• Providing a framework to allow Quants to publish maths in a uniform fashion
• Providing a framework to allow Quant library errors to be diagnosed and resolved easily
• Ensuring that the quant libraries can be used by IT effectively
• Ensuring that the IT people use Quant libraries effectively
• Explaining IT to the Quants
• Explaining maths to the IT people
• Ensuring that IT requirements that result in Quant requirements get flagged up early

That's All Very Well, But What Do I Need To Know?

Languages:

• You have to become absolutely fluent in the relevant technologies as a basic precondition of being effective.
• C++ absolutely backwards. Think ACCU member. Without going to the meetings.
• Excel/VBA absolutely backwards.
• At least one of C#/Java reasonably well so you can engage with the IT side
• At least one functional language so you know what you're missing

Financial Maths:

• If you work in Interest Rates, you should be able to bootstrap a yield curve from scratch without really thinking about it.
• You should have a reasonable understanding of how Stochastic Calculus works.
• You should have an instinctive appreciation of why credit quant papers are so funny.

Non-Technical Skills:

• Learn to communicate effectively. If you can't write and present this is not a good role for you.
• Get some experience in management. You cannot understand the pressures affecting IT team and project managers unless you have gone at least a little down a similar road yourself.
• Read up about consultancy, since that will be a large part of your role. Gerald Weinberg is a great place to start.

Getting Traction:

• Get a track record in implementing stuff that works before you tell people how to do things. You will be surrounded by people who can argue the toss so you need a record you can point to if you are to get people to follow your lead.
• Sort out projects that have gone off the rails. These are the hardest work, but the ones that get you respect.
• If you are getting people to change tack, you need to be able to sell it in terms of presenting reasonably immediate and visible returns, or it won't get done. This may change the order you schedule execution of a plan.
• Always order work as if, tomorrow, someone is going to give you a few day's notice to drop tools and do something else entirely for six months. Checkpoint and document as you go, or months of work will be lost at some point because you had to abandon it.