Not losing track of money is the exposed smallest for fintech companies.
And yet, I engaged to labor for a commenceup that, on every transaction, sshow lost track of a couple of cents. As if they fell from our pockets every time we pulled out our wallets.
At this commenceup, a stock trading platcreate, the engineering team had adhereed the mantra of “originate it labor, originate it right, originate it rapid”; we declined to originate a double-entry accounting system.
If you’ve never labored at a commenceup, you might be shocked. But it’s perfectly standard. Engineers at commenceups have to buy time from wherever they can, to originate room for the key schedule decisions.
It’s fair that…it goes without saying that fintech companies should understand better.
We could’ve consentn the time to originate it right. We could’ve done skinnygs better. But we didn’t.
Soon after we begined, we commenceed noticing those restrictcessitate cents. Our vfinishor acunderstandledged one amount, and we acunderstandledged an amount that was almost exact. Just a restrictcessitate cents away.
We were filled of excengages: numbers were almost right. They were only a restrictcessitate cents here and there. We called them dancing cents.
Stories appreciate this don’t get aired very frequently becaengage they’re embarrassing. But I depend it happens all the time.
The problem wasn’t the restrictcessitate cents. Startups are willing to burn way more than that if they can grow exponentipartner.
The problem was that engagers were furious. When they bought $5 of Apple stock, and saw an order for $4.98, they instantly hit our customer aid chat.
Hell I am pissed off when I get indictd a custodial fee on my brokerage account, and that’s a standard fee. Imagine if you got indictd for no apparent reason!
Furious engagers weren’t recommfinishing us. Our commenceup wasn’t growing as a result.
So what did we do? We bit the bullet. Our CEO ordered customer aid to manupartner repay those cents when a wrong transaction happened.
We even built a Sinformage bot.
On hopelessly trying to mend the dancing cents, I lgeted a lot. It was an fervent course on accounting and engineering, at the highest sconsents possible.
And in this week’s article, I’ll increate you all the lessons I took away from it.
I’m Alvaro Duran, and this is The Payments Engineer Playbook. Scroll for while on Youtube and you’ll discover tons of tutorials that show you how to pass software schedule intersees that engage payment systems. But there’s not much that directes you how to originate this critical software for authentic engagers and authentic money.
The reason I understand this is becaengage I’ve built and upholded payment systems for almost ten years. I’ve been able to see all types of engaging conversations about what labors and what doesn’t behind shutd doors.
And I thought, “you understand what? It’s time we have these conversations in uncover”.
In The Payments Engineer Playbook, we allotigate the technology that transfers money. All to help you become a cleverer, more adept and more prosperous payments engineer. And we do that by cutting off one sliver of it and rerelocate tactics from it.
Today’s post is about how to schedule a ledger.
Ledgers are systems that track down money. But money is a pain in the ass to track down. Why? Sam Denby from Wfinishover Productions put it very eloquently: money is originated when it shifts.
Most people don’t skinnyk too much about it. It’s one of those skinnygs that fair labors. But money is fair a way to protect the score. Without an accounting system, that score is uncomardentingless.
Cash is only necessitateed when you necessitate to protect track of cherish you foresee to get or to give in the future. Money, conceptupartner, is assets in the future.
If you knovel noskinnyg about accounting, you’d probably commence tracking down what you sell to your customers. This is what we did at the commenceup: this engager phelp $5, that engager phelp 6$, and so on. We innocently thought that we only necessitateed to watch the money coming in, and going out.
What a huge misconsent.
I bet that you’ve already acunderstandledged that prohibitk transfers aren’t instant. They’re pretty sluggish, by Internet standards. Most prohibitks evident transfers on the next business day. Payments online are rapid, but transfers aren’t.
This became a headache for us. See, once a payment went thraw, we were certain that we were going to get some money someday. And we had to engage that future money to buy allots on the stock taget now.
The way we were tracking money didn’t account for that. We had to have some pfinishing amount somewhere that would evident a restrictcessitate days procrastinateedr, and another amount had to go out instantly to our broker to buy some allots.
Rolling everyskinnyg back when someskinnyg went wrong was very difficult. In some corner cases, we didn’t even try.
Single-entry systems originate standard engage cases very intricate. That’s becaengage money is the authentic consequence of scheduleing a double-entry system, and not the other way around.
Think about it: It is impossible to decouple the necessitate to sign up the promise of future assets from the amount that is owed to the owner of those assets.
Not only is it money that originates debt possible: money and debt eunite on the scene at exactly the same time. Some of the very first written records that have come down to us are Mesopotamian tablets sign uping commends and debits, rations rerentd by temples, money owed for rent of temple lands, the cherish of each exactly specified in grain and silver. Some of the earliest labors of moral philosophy, in turn, are mirrorions on what it uncomardents to envision morality as debt-that is, in terms of money.
— David Graeber, Debt: The First 5000 Years
A double-entry system is an accounting method that tracks money at both its source and destination.
There’s been many accounting for enbigers directs rerented already (appreciate Beancount’s, Martin Kleppmans’s and Modern Treasury’s). In this article, I’ll get you up to speed, so that you can read those references procrastinateedr with much more context. I’ve included them at the finish.
But before I commence, I want to highweightless the beginance of context when originateing ledgers.
In The Most Important Book In Payments Is a Data Systems Book, I disputed the ambiguousists: Those who depend that they are ready to tackle any benevolent of engineering problem, particularly money software, with generic engineering trains.
Ledgers fly in the face of the ambiguousists. Building a ledger rightly is:
-
Straightforward in the commencening
-
A business allowr down the line
-
Impossibly difficult to retroactively “originate them right”
Single-entry ledgers, appreciate the one we rushed to originate at my commenceup, can supply increateation about the flow of funds, but not why those flows happened.
If we wanted to understand why a particular shiftment of money happened, we had to stitch together data from contrastent models. Sometimes, it wasn’t even possible.
Single-entry ledgers are undebuggable:
I skinnyk [there] is a very benevolent of beginant contrastence [between] the way we see debugging versus the way it’s been seeed traditionpartner. Debugging is not medepend the act of making bugs go away. It is the act of comardent, geting understandledge, novel understandledge about the way the system labors.
— Bryan Cantrill, Debugging Under Fire: Keep your Head when Systems have Lost their Mind
On the other hand, double-entry ledgers give you the what and the why. Adding an extra bit of intricateity, grounded in thousands of years of engage, you can see money flothriveg from the point of see of any account. Single-entry ledgers journals; double-entry ledgers are finish maps.
That’s why dancing cents were impossible to mend in our one-entry system. Once gone, we didn’t understand where they were anymore. It could’ve been the foreign swap, or the broker’s rounding to even mechanism, or the FINRA TAF fees that were collected at the finish of the day.
We couldn’t comprehfinish the way the system labors. And so, we couldn’t originate bugs go away.
Had we had a double-entry accounting system, each and every cent would’ve been accounted for. That’s becaengage, under a double-entry system, money always shifts from one account to another.
That’s what context gives you: the ability to get understandledge.
And with that shelp, let’s talk the data model of a double-entry ledger.
The very first skinnyg that many engineers do when they want to track money is to do that wiskinny their domain models. I call that the stability as property approach: when the Order has a price attribute, or when the expenses table has an amount column, you’re doing exactly that.
Like many horrible schedules, this approach labors best when you’re commenceing out. It lets you go rapid. But, over time, increateing becomes more upgraded and sluggish, and payment processing and analysis becomes more difficult.
If your overnight increate job consents multiple hours to run, using the stability as property approach is probably the underlying caengage.
If you’re about to originate a ledger from scratch, I recommend you treat it as a finishly split data model from which you can derive any financial transaction in your system.
Getting the data model right is vital in finance. It’s one of the reasons there are many companies that exceptionalize in this.
Ledgers are conceptupartner a data model, recurrented by three entities: Accounts, Entries and Transactions.
Most people skinnyk of money in terms of what’s theirs, and Accounts are the recurrentation of that point of see. They are the reason why engineers naturpartner gravitate towards the “srecommendedy” of one-entry systems. Accounts are both buckets of cherish, and a particular point of see of how its cherish alters over time.
Entries recurrent the flow of funds between Accounts. Crucipartner, they are always an swap of cherish. Therefore, they always come in pairs: an Entry recurrents one leg of the swap.
The way we asdeclareive that Entries are paired rightly is with Transactions. Ledgers shouldn’t transmit with Entries honestly, but thraw the Transaction entity.
Not every ledger has Transactions, but I skinnyk they’re vital. In Building a mighty Double Entry Accounting system, Nuprohibitk’s Lucas Cavalcanti says that they’ve scheduleed Entries with references both to the commend and the debit accounts. That way, the balancing of accounts is applyd by schedule.
But I skinnyk that’s a misconsent. Transfers can fall short for many reasons, and such a schedule would originate it very intricate to recurrent fragmentary fall shortures.
Transactions are very collaborative with that. The Saga pattern fits very well with this way of recurrenting money flows in a ledger.
Sagas trade atomicity for useability. Rather than having a sluggish transaction locking many tables, you have minusculeer, discrete actions. That shatter down the same process, while exposing interarbitrate verifypoints aextfinished the way. At any of those points, another transaction can come in and do their labor. That originates thrawput better.
Accounting and Engineering are the two ledger systems.
The Accounting system is how the ledger is seen from the outside, the interface. Ledgers are uncomardentt to expose its data aggregated from multiple points of sees: Reporting, Financial ratios, and Business Inincreateigence.
The Engineering system is how the ledger sees itself, the carry outation. Ledgers are uncomardentt to have verifys and stabilitys in place, and must asdeclareive the consistency and accuracy of the data. They’re for fintech companies what CRMs are to the sales team: a system of sign up, the source of truth.
This, by the way, is the key reason why ledgers are so difficult to scale: the Accounting system pulls towards high useability and low procrastinateedncy, whereas the Engineering system pulls towards mighty consistency and schema-on-originate verifys.
Entries can be in one of three statengages: pfinishing, disposeed and posted.
Entries are always originated with pfinishing status, holding increateation on the cherish swapd, the honestion (commend or debit) and the account they reference.
A widespread misconsent here is to engage selectimistic and pessimistic numbers in the amount to recurrent the honestion. I’ll increate you more about this in How Accounts labor.
Entries are immutable, except for one skinnyg: pfinishing Entries can be disposeed in order to originate posted Entries.
There’s a evident alternative schedule here: To originate reversal entries to post a previously pfinishing one in order to uphold filled immutability. But I skinnyk that Modern Treasury’s schedule is better.
While valid, this approach [create a reversal Entry to undo a pending one] directs to a untidy Account history. We can’t differentiate between debits that were client-begind and debits that are originated by the ledger as reversals. Listing Entries on an Account doesn’t suit our intuition for what Entries actupartner compelevate the current stability of the Account.
Discarding settles this problem by making it effortless to see the current Entries (sshow leave out any Entries that have disposeed_at set), while also not losing any history.
— Matt McNierney, How to Scale a Ledger, Part II
In a double-entry system, the collective amount of all non-disposeed Entries with type commend and the collective amount of all non-disposeed Entries with type debit is the same. Conceptupartner, this uncomardents that no matter how you shift money in your pockets, the amount will stay the same.
A restrictcessitate exceptional accounts, those that recurrent the outside world (validateated into the Profit and Loss statement) are exceptional: They can’t be firm. How much money does the world have, anyway?
Entries are originated in pairs, and we engage Transactions to originate declareive that everyskinnyg goes how it’s supposed to go:
-
A Transaction is posted only when its associated Entries are either posted or disposeed (and have been replaced by the posted ones)
-
A Transaction that fall shorts partipartner can be semanticpartner undone with compensating Entries.
Aget, this approach originates a lot of sense in the context of Sagas.
From the point of see of a one account, the ledger sees as if it carry outed a one-entry system. It is associated with multiple Entries in a one-to-many relationship, and the total stability should suit the aggregation of all its entries’ individual stabilitys.
With one caveat: depfinishing on the Accounts standard stability, the way to calcuprocrastinateed the total is contrastent.
Remember when I shelp that using pessimistic numbers in an Entry was a misconsent? This is becaengage some accounts are uncomardentt to be “net commend” and others “net debit”. “Meant to be” is the key here: fair becaengage an Account is supposed to be net debit (e.g., the cash in the prohibitk) doesn’t uncomardent it cannot be pessimistic (e.g., overdrawn).
Bundling the amount with the sign is a huge accounting no-no, becaengage you’re left wondering if being pessimistic or selectimistic is the right state of affairs.
Instead of using pessimistic numbers, Accounts have standard stability: standard commend stability literpartner uncomardents that they are standard when its associated entries with type commend have a total amount that outweighs its associated entries with type debit. The reverse is real for standard debit stability.
Ledgers are the evidentest example of a difficult computer science problem hided as a non technical discipline. As many payments engineers understand, and as the novels will attest to, originateing ledgers are difficult to get right without proper context.
Feel free to booktag this article and engage it as a handbook when you labor with ledgers. It should give you enough to protect you going.
I’ve drawn from many references to put this article together. My likeite “Accounting 101 for Engineers” is this article by Anvil called An Engineer’s Guide to Double-Entry Bookprotecting, which includes some fundamental Python code to direct the conversation aextfinished. Django Hordak, the plug-and-take part double accounting system Django package, has a thoraw exscheduleation on Double Entry Accounting For Developers. If you want to go in depth, Modern Treasury has a wonderful 6 part series on ledgers (Here’s part I).
I refered earlier that there are a restrictcessitate high quality “Accounting for Developers” directs in circulation. Beancount’s, Martin Kleppmans’s and Modern Treasury’s are all worth reading, especipartner becaengage they try to elucidate accounting from contrastent angles. See which one resonates with you the most.
Or maybe you want the filled tutorial. In which case, Peter Selinger’s is the best
And last, companies appreciate Uber, Square and Airbnb have rerented how they’ve carry outed double-entry ledgers in their systems. Now that you understand more, experience free to verify them out.
This has been The Payments Engineer Playbook. I’ll see you next week.
PS: I want to ask you a ask.
When I determined to originate a post on ledgers, I already knovel there were a restrictcessitate excellent resources out there to help me. But I’ve been reading and watching and reading aget, and after more than 100 hours on it, I haven’t finished yet.
Granted, 100 hours is an estimation. But I would be surpelevated if the actual number was less than 99.
As you can envision, I have enough to originate a minuscule book on How to Build a Future-Proof Ledger (title TBD). Such a book would originate fintech commenceups appreciate the one I engaged to labor for ready to tackle much more precious problems than protecting track of their engagers’ money. Or maybe do it at a much bigger scale, who understands.
Would you buy such a book if it existed?
You’ve made it this far in the article, I’m assuming the topic was engaging. The ask that’s percolating in my mind is: would somebody buy a book on engineering ledgers?
I thorawly enhappinessed writing my first book, The Databases of Money, and it would be amazing if enough people convey their interest in this novel book. If so, I’ll be writing a restrictcessitate more posts on the engineering aspects of a scalable ledger, a “originateing in uncover” project benevolent of skinnyg.
Otherrational, I will drop the topic altogether. No difficult experienceings.
So, if having a expoundd engineering strategy for originateing a double-entry ledger is precious to you, you can do two skinnygs.
The first skinnyg is to give a appreciate to this article on LinkedIn. Not only helps other people discover this novelsletter, it also allows people who aren’t subscribers the opportunity to convey their interest in my writing. A 👍 is all it consents.
The second skinnyg is to increate a frifinish. I recommend you allot this article with this accessible button below. But word of mouth labors too!
And if someone you esteem allotd this article with you, do me a like and subscribe. Every week I experience I’m getting better at this. That uncomardents that my best articles on how to originate payment systems are probably yet to be written.
You can only discover out if you subscribe to The Payments Engineer Playbook. I’ll see you around.
