Escaping the Matrix

When Beyond Work first embarked on developing a human-AI collaboration platform, the vision was something along the lines of: create a "chat, on steroids, for work" inspired by tools like ChatGPT and Claude. The idea was to build a platform where users could collaborate with an AI co-worker—and possibly other colleagues—to handle tasks too complex or varied for traditional automation but repetitive enough to be borderline inhumane for humans to do continuously. We decided to jump-start our development by building on top of the Matrix Protocol, an open network for secure, decentralized communication. While we bought ourselves some time by not building yet another chat service, it did not take long to find the rough edges of both the client and server offerings, and avoided “sunk cost fallacy” by quickly moving away from it.

Let there be chat

Whenever starting a new project, it pays dividends to evaluate “build versus buy” (or in the case of open source software, “use”) the various parts of your architecture. Ideally, one can “stand on the shoulders of giants” by figuring out how to combine other solutions in unique ways to solve the problem at hand. Other times it pays to build exactly what you need, how you need it. Using the Matrix Protocol as the backbone for our app exemplifies the former: there are mature and well maintained open-source clients and servers for Matrix written in many languages. These provide a lot of the plumbing needed to create a chat-like experience, so it was just a matter of finding one that worked best for us and adapting it to our needs.

The protocol allows for overloading of the messages going back and forth from the client to the server and back to turbocharge chat with additional functionality. We created an analog between Matrix “rooms” and our own “work blocks”, which represented a set of defined, repeatable, “work to be done” tasks. We envisioned users collaborating on these work blocks in real-time, discussing solutions and solving work together. Rooms would be created as there was work to do, and our users could join and leave as required.

Square peg, meet round hole

Of course, nothing is “free”, and as we started building we were confronted with both oddities and inefficiencies that started to slow our development. When one decides to not build one’s own solution, there will inevitably be mismatches between intended and required functionality. That’s the trade-off for not having to build it all in the first place. While not insurmountable, we began to feel that we were already accumulating a fair amount of “cruft” (a technical term) in our codebase to make the pieces fit.

As we rolled out our solution to customers, more limitations became apparent. The way we needed to scale operations was not a clean fit for how the various server offerings were designed. While Matrix excels at managing “chat-like” functions—such as handling rooms with multiple users, connections, and notifications—it struggles with creating and destroying rooms rapidly and managing the additional data we required. Matrix became a bottleneck.

This realization led us to reevaluate our approach. We started thinking about how much chat functionality was a primary concern vs the distributed, asynchronous and parallel processing of work. Desmond Tutu once said, “there is only one way to eat an elephant: a bite at a time.” While literally eating elephants is certainly not advisable, the analogy perfectly captures the approach to tackling large, complex tasks. While having a conversational interface with AI to complete work remains an important part of our application, we realized it was more of a side-effect. It was more important that we could “slice the elephant”, so to speak, and do it quickly and efficiently.

Matrix has exited the chat

Earlier this year–and less than a year after “breaking ground” with Matrix–we decided to start migrating away from the protocol and its OSS client and server solutions. We were already using Temporal.io for our workflow management and execution. Since we found ourselves really only using the “rooms” and “chat” functionality of Matrix, and “rooms” were really just a proxy to workflows, surely we could whip up just enough chat functionality to continue providing some level of conversational interface? Yes, we could.

Doing so was a good, practical reminder about writing layered, abstracted software where one solution could be plugged in as a replacement for another. It was also an exercise in, as Reid Hoffman puts it, “[assembling the] airplane on the way down” as we were also going live with our first customer at the same time. Thankfully, we were able to do so without major glitches, and the overall speed increase made the extra effort worthwhile.

Is Matrix a bad protocol? Are the OSS solutions implementing the protocol bad? No. They’re probably fine if you’re building a chat client/server solution. Can they be extended to do other things? Probably. Building on Matrix initially gave us a jump-start, and some of the early limitations helped define what we needed and did not need. It was alway possible that we'd need to move away from Matrix–or at least the OSS implementations we were using–to something built specifically for our use case.

As Warren Buffet once said: "Don’t keep making the same mistake just because you spent a long time making it", and be it investments, building a startup, or writing software, willingness to pivot from something you know isn't working to something that might work often pays dividends.