Startup Trelent speeds source code documentation by 4x with Azure OpenAI Service

Trelent uses cutting-edge AI algorithms to provide intelligent source code documentation for its clients. Although the company is relatively new, Trelent has found its audience: high-growth startup and enterprise tech companies that need quality documentation to onboard new engineers. To scale its solution and accommodate the data security and residency needs of enterprise customers around the globe, Trelent chose to migrate its solution to Microsoft Azure OpenAI Service. After a smooth transition, Trelent has seen promising returns, including significant improvement in response time thanks to distributed regions in Azure.

“With a product like OpenAI Service behind you, you can focus a lot more on what really matters, which is delivering a great experience, a great product, and a lot of value to your customers.”

Calum Bird, CEO, Trelent

Helping developers keep developing

Trelent was founded to “let developers keep developing,” a goal it accomplishes by automating code documentation on their behalf. Still in its pre-seed period, the startup has found an audience it feels well-positioned to help. “We started talking to and helping individual developers, but quickly realized that documentation is much more of an issue within a larger organization where there may be thousands of engineers touching documentation,” says Calum Bird, Founder and CEO of Trelent.

High-growth companies and organizations with distributed teams need quality documentation to ensure a consistent understanding of their code as they onboard new engineers or work across regions. Pivoting to focus on meeting this need made sense, but this left Trelent—which had just two employees at the time—facing the question of how to provide the security, stability, and regulatory compliance that enterprise organizations expect and need.

When Bird attended a Microsoft for Startups webinar, he found the answer: Microsoft Azure OpenAI Service—a service pairing the OpenAI API with the security, compliance, and regional availability of Microsoft Azure with a library of large, pre-trained models. Since Trelent uses the OpenAI Codex model for docstring generation, Azure OpenAI Service represented an opportunity to get access to enterprise-ready features without having to re-engineer the company’s core innovation. The simplicity of the switch made it a compelling option. “Moving to OpenAI Service was so easy, in fact, ultimately in our code base it came down to changing a few lines of code, which sounds ridiculous considering the benefits we gained," says Bird.

Those benefits include the comprehensive security and regulatory compliance built into Azure, meant to both protect business customers from bad actors and encourage responsible use of AI. After the straightforward process of switching the Trelent backend to refer to OpenAI Service, Trelent went into testing to make sure all the safeguards and precautions worked without impacting the quality of generated docstrings delivered to customers. From there, Trelent worked on a rollout plan, providing the newly upgraded service to authenticated users and monitoring performance.

“Moving to OpenAI Service was so easy, in fact, that ultimately in our code base it came down to changing a few lines of code, which sounds ridiculous considering the benefits we gained.”

Calum Bird, CEO, Trelent

Accelerating performance and satisfying customers

Beyond the improvement to the company’s business offerings, the switch to Azure OpenAI Service brought Trelent significant performance gains. Rather than operating out of only North America, Trelent can take advantage of global coverage with Azure to cut down on response times. “We have users across 50 countries, so the availability that came with multiple regions, and the benefits on top of that that just didn’t exist in any other platform—it really made the move to OpenAI Service a no-brainer,” says Bird. Where previously, requests could take a second—occasionally as long as three seconds—to generate, the average is now down to around 750 milliseconds, a small but impactful change when an individual developer might make dozens or hundreds of requests each day.

Qualitative feedback from developer clients has been positive, especially with the speedier response times. Clients share that they’re more inclined to keep using the Trelent service as a result, and Bird says the company’s internal metrics have backed that up—retention, referral, and usage rates are climbing.

For Trelent, the transition to OpenAI Service has simplified the company’s growth and expansion. Without having to build their own security and compliance features, Trelent engineers are free to dedicate their time to the company’s core innovation: the prompts and controls that shape the docstrings generated by the Codex model. “With a product like OpenAI Service behind you,” says Bird, “you can focus a lot more on what really matters, which is delivering a great experience, a great product, and a lot of value to your customers.”

Find out more about Trelent on LinkedIn, Discord, and GitHub.

“We have users across 50 countries, so the availability that came with multiple regions, and the benefits on top of that that just didn’t exist in any other platform—it really made the move to OpenAI Service a no-brainer.”

Calum Bird, CEO, Trelent