February 23, 2024


The success of large language models like GPT has sparked a frenzy among developers eager to build AI applications. But building AI services can be tricky, especially due to the lack of skilled developers to meet today’s growing demands.

that’s there Chaoyu YangEarly software engineers from data giant unicorn Databricks joined.Together with his co-founders, he built the AI ​​development framework BentoMLjust announced a seed funding round.

In an interview with TechCrunch, Yang explained that today’s AI services are often built on multiple machine learning models, which complicates their management and operation. Many of the competing programmers have full-stack or application development backgrounds, meaning they often lack the skills to build the required AI infrastructure, resulting in a lengthy development process.

Demo AI applications like Microsoft’s Visual ChatGPT, an updated version of chatbots that can generate responses based on text and image prompts, could take at least three to six months to go into production, Yang said.

While tech giants like Microsoft have the financial muscle and human capital to train AI models and apply them in the real world, small businesses are collecting, in Yang’s words, “precious data that can greatly benefit from AI,” But “lack of resources” to build infrastructure for development. “

BentoML provides a high-level API that abstracts away the details of the infrastructure required to run an AI model on the cloud, Belongs to the camp of tools like SageMaker that hope to pave the way for developing AI services. It’s a so-called AI application framework, a set of tools that make it easier to build, deliver, and scale AI applications, much like a building kit people use to build a house.

Specifically, BentoML targets data scientists who train AI models, DevOp engineers who manage their lifecycle, and developers who actually build applications on top of the models.

With BentoML, developers can make Visual ChatGPT scalable and cost-effective for production use in just two days, Yang said. Users also use the framework to run art generator Stable Diffusion and open source LLM In the clouds.

Yang compared his company to Vercel, which focuses on serving front-end developers and was last valued at more than $1 billion. BentoML aims to be the Vercel of artificial intelligence, he said.

While Yang predicts that AI will eventually become more production-ready, he admits he doesn’t think the wave of AI adoption will come so soon. The founders predict that AI application developers will account for more than 90% of platform users in the future.

“If you had asked me a year ago, I would have said that maybe 90% of companies would train their own models, but the underlying models that have emerged recently are so powerful that they perform well even given datasets they have never seen before. Yes,” he said.

“Developers don’t need to focus on model training now, but only on fine-tuning and product engineering, which itself constitutes a bottleneck because there is a shortage of AI-focused developers.”

BentoML was open-sourced in 2019 and has since rolled out a self-hosted SaaS version to enterprise customers. It has been acquiring users organically through its open-source community, which has quadrupled its membership to more than 3,000 in the past year, with Korean social networking giants Line and Naver among its early adopters.

Yang declined to disclose the size of the company’s revenue.

Investors are noticing BentoML’s traction in the developer community. The startup recently raised $9 million in a seed round led by DCM Ventures, with participation from Bow Capital. Hurst Lin, general partner of DCM, joined BentoML’s board of directors after this round of financing.

Yang acknowledged that the booming AI market has been a boon for BentoML, but the rapidly changing industry also made it difficult for the team to balance short-term and long-term goals.

“You might have to build something that follows current trends, but in the long run, of course we want to have our own moat. The question is how do we balance time and human resources between the two.”