Tom Rikert is the co-founder and CEO of Masterful AI, the training platform for computer vision that helps developers build models faster and with much less labeling. He is a former VC and invested in enterprise software and AI/ML at Andreessen Horowitz. Before VC, he held product management roles at Wildfire (acquired by Google), YouTube, and Autodesk. He began his AI/ML journey at MIT and started his career as an engineer at Silicon Graphics. Tom lives in the Bay Area with his wife and daughter and loves going fast, whether on a mountain bike or flying racing drones.
My conversation with Tom was recorded back in May 2022. Here is the note from Tom regarding updates with Masterful:
The latest at Masterful AI is that we’re launching a new generative AI product. We saw a need to make generative models more customizable and more reliable, so companies can trust them for real business applications. We’re starting by enabling companies to tell a more vivid and personalized story about their products at scale.
Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.
Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email khanhle.1013@gmail.com.
Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:
If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.
Here are the highlights from my conversation with Tom:
I am from a small town on the Hudson River Valley, on the outer edge of the New York City sphere of influence. I was always interested in disassembling things and seeing how things work. It was interesting because IBM headquarters was a couple of hours south. Some people in our area worked for IBM, but my parents did not. My dad sold advertising for a newspaper, and my mom was a secretary. I was the only person digging into this stuff on computers.
In the early days, I loved the things that were a little odd. I loved model airplanes and spent hours researching different components of airplanes that could fly. I spent a few years building one in my basement, but there was nowhere around where I could fly it. I would read up on all these kinds of stuff and build the components (the engine, the electrical system, the airframe, etc.), but it was not until years later that I finally had a friend who was able to take me to his farm and get the airplane in the air. I learned a lot of patience to see a project through.
Similar with scuba diving: I was fascinated with scuba diving, even though I was hundreds of miles from the ocean. I subscribed to scuba diver magazine for five years through elementary and junior high school and learned everything possible about scuba diving. But I did not get to go scuba diving until I went to college in Boston as part of a PE class at MIT. I wanted to know how things work and was willing to go through it in my mind. Even if I could not do it firsthand, it was satisfying later in life to finally do something I had imagined.
I felt fortunate to go to MIT, which was my dream. It was rare for people from my school to leave town and do that kind of thing. I really appreciated that MIT did not really care where you came from and only cared that you were passionate about building and inventing things. It was also a place where everybody suffered together. If you did not come in humble, you would quickly get humbled. It helped people bond. For me, I love that MIT was a very pure environment.
On the research side, I was part of the MIT Lab, which is called CSAIL now. My specialty was computer vision. Back then, I was involved in making a number of innovations to use statistics (like classical computer vision) to do object recognition (like detecting faces, cars, and so on). In the early days of deep neural networks, I worked on techniques to understand images at different resolutions. One of the cool things we did was use texture analysis. One of my papers was about the synthetic generation of faces with that technique. Today, this can be done very well with GANs (Generative Adversarial Networks), making it look beautiful and real. But my early version was more of a Frankenstein version, in which we would composite features from different faces into a new face.
Those were very formative for me to understand how computers understand the world through vision, which is the highest-bandwidth way of digitizing the world and helping machines understand. That hasbeen a theme throughout my life: how to make this computer vision stuff works well for many more people and industries. That led me to start Masterful AI in the last couple of years.
I was planning to continue on the research side after MIT and do a Ph.D. in computational neurobiology. I deferred on that program and decided to go to Silicon Valley, which led me to do other things. But for a period of time, I thought, let's even go further into the wetware and study how the brain works.
I was fortunate to be in Silicon Valley, which is now like the Renaissance period in the 1500s-1600s. The concept of Silicon Valley, not just the place, is changing the world forever. There was an excitement being here on the ground in the early 2000s. We were changing how life works in many dimensions. The West Coast is more open to trying something new and not afraid of failing. That was a different mindset from the East Coast, which tries to improve upon what has worked in the past.
Silicon Graphics was the place to be at the time with many top engineers. They were building both hardware and software systems, generating graphics for movies like Jurassic Park and Terminator 2. The challenge was about how to get the best quality visuals. I learned how to bring the most performance out of the system. I was working with people who are designing compilers to compile code more efficiently and designing the instruction set for chips to be able to execute code more efficiently and quickly.
I focused on parallel systems to make an early version of the metaverse - an online design space for people to collaborate. It had to be almost real-time and work across even slower connections back then. The bottom line lesson was that: you have to get into the weeds. You cannot rely on the abstraction layer solely. You must understand how it works under the hood and take a systems view to get the performance out of technical and people systems.
I helped build a platform at Autodesk that delivers location-based services tracking where your cell phone is. One of the biggest challenges was working with wireless carriers so that they would enable this platform and help them communicate to their users (why it is important and helpful to share your location and not a violation of privacy). There were not so many features. In fact, it was more of helping to paint the vision and evangelize why location services were so important and why the end user would be willing to share the location. Today we take that for granted. Back then, it was a big high-risk decision in the eyes of these carriers.
Secondly, iOS and Android were not around yet. Different mobile operating systems like Brew and J2ME from different companies supported our platform. The range of hardware was very complex. We had to find a way to make the user experience as simple as possible - whether you are doing a text-based workflow, a web browser, or a downloadable app. For me, that was a great experience exercising muscles around UX design thinking to solve a problem for the end user (no matter what device they were on).
I had not planned to attend business school earlier in my engineering career. But over time, I began to appreciate some of the skills I observed in people who happened to have gone to business school around communication, leadership, and a holistic understanding of problem-solving. Half of the value at HBS was in the classroom experience - just looking at lots of examples of different businesses in different situations and discussing them with smart people.
The other half of the value was being around amazing people from different backgrounds, places in the world, and industries. I could compare notes and get a crash course on how the world work that I had not seen in Silicon Valley. That was personally satisfying to meet new people and see the world through different eyes.
At Google, I led a team called Product Specialists, who served as the voice of the customer and product development process. They worked closely with product management and engineering teams to help build the right products and help users succeed. We had a great culture on that team, and I am excited to see many people from the team going to do amazing things - becoming directors and VPs at leading tech companies.
One example of products I am proud of is building the sponsorship marketplace at YouTube, a platform that enables a brand to find brand-safe content on YouTube to advertise against. For instance, Nike wanted to put their ads against the video and make sure the video did not do anything dangerous to their brand. This was one of the critical issues we had to solve in the early days.
At the time, finding an automatic way to do that was hard. How could you look over hours of user-generated content? We had done some early experiments with different tools in the early days of computer vision to be able to do that. We found at least some content that would be brand-safe to get the Nikes, the NBAs, and so on to run a couple of campaigns. After getting positive results, we bootstrapped it into broader campaigns and automated them along the way.
I learned that you get everything to prove to customerswhen you are a first-time founder. I came out of Google, where there was a given momentum, a nameplate, and a willing audience to trust. As a first-time founder, it was a great trial-by-fire to be at the front line talking to a skeptical customer or user and be able to get them excited. I remember talking to one sale prospect who had sent me some questions. I took two days to get back to him. Once we got on the phone, he said: "Hey man, taking two days to reply to me is not good enough. You are lucky that I am taking this call with you." That was a good lesson to learn early on: when you build something new, it is all about finding people who believe in you. Everyone will have a high bar to start with, so you have no goodwill to start at zero.
I had always been looking for the right time to build something. I have seen a number of experiences from being an engineer, a product manager, and a team lead. I love integrating these experiences and building something from scratch that is meaningful. Thanks to the lens of YouTube and video, I saw an opportunity in social media. This was in 2009 when social networks (Facebook, Twitter, etc.) were becoming more interesting for marketing purposes. I thought this would be a great time to build a company with a product that would help connect users with different brands that were coming online. At first, I thought it would be helpful from a commerce point-of-view to help people find the best deals they were looking to buy. But then I later found out it was too early. People were not buying the social networks, but it was the marketing that helped brands build and engage with their audience. That is what helped me get connected and eventually join Wildfire, which is a start that focused on marketing.
It felt really fortunate to be part of Wildfire, which had a great team and culture. We scaled it from 10 to 450 people over two years. It was a hyper-growth experience for me: we opened seven global offices and added some pretty sophisticated product features around analytics. I learned the fundamentals I brought later to being a VC and an entrepreneur - seeing what it takes to scale something quickly while maintaining a great culture and making it a place where people are empowered to experiment.
Wildfire had a culture of empowerment and allowed people who are sometimes junior to take on roles they could step in and start leading. We had a SalesOps team with people who were mostly in sales and wanted to do something a little more technical. They started building some tools to help their fellow salespeople do things more quickly. Seeing people staying late in the office, coding up simple webpages, and sharing them with their team was amazing. The culture is a meritocracy, where people can benefit from it. People at Wildfire was able to grow their careers and have more opportunity than they would have otherwise.
After Google had acquired Wildfire, I boomeranged back to Google and helped integrate Wildfire into some of Google's ads products. I got a call from Marc Andreessen, who said: "Hey, we are looking for someone to join the enterprise investing team and come across you. I am excited to talk!" It was serendipitous and, fortunately, worked out. I focused on enterprise applications, AI/ML, and frontier tech (drones, 3D printing). It was a broad swim lane to look at new things, and I was able to learn from working with great people.
One of the harder things to adjust to as an investor was taking a much longer timeframe view: What will this company look like 5 to 10 years from now? How is this team going to scale? (not just in terms of size, but also the ability to think in new ways). Every company becomes a brand new one after each stage (seed, A, B, etc.) From my own experience, I had been a founder, went through the scaling journey at Wildfire, and worked at a big company like Google. At a16z, I got to connect the dots and see the ramps between different points - looking at it from the multi-year perspective and knowing directionally the structural things that would help or hurt a company.
I was excited to be an entrepreneur in the sense of wearing a VC hat. NextWorld Capital was a new firm in place to grow. It felt like an opportunity to be an entrepreneur and build a portfolio, team, and platform. It was a firm specializing in enterprise and looking to get involved with AI/ML and frontier tech. While I was doing great at a16z, they were already very established. The firm also focused on how to service entrepreneurs by pioneering value-added services to founders.
Furthermore, NextWorld had the aspiration to further its platform and help companies expand into international markets. I had a unique take on international expansion thanks to my operational experience. Having worked at Autodesk and with wireless carriers worldwide, I saw how big those opportunities could be outside the States.
One thing I looked for was: Is this AI application augmenting or replacing workers? Being able to help people do their job better is essential. It is a better way to get your product to market versus setting a much higher bar to fully automate something or replace a person entirely. The bigger issue is: How do you create opportunities for people and businesses? That is where you can get into the virtuous cycle: By giving people better tools, they can do their job better, which leads to the creation of even more advanced tools.
Another example I looked for in companies was where they would sidestep having to do any legacy infrastructure integration. If you have a new tool, can you avoid having to rip and replace things already deployed and instead sit adjacent to them? You do not want to tell big old companies to change their status quo. People become very passionate about how they do things in their workflow, especially in big enterprises. to the extent that you give them something new and helpful (versus asking them to change what they already knew and loved), you will find a lot more receptivity and grow your company faster.
While at a16z, I invested in companies that were building ML and computer vision models in the drone, mapping, and self-driving cars space. I kept seeing a problem: It was much harder to get these models to work in production once you took them out of the lab. They would break in the real world. They were never accurate enough and took a ton of training data for labeling and experimentation.
My two co-founders, who have known each other for years, regularly compared notes. Sam had been part of the Google AI Research team and had seen similar inefficiencies around experimentation cycles. Yaosh had been building deep learning projects on his own. He is a software engineer who had picked up deep learning and seen how hokey this was compared to how software engineering ought to be.
We started to think about one of the big blockers that made ML development so different from software development. We designed a product we wanted to use ourselves and started talking to folks in various companies to see if it resonated with them. We were excited by the positive feedback and went out to build it.
We intentionally stayed in stealth mode for about two years. As we talked to many design partners, we wanted to build a platform. There was value in building a more unified platform than just a simple tool. We recognized that the problem we wanted to solve was a system issue: How do you combine the right training data and the experimentation cycle in a compressed and automated way? We also saw this as a way to ensure our platform did not require tuning/customization and would only work for certain kinds of data. We wanted to test it to work on any image or video data you give it. We made that investment upfront, so the platform became complete enough, and then we made it available to everyone.
An important element in computer vision is getting the right training data to build the model - whether for classification, detection, or segmentation tasks. You get labeled data today through an army of human labelers who bring their understanding of the world and their biases/errors to that labeling process. This is also expensive and time-consuming. Some of our customers said they have multiple labelers going through the same data to ensure correctness.
Our point of view is: What if you could use unlabeled data and reduce the labeling effort? Recently, there has been the emergence of semi-supervised learning, a field of research where you can group similar-looking images together and assign a label to the group. This dramatically reduces the amount of hand labeling. We saw this as an opportunity: Could you label just 1/10th of the data you label today and, instead, take the massive volume of data you are probably collecting and pipe it directly into your training loop? That will (1) lead to a more accurate model since the model sees more nuances in the world and (2) reduce the cost and time to label so you can iterate more quickly. This gets back to our mission of making ML development more like software engineering.
Secondly, ML experimentation requires trying different training parameters/hyper-parameters. If you get them wrong, your model might not converge, and you will have to do the whole loop again. These are acute issues for many ML developers. We have built our solution with an easy-to-use API, where you can pass some labeled data and any unlabeled data you have, then get a trained high-performance model from Masterful.
The opportunity I am excited about for our next step is helping maintain models at scale. I talked to many developers who said they have to babysit all their models in production since their performance degrades as the data drifts (the model sees new data in the real world that it has not seen before). We see companies developing tens or hundreds of models they put into production, so Masterful will be the platform to retrain and improve model performance over time automatically.
A part of our advantage is that we can use unlabeled data. There is a loop from the data that the model sees in production to the phase when the model automatically retrains. That does not involve many people to label and QA the data. The ML world is moving quickly past the initial experimentation phase to the scaling phase with multiple models in production all the time, where they need to meet a critical bar performance.
The world is at a unique inflection point. We have started to trust machines to do very important things (drive our cars, diagnose medical images, etc.) It is important to make these machines fair, accurate, and trustworthy. I believe Masterful can be the company to do that. In our interview process, I look to find people who have a missionary view and believe AI will make the world a better place. What is their WHY behind being in the AI field? I have been doing this for a long time, and this is a consistent, authentic passion. But many people just got into AI more recently because they think it is a cool new career path. Nothing is wrong with that, but I prefer missionary folks to mercenary folks.
Secondly, I also look for curious people. Building a startup is hard and requires a lot of persistence. Curious people are humble to admit their ignorance and commit to finding the answers. That is important to us as a company and to me as a founder - getting people who will further shape the culture as we hire and grow over time.
We want to find teams that are hitting a real acute pain point. Many people are curious about AI and experimenting with it, but good design partners and early adopters must be folks who hit a brick wall. They built a model and could not launch it in production because it was not accurate enough. It is expensive to continue progressing forward because they need to label more data and hire more researchers. That is when they are open to finding a new tool or platform that would enable them to break through this wall.
Furthermore, we also want design partners who are curious. They might have heard of the ML techniques we are using and want to try them. But they were not able to develop or combine these techniques themselves. They wanted to collaborate with us and help us shape what capabilities we have by being expert testers of our product.