I’m writing this blog post to summarize to ourselves, employees, customers, and investors why we’re pivoting away from providing usability testing for developer tools through Usabl. Stephanie and I started working on Usabl in July 2021 while I was still at Microsoft. I was inspired to start the business because I had noticed my team building lots of features that no one knew how to use, and I wished there were an easy way to quickly show my team evidence that developers were struggling.
We signed our first customer, Courier, in September 2021, raised a pre-seed round led by Haystack and I quit my job at the end of the month. I moved to Chicago in October so we could work on the business in-person. Subsequent sales momentum was slow but we signed Parabeac and Dapper Labs to monthly subscriptions in March 2022 and started paid pilots with AWS Amplify, Cloudinary, and Okteto in March as well.
Why didn’t we pivot sooner?
There were a number of signs that this business might not work well that we chose to ignore:
Haxor, a company that offered virtually the same service as we did had pivoted away a few months earlier
UserTesting and UserZoom both had relatively low valuations (around $1B) after working for 10-15 years
UserTesting’s stock price dropped significantly after their IPO in November
Our bottom-up market sizing for developer-first usability testing always always estimated the Total Addressable Market (TAM) at ~$100M, even with generous assumptions, making the business heavily reliant on an “Act 2” for growth
Our TAM calculations assumed we could price as a Software-as-a-Service (SaaS) (70% margin on top of tester fee), but customers valued us like a labor marketplace (20% of tester incentive as a “rake”)
We had to do significant amounts of manual work to make the business “go”:
Long sales processes as we attempted to convince businesses that didn’t already do usability testing to use us
Helping customers set up tasks
Setting up sample environments for customers
Contacting testers to:
Arrange payments
Remind them to do tasks
Provision accounts
Debug failures
Manually reviewing videos to help customers understand their results
In each case, we felt that we might know something that other companies were ignoring:
Haxor had focused much of their energy on building out a client application that would record every keystroke a tester made during a task. We felt that our key value add was in our labor pool and in the way we presented results back to the customer
UserTesting and UserZoom were both focused on helping B2C businesses with user experience testing, meaning they had slimmer margins and struggled to differentiate. Our focus on developers would open up an entirely new market for us
UserTesting’s stock price dip was a sign of broader economic headwinds, not indicative of any fundamental issue with their business
TAM is more of a theoretical exercise anyway; if we could just figure out how to get to $1M ARR we could do the rest
We could still make the labor marketplace pricing work, we’d just have to expand into offering other kinds of services, like sample code creation or monitoring of StackOverflow and community forums. Karat and HackerOne seemed to have been able to do this for recruiting and security research, respectively.
We could automate the manual work or offload it onto the tester by building up marketplace density and having testers label their own videos
Ultimately, though, as each slog of a sales conversation turned into a slog of a customer success motion, we were unable to convince ourselves that our business was sound. While we’ve validated that it’s possible to build a profitable developer experience consulting business (probably wrapping Upwork rather than a standalone marketplace), we don’t believe this business can get to venture scale on its own.
What did we do well?
Quick iterations - we avoided building large numbers of features without getting customer feedback on those features and (with the exception of our GitHub action), avoided building features that were not going to be used by paying customers
Minimal burn - we paid ourselves low salaries and strove to be good stewards of our investors’ money. We also ensured our margin was non-negative for all sales deals
Careful hiring - our first hire lacked a formal college degree but was exceptionally well-suited to the startup environment, shipping code quickly and consistently sharing new ideas for taking the business forward
Customer retention - we never churned a single subscription customer. This focus on customer satisfaction is going to be key to success of any future business
Great investors - our pre-seed investors (especially Haystack) have been phenomenal partners to us even through the ups and downs of the past few months
How did Usabl prepare us for what’s next?
Sales expertise - there’s nothing like selling the wrong product to help you get better at sales. Using resources including paid consultants, podcasts, a close.com eBook to this Tyler Bosmeny talk and David Sandler’s You Can’t Teach a Kid to Ride a Bike at a Seminar, we developed a practical understanding for the key steps of an enterprise sales motion:
Qualify leads/understand pain
Demo the product
Close the deal
Team chemistry - because we met online and have known each other less than a year, the risk that Stephanie and I would go separate ways has always come up in later stage diligence with VCs. Though we’ve definitely had our disagreements (for example, the disagreement about product direction that resulted in Stephanie becoming Head of Product), we’ve weathered those disagreements and come out the other side with stronger chemistry and mutual trust than we had a few months ago
Entrepreneurship 101 - we acquired a large number of transferrable skills that anyone setting up their own business will learn, including:
how to incorporate
how to set up banking/credit cards/bookkeeping
digital marketing strategies and channels
hiring/firing employees and contractors
how to fundraise and manage relationships with investors
What will we do differently?
Relentless TAM focus - while I’ve heard at least one founder call TAM “the stupidest f***ing thing ever,” I’ve realized that a >$5B bottom-up TAM you believe in can be hugely helpful:
Selling the business - it’s way easier to win investors and potential employees if you’re convinced you’re operating in a market with massive growth potential
Qualifying buyers - a good bottom-up TAM exercise forces you to identify a) who your economic buyer is b) how much budget they have for the problem you’re solving and c) how many of this economic buyer there are. Surprise, surprise, all three of those criteria are the first steps in sourcing enterprise software leads. So if you do TAM sizing well you’re also ready to kick off your enterprise sales motion
Stronger core - this is not just a personal fitness goal. One of the key takeaways from Usabl is that we were telling a growth story that was largely focused on the business’ “Act 2” - how we’d ask customers to integrate a Usabl server-side logging client so we could grow into being an observability tool rather than simply a usability testing platform. Generational software businesses start with an idea that is massive on its own. Think Google with Search, Stripe with payments processing, Figma with mockups. We want the first problem we solve to be an obviously big market on its own.
Know the signs - Semil at Haystack always talks about “intellectual honesty.” One of the challenges with an idea that sat in the uncanny valley of product-market fit was that we had enough traction to talk ourselves out of most objections to our business model. The one sign that we definitely should have paid more attention to, however, was that we didn’t have a crystal clear understanding of how more money would grow the business faster. We had some ideas that sales+marketing would help grow labor marketplace density but not the typical urgency of “our systems are unable to scale to handle this load and we need to hire 5 engineers yesterday.”
We’re grateful for everyone who has embarked on the Usabl journey with us in the past few months. We’ve grown as individuals and as a team, and we feel excited for our next chapter. If you’d like to stay up to date on our progress, feel free to subscribe:
Resources for current customers
We have loved working with all of you and appreciate having had the opportunity (however brief) to identify and resolve developer experience issues in your products. While our new business is unlikely to solve your problems, we’d recommend checking out the options below to continue improving your developer experience.
Consultancies/Full-stack user testing
Hoopy is a “developer relations and experience consultancy” that hosts DevRevCon every year. They offer a “developer experience” service.
UserTesting and Userzoom also offer research hours to enterprise customers
Userbrain is a slightly cheaper offering
Maze is good if you are testing just Figma prototypes
Participant recruitment
Three options we recommend for finding participants to recruit for user research are:
Upwork - post a job of the form (XX min, $YY) - <Adjective> developer needed for a XX min research study
UserInterviews - Use their B2C recruiting option ($45/candidate) but use your screening questions to confirm that participants know how to code. I.e. “Which of the following is not a built-in Unix command: man, curl, ldap, sed”
GreatQuestion - YC startup that lets you recruit participants from your existing user base
Server-side logging
Moesif has a server-side logging tool for APIs that can provide some usability feedback but we’ve heard it can be quite expensive
Readme also has a server-side logging client so if you’re already using them for docs there might be some benefit in adding their logging client to your API
Surveys
There are a number of companies that will help you run surveys inside your web properties, including:
Delighted (now part of Qualtrics) for general NPS surveys
Sprig (formerly UserLeap) - does some cool integrations with reverse ETL tools to do contextual “microsurveys” so you don’t have to ask the same question to all your users
Hotjar - also does recording of user sessions in the browser
Evidence management
Lookback, Rewatch - great tooling for collaboratively reviewing recorded videos
Dovetail - UX researcher-focused platform for reviewing video evidence and interview transcripts
Productboard - central hub for feedback from customers, support teams, and sales teams that can easily be turned into action items
Thanks for sharing your learning, best of luck in Act 2!