The pitch deck is the least useful part of an agency evaluation. Every deck has the same logos-and-graph slide, and the case study numbers are almost always the best result the agency has ever produced, presented without the starting position, the budget, or how long it took. You learn more from twenty minutes of pointed questions than from any deck.
So this is a guide to the questions. Most of them are designed to find out one thing: whether the agency understands that software is bought differently from everything else they sell marketing for.
Why the generalist problem is so common
Agencies follow demand. SaaS budgets are large and recurring, so almost every general digital agency has added a SaaS page to its site. That page usually says something about product-led growth, shows a dashboard screenshot, and lists the same six services the agency sells to dentists and law firms.
None of that is fraud. The services really are the same at the category level. Everyone does content, technical SEO, paid, and reporting. The difference is in judgment, and judgment only comes from having been wrong a few times in your specific market.
An agency that has run marketing for a self-serve developer tool knows that gated content will destroy your signup rate and that your best-converting page is probably documentation. An agency that has run marketing for a $60k ACV compliance platform knows that nobody signs up, that the conversion event is a demo request from someone who has already read your G2 profile, and that six people will need to be sold internally after that demo. Ask a generalist which of those two situations you are in and they will describe your funnel back to you using the words on your own homepage.
The questions that expose a generalist
Ask these in a live call, not over email. You are listening for whether the answer is specific or whether it retreats into the abstract.
What is our ACV, and how should that change what we do first? A good answer treats a $400 annual contract and a $70,000 one as different businesses. Low ACV means volume, self-serve, and content that gets someone to value without a human. High ACV means fewer, better-qualified conversations and a lot of content aimed at people who are not the buyer but can veto the buy. If the answer is the same either way, they have one playbook.
Which of our competitors' comparison pages are beating us, and what would you do about it? Anyone serious will have looked before the call. The follow-up is more revealing: ask what they would do about a competitor whose alternatives page outranks yours and has done for two years. "Build a better one" is not an answer. Look for something about earning mentions on third-party sites, because that is usually what actually moves a comparison query.
What will you report in month two, when there are no results yet? The honest answer is leading indicators and work completed. Agencies that promise a revenue chart in month two are either lying or planning to attribute your existing organic traffic to themselves. This is the question that catches the most people.
Who actually does the work? Ask for the names of the people who will touch your account and how many other clients they carry. The pitch is often run by a partner you will meet twice. Nine clients per strategist is normal. Twenty is a content mill.
What would make you turn down this engagement? An agency with a real point of view has a list. Not enough budget to matter, a product with no differentiation, a founder who wants rankings rather than revenue, a six-month runway. If they cannot think of anything, they are optimising for closing you.
What the first 90 days should look like
The honest version of a SaaS SEO engagement is that the first quarter mostly produces work you cannot see from the outside. That is not a stall. Technical fixes, an actual keyword map tied to funnel stage, rewriting the pages that already rank badly, getting analytics to the point where a demo request can be traced to a query. Anyone who tells you otherwise has not done this. Our own write-up on how long SEO takes goes into the timeline in more detail, and the shape is the same for software.
What you should see by day 90 is a keyword map you disagree with in places, which is a good sign, because it means they made choices. You should see the first pieces of bottom-funnel content live. You should see whichever technical problem was suppressing your existing pages fixed and verified. And you should be able to open a report and find the phrase "we were wrong about" somewhere in it.
If month three arrives and the deliverable is four blog posts about industry trends, you have bought content marketing rather than growth.
The AI answer problem nobody priced in
Here is the part of agency selection that changed recently, and where most of the market has not caught up.
A meaningful share of software evaluation now starts inside ChatGPT, Claude or Perplexity rather than Google. Someone types a description of their problem and gets back three or four named products. If you are not one of them, you never enter the shortlist, and no analytics tool will tell you the search happened. This is the mechanism behind organic search quietly shifting under everyone's feet, and it does not show up as a ranking drop.
Being cited in those answers is a different job from ranking. It depends heavily on being mentioned on sources those models trust, which means the work looks more like digital PR and placement than on-page optimisation. Ask any agency you are evaluating what they do about it. A lot will say they are "watching the space", which is a polite way of saying nothing.
We are not neutral here. This is the single biggest change in how our own clients get found, and it is where our own inbound comes from. A founder asked ChatGPT for someone to finish a half-built application, our agency was named, and the contract was signed three days later for €20,000. That is one data point, but the mechanism behind it is not luck. It is the same thing that gets any brand cited: relevant articles on your own domain aimed at the exact question, plus mentions on other sites the model already trusts.
Contract terms worth arguing over
Three things matter more than the monthly number.
Own your assets. Content, accounts, tracking setup, and any tooling configured on your behalf should be yours on termination, and the contract should say so. Agencies that host your blog on their own subdomain or run your ads from their own account are building a hostage situation, usually without meaning to.
Keep the notice period short at the start. Thirty days for the first six months is reasonable. Twelve-month lock-ins with a 90-day exit exist to protect the agency's revenue forecast, not your results, and any agency confident in its work will take the shorter term at a slightly higher rate.
Get the team named in the statement of work. Not "a senior strategist" but a person. It is the single most effective clause against the bait-and-switch where the people who won the pitch are never seen again.
One thing not worth arguing over: performance-based pricing. It sounds like alignment and almost never is. The agency ends up optimising for whatever the contract measures, which in practice means branded traffic, easy keywords, and demo requests from students. Pay for work, measure outcomes separately, and leave if the outcomes do not come.
A shortlist of three is enough
Talk to three agencies, not eight. Past three the answers blur, the process eats a month, and you start choosing on rapport because everything else has stopped differentiating. Pick one generalist with strong fundamentals, one specialist in your exact ACV band, and one that is unusually good at whatever your biggest single weakness is.
Then give the shortlist the same brief, with a real number in it, and see who pushes back. The one that tells you the brief is wrong is usually the one worth hiring. If cost is the deciding factor at this stage, our breakdown of what a B2B SaaS marketing agency costs covers what each retainer band actually buys, and if you are still weighing the build-versus-buy question, in-house versus agency lays out the case for hiring instead.