Half the threads I was counting could never be marked as answered. Dropping them moved my number by thirteen points.

In August I counted 300 discussion threads and reported that 19.7% of them were ever marked as answered. I put that number in an article. A client put it in a board deck.

Four weeks on I re-ran the script to check the figure had held. It had not moved much. What moved was my understanding of what I had counted.

GitHub has two kinds of discussion category. In one, somebody can mark a reply as the answer. In the other, that button does not exist at all. Announcements. Show and tell. General chatter. I had counted every thread in both kinds and then divided by the lot.

So a thread where three people solved a problem in ten minutes, posted in a category with no answer marker, went into my denominator as unresolved. Forever. There was never a mechanism by which it could be anything else.

Filter to answerable categories only and the same repositories give 33.1%, not 19.7%. Silence drops from 31.3% of threads to 10.6%.

Thirteen points and twenty-one points. On a metric I was handing to people who make decisions with it.

Now the part I was hoping the new run would settle.

Every community playbook has a response-time target, mine included. I have written the sentence about replying within four hours more than once. So I ranked the five rooms by how fast the first reply arrives, then ranked them again by how often the question gets closed, and looked for the pattern.

There isn’t one.

Tailwind CSS answers in a median 0.7 hours and resolves 45.5%, the best of both. Then Prisma: 30.6 hours, the slowest room here by a factor of eight, and 40.0% resolved, which is second best. Vite sits in the middle on speed and near the top on resolution.

And Next.js replies in 1.6 hours. It resolves 15.4%, the worst number in the set.

Fast and good. Slow and good. Fast and bad. The two rankings share almost no order, which is the shape of a relationship that is not there. I cannot tell you speed hurts, and I was hoping to.

What Astro did to the sample

Astro contributed 60 threads in August and contributes none now, because not one of its discussion categories allows an answer to be marked.

A room with no answer marker has no resolution rate. Not a low one. None. If I reported 0% for it, which my August method effectively did, I would be reporting the absence of a button as the absence of solutions.

Next.js nearly went the same way: 13 of its 60 threads survived the filter, and I am wary of what a 13-thread sample is worth. It is in the chart because leaving it out would be choosing my data after seeing it, and it is flagged because thirteen is not many.

What I would change on a dashboard this week

Find out whether your platform has a marked-answer concept. Discourse has solved. Slack has nothing, and nobody should be reporting a resolution rate from Slack without saying how it was derived.

Then compute resolution over answerable threads only, and put silence next to it.

I keep coming back to Prisma, thirty hours to a first reply and the fewest abandoned threads of anyone here. Three out of sixty. Either something deliberate is happening there that the API cannot see, or the questions in that room are different in kind from the questions in the others. Both are testable and neither is testable by counting, which is the problem with having a script that runs in ninety seconds.

Questions people ask about this