How We Gather (and How to Interpret) Our Data
Last updated October 2023
Our tables rating individual companies will be more valuable to you if you know how the data were gathered and how they should be interpreted.
Opinion Surveys
We regularly survey area consumers for their feedback on services they have used. We primarily survey our subscribers, but also invite other randomly selected consumers to provide ratings.
On our surveys, we ask consumers to rate their experiences with services they had most recently used on several aspects of service. Our ratings tables show the percent of each company’s surveyed customers who provided a rating of “superior” (as opposed to “inferior” or “adequate”) for questions on our survey.
Since many companies were rated by rather small numbers of raters, small differences between two firms in the percentage of raters who gave a particular rating (say, “superior”) should be ignored. The table below gives a rough guide to minimum differences you should look for in deciding on one company over another.
When using these survey data, remember that the questions are to some degree subjective and that the differences among companies might be explained by differences in the personalities, backgrounds, critical standards, and other characteristics of the raters or by biases these raters might have.
Timeliness of the Data
All of the data must be interpreted in view of timeliness.
Our customer survey data are from surveys conducted from January 2017 to November 2024. Survey respondents were asked to report on experiences in the preceding year. Our survey of auto body shops asking for their ratings of auto insurers was conducted from May to July 2023.
Price information was collected from the Massachusetts Division of Insurance’s excellent auto insurance premium comparisons website. You can access the state’s comparisons, which include sample premiums for several more coverage amounts and locations than we report on, by visiting mass.gov/service-details/auto-insurance-premium-comparisons. Rates used in our comparisons were those reported on the state’s website in September 2023.
Top Ratings
We give checkmarks to companies that score highest on a scoring system that we devise for each service field. Our scoring systems weight the various data in our tables and text based on our subjective judgment of their importance. Since the scores are based entirely on information presented, you can apply your own subjective judgments, and decide whether you prefer companies we have not given checkmarks. Where we do not have important data on a company, we cannot give our checkmark.
| A Rough Guide for Deciding Whether the Difference Between Two Percentages is Important | If one company had this number of ratings: | And a second company had this number of ratings: | Do not give much importance to the difference between the frequency with which the two companies experience a particular rating or result unless the difference is at least this many percentage points: |
|---|---|---|---|
| Assuming the average of the two companies' percentages is 50 percent | 10 30 60 120 |
10 30 60 120 |
45 26 18 13 |
| Assuming the average of the two companies' percentages is 80 percent | 10 30 60 120 |
10 30 60 120 |
36 21 15 10 |