Is Consumer Reports Letting Readers Down?

I don't think it's anything sinister. I really do believe they try to be objective. My take is that they uncovered some significant methodological flaws with their car seat research, whereas the only (admitted) error with the hybrid report was with the math. Yes, their methodology there is open to scrutiny, but they thought -- and apparently still think -- it was good. So the response was probably appropriate to what they genuinely believe the seriousness of each issue to be.

 

Who knows but the car issue has a picture of the Prius in the box talking about reliability.

Wildkow

 

<div class='quotetop'>QUOTE(micheal @ Apr 25 2007, 06:59 PM) [snapback]430184[/snapback]</div>

I think it might be a case of apples to oranges. When the initial carseat findings were published, they were on radio and television calling for a recall of most of the seats tested as well as governement investigations. This whole fiasco has hurt their credibility, but knowing CR they investigated how this happened and will probably be a better organization because of it.

 

<div class='quotetop'>QUOTE(micheal @ Apr 25 2007, 07:59 PM) [snapback]430184[/snapback]</div>

I didn't even see the correction about the hybrid calculations -- what is the miscalculation you are referring to and is the correction in the same issue that analyzed the carseat test problems?

I do think that they had to put out the report about the carseat testing problems because that was a major big deal. They had mistakenly TRASHED several of the leading carseat brands

 

Unfortunately, Consumer Reports is about the only consumer product testing agency out there, and as such, I see the potential for many mistakes like this. There almost certainly will be future instances like this.

Product testing is a science, and as a science, product testing should have verification testing done. This is not the case with CR. You take CR's word for it, or you don't. There is no other organization like CR so that you might compare their report with CR's to see if the two reasonably agree. That, as I see it, is a problem.

My wife purchased a couple of Maytag appliances because they were recommended by CR. We have had several problems with each. My bet is that there are other consumers out there like us who have had similar results with the same and/or other products.

Also, its my opinion that CR's methodology often includes purely subjective statements. For example, their vehicle reviews have statements like "plenty of pep" and/or things like "handles well." Unfortunately, there is no standard for what exactly is meant by statements like those, and IMHO, including such statements in their reviews "cheapens" their reviews. Those types of statements strike me as perhaps, just perhaps, one notch above vehicle manufacturer commercials that try to entice the "macho" male to buy a vehicle.

Their reliability data may be slightly better since it is collected from CR subscribers that respond to surveys, however, I think it could be even better if CR were to open up that reliability data collection so that people beyond their subscribers could provide input. (AFAIK, CR does not do this. If I'm wrong, please correct me.) Only collecting data from your subscribers who, presumably, only buy products that you recommend just might set up a catch-22 situation that skews the data toward one product or another. In one sense, it could be construed as a bias (though presumably unintended) towards particular products.

So, I suggest taking CR with a grain of salt. Unless another agency comes along that does the same or similar testing and data collection on similar products, there can be no comparison with another set of data and all the consumer has to go on is CR's word.

 

<div class='quotetop'>QUOTE(PriusRos @ Apr 26 2007, 09:12 AM) [snapback]430411[/snapback]</div>

The hybrid (mis)calculation and subsequent correction were in last years' issues. I think the original report was sometime in the spring and the correction a month or two later.

I don't remember the mistake exactly. I think it had something to do with double-counting the Prius' depreciation.

I agree with Wiyosaya, especially about the surveys. Survey respondents are not selected from among the overall population of users of a given product. There is inevitably a selection bias. Even among CR subscribers, they have introduced another potential source of bias. I recently received my annual survey, and beginning this year they are asking for a monetary donation to help fund the survey process. Sorry, but they get my money with my subscription. I didn't return the survey.

 

the hybrid correction, if i remember right was actually pointed out to CR by a Priuschat member. and the car seat correction should have been taken more seriously. the hybrid premium article was just a generalized article not pointed at one car manufacturer or model (they did make a mistake on the Prius)

because of that CR article, car seats have been a nearly impossible thing to buy. the top-rated snug ride is virtually un-buyable. we were able to get one, but cannot buy an additional base for it. (know of anyone who has one for sale??) we have been on a waiting list from the factory for over 3 months. we are now forced to move the base from car to car and its a hassle.

also the companies slammed in the carseat evaluation, some of them are suffering financial hardships, so ya, it should be taken MUCH more seriously.

 

<div class='quotetop'>QUOTE(JimboK @ Apr 26 2007, 10:39 AM) [snapback]430465[/snapback]</div>

Okay, I seem to remember the original article (which came out before I bought my Prius) and it struck me that the depreciation was way overestimated. I don't remember seeing the correction.

<div class='quotetop'>QUOTE(DaveinOlyWA @ Apr 26 2007, 11:47 AM) [snapback]430523[/snapback]</div>

I agree. The carseat issue is much more serious.

 

Consumer Reports is a starting, not an end, point. While CR strives to be objective, too often an "East Coast" bias comes through because they are not aware of similar products sold in different regions of North America.

The "frequency of repair" tables are an indicator, not an absolute.

 

I've never thought much of Consumer Reports' testing, so it doesn't surprise me when they get caught making an error. Their testing methodologies are often very subjective and simplistic, often focusing on factors that I find insignificant. I would much prefer an objective list of features and faults, and some reliability information based on something objective. I no longer bother to read any of their reviews.

Tom

 

I agree that CR is just one tool to use, the beginning, but not an end. Their review of 2004 Prius seems fairly objective to me. The drawbacks they mention have all been aknowleged by us here on PC.

Highs: Fuel economy, low emissions, transmission, ride, hatchback versatility, reliability.
Lows: Steering feel, multifunction display.

A lot of the Prius's appeal is its 44-mpg overall fuel economy and ultraclean emissions. But it also provides a roomy interior, hatchback versatility, and competitive road manners, making it among the top-scoring cars in this segment. Although relatively small on the outside, the Prius has a long wheelbase and efficient space utilization that give it interior room comparable to that of a midsized car. The unusual control layout and gauge cluster, however, may be confusing to some, and the electric steering lacks feedback. Reliability has been excellent.

THE DRIVING EXPERIENCE

The Prius's ride is well controlled and provides good isolation and compliance. The highway ride is steady and composed, but road noise is pronounced on coarse pavement. When starting, the Prius is eerily quiet if powered only by the electric motor. When needed, the gas engine starts automatically, emitting a low-level engine hum. The Prius is not particularly agile and exhibits moderate body lean. However, it was predictable and secure in our emergency-handling tests. Acceleration is similar to that of most four-cylinder models in this class, although the electric motor provides more initial response on takeoff. The vehicle is classified as a Partial Zero Emission Vehicle by the California Air Resources Board. To conserve fuel, the gas engine shuts off when the car comes to a stop, starting automatically when the brake is released. The smooth continuously variable transmission (CVT) is responsive and the dash-mounted shifter is easy to get used to. Braking performance was average, although when stopping there is a faint whine from the car's regenerative braking system, which helps recharge the battery pack. The halogen headlights have good intensity and distance.

INSIDE THE CABIN

Interior fit and finish is very good, and the materials are high quality. The high seating position offers a good forward view, but the two-piece rear window hampers visibility aft. Taller drivers may find the pedals too close. The firm front seats offer good side support, but the lack of height adjustment limits thigh support. The roomy rear seat offers generous leg room for two adults. Head room and thigh support are both sufficient. Access is easy. A center dash-mounted display shows fuel consumption and which power unit is working. Most audio and climate functions are located on the steering wheel, but others, accessible only through the dash display, can be inconvenient to use. Cabin storage space is good. There are two sturdy cup holders, both front and rear. The 60/40-split folding seats expand the cargo area.

SAFETY NOTES

Side-impact air bags housed in the front seatbacks and curtain-style head-protection air bags that extend to protect front and rear outboard passengers are standard for 2007. They supplement required front air bags and three-point seatbelts in all of the five seating positions. An occupant classification system is new for 2006. The system in the front passenger seat monitors the weight of whoever is seated there and will activate the front air bags only when someone of adult size is present. Front air bags and pretensioners will also not deploy if the front passenger seat is not occupied. A driver's seat position sensor and driver and front passenger seatbelt usage sensors also influence the correct level of air bag deployment. Upper anchors for the front seatbelts adjust and pivot for improved fit. Front belts are also equipped with pretensioners and force-limiters to reduce belt slack and forces in the event of a crash. All seats have adjustable and locking head restraints, but the rear versions are too low when fully down to sufficiently reduce rearward head travel. Driving with kids: Some rear-facing infant seats may tilt too much in the center rear seat, but should prove secure in the outboard rear seats. Front-facing seats should also prove secure. There are three tether anchor attachment points on the backs of the second-row seats and LATCH anchors in the rear outboard seats. The LATCH anchors are recessed between the cushion and seatback making attaching and detaching LATCH hooks a bit awkward.

RELIABILITY

We expect reliability to be much better than average, according to our latest subscriber survey.

 

<div class='quotetop'>QUOTE(PriusRos @ Apr 26 2007, 08:03 AM) [snapback]430543[/snapback]</div>

CR retraction: http://www.consumerreports.org/cro/cu-pres...archTerm=hybrid
CR follow up report:
http://www.consumerreports.org/cro/cars/ne...rview/index.htm