Driver's side sun visor problem on 2004 Prius.

I had the same problem with both visors on my 2004, now with 192K miles. For a while, my workaround was to force the visor into the position that I wanted it to be in. Eventually the hinge loosened up so that the visor just dangled in the down position. I also noticed the cloth was beginning to split at the hinge, which shows it was exerting pressure on the black plastic which makes up the body of the visor, beneath the cloth cover.

I replaced the passenger side visor with a used unit purchased from autobeyours.com several months ago. That was great, the used visor was like new.

For the driver's side, I bought a new visor from AutoNation Toyota Gulf Freeway (Houston, TX) since autobeyours does not have driver's side visors in stock. I just received that part yesterday and installed it in the afternoon.

I noticed that the fabric weave of the new visor is slightly coarse compared to the cloth headliner and the passenger-side used visor. The fabric shade is correct (ivory), it is just that the cloth is slightly different.