Confirmed: Dealers do move cars around, even from state-to-state. My friend wanted a specific new vehicle and the dealer here in Wisconsin did a trade with a dealer in Indiana. They had a guy drive from IN to WI with the new car to do the swap. And that is just an extreme example. Local dealers will shift cars around from "storage lots" to the main lot, move around the parking lot, etc. When we have car shows or dealer renovations (parking lot paving, plowing even) cars get moved. As long as your warranty starts at that mileage I don't really see an issue if there are some miles.
I would say as long as they are upfront and give you the option to decline, that may be OK. Our dealer supposedly was going to order a 2WD 2000 Tundra for us. Instead, our Virginia dealer made a trade with a dealer in Tennessee and drove it 500 miles. They then swapped out some parts, including the leather interior, with a 4WD Tundra they had on the lot. We did not find out about it until afterwards. We did not end up with the bench seat we wanted. It took a couple of trips to the dealer to get the interior swap correct. When I first drove our 2WD, it had the 4WD controls on the dash. Later, they swapped the passenger airbag keyswitch. We were not too happy but bought anyway due to the long wait. Interestingly, all our subsequent purchases have been from a dealer 150 miles away, saving money in the process ($2000 on the Prius).