Selecting contact groups

I just got a 2016 Prius and have been spending the weekend setting it up.
I was annoyed that my whole contact list was displayed because 99% of them I will never call from the car.
Then I discovered an option that limited the list to only selected contact groups.

I selected my family group and it worked! Only my family members showed up in the Prius contact list.
I went back to my computer and made a new group from people I might call from the car.

Now, back at the Prius, I can't find the option to select contact groups again.
All I can see is my family and there's no way to view any other contacts or groups.
I've been through the manuals and can't find any documentation for it either.

Does anyone know how to access this?