"Make sure there is a phone icon next to the contact."
So I made a new contact with a phone icon and suddenly it worked! Then I needed a way to get them all to have phone icons and another person suggested exporting all contacts to the SD card and importing them back to the phone, which also seems to have worked. If I press "merge with google" all the phone icons go away.
Now to play more with internetless voice dailing and try to get that working too.