A small DC plan is terminating. A participant is deceased, his wife is deceased and there are only children and step children available. The plan is wanting to close out his account. Can they pay the balance(appx $5300) to the children and be done with it? There was no estate trust set up.

Can you use the plan's definition of beneficiary to identify the appropriate party/estate?

As david suggests, if there is no valid benficiary (either because there is no election of beneficiary on file or because the named beneficiary pre-deceased the participant), then the only valid beneficiary is the defualt beneficiary as per the plan's terms.