In a batch like yours that has had a problem starting I would continue adding slowly, maybe adding a quart, then 1 gallon ... 2 gallon then the rest. There is a good chance that adding the starter to the 5 gallons will work out but your chances improve doing it slowly.
SJO
SJO