There are a few ways:

Fighting battles with the current party builds affinity, when you use chain attacks, dodge attacks or encourage one another.

Doing the crazy number of sidequests builds it quite well.

Giving "Presents" to each other through the inventory section of the menu. Certain items are more liked and appreciated then others and I'm sure you can find a chart by looking on-line.

Essentially, anytime you see heart icons appear in the lower third of your screen the affinity is being raised through your actions. This also means that once you have more then 3 party members, which is coming fairly soon for you Del, you'll want to swap characters out periodically because you can only build affinity with active members. This means also putting other characters in the lead as well.

Take care all.