You can use the A button to pat/lift them, the brush to stroke them (not applicable to chickens, ducks, the cat or the dog), or you can use the Touch Glove to pat/brush them. It's been a long time, and I'm too lazy to check, but I believe that using the milker/shearer will also increase hearts, both alone and in conjunction with the Touch Glove.
Performing these actions gives you heart points, just like giving gifts to villagers. Once you get a certain amount of heart points, you'll gain hearts with your animals.
Hope this helps!