Even though hornworm caterpillars can grow up to 6 inches in length, they are adept at hiding in your garden until they have chewed their way through a lot of foliage or even fruit. They feed primarily on plants in the nightshade family. In your garden that usually means tomato plants, although they love my pepper plants too. They may even munch your eggplant and potato plantings.
I have found that a good way to watch out for these pests is to look for their black frass (poop pellets). I often find these easier to spot than the camouflaged caterpillars. If you don’t notice the frass or the caterpillars soon enough you could be confronted with stems stripped of leaves (or even completely missing stems) and chewed or missing fruits.
Now before you go and reach for something to spray your plants with, remember that you want to eat the fruits of these plants, plus I find their adult stage to be very interesting. The sphinx moths of both of these caterpillars often make me do a double take because they look a lot like hummingbirds in flight. Therefore, I always handpick the hornworm caterpillars and fling them into my pasture where they can munch the purple and silverleaf nightshades that they find there. If you must spray, Bt is a natural control of these and other caterpillars.
(Click on images to see larger views.)