If you're not sure and cannot find a caresheet on the type of frog you have, always start with crickets.
Crickets are easy and basic. You can even make a cricket house so you don't have to touch those icky bugs! See Dealing with Crickets for the Squeamish Pet Owner for more information.
Most frogs and toads will eat all sorts of bugs and wiggly jiggly things that most people would cringe to see crawling on their bedroom wall at night... so if in doubt, start with crickets, and add anything else you find them interested in eating. The big fatties sometimes eat goldfish and guppies, and some will even eat mice!
As for teeny tiny frogs, you can try baby crickets or flightless flies or even live bloodworms placed in an upside down milk-cap (or some sort of shallow dish that has a bit of water for the worms.
If you've got an aquatic frog (i.e. it stays underwater all the time), start with frozen bloodworms or brine shrimp.

This is probably one of the most common questions that I get asked. Your best bet is to experiment and try out how much food seems to be good for your frog. Try dumping in 3 or so crickets per feeding session per frog, and see whether the frogs seem to still be hungry or starts to look underfed. Remember, frogs really do seem to have distinct behaviors, even within a species! Some will gorge themselves on any available tasty morsels regardless of need, while others will only eat once in a while when hungry. Excessive feeding is not healthy for many frogs, though some simply choose to ignore excessive food when not hungry. Determining how much to feed the frog is often a matter of understanding your frog's personality!
Other types of food can be tried (ex. mealworms, waxworms, grasshoppers,etc.) But crickets seem to be the "food of choice."
There are some guideline recommendations listed on the Dealing With Bugs page.

Some frogs will only eat moving food. Some may be eating but you just don't see them do it...(I have often seen my firebellies sit around with crickets hopping all over their tank and seem not to eat...yet somehow the cricket population in my frogtank is gone within the next few days...(and I'm positive that the crickets haven't found an escape route!) More often than not, however, lack of appetite is a side effect of other illness. Usually, when the frog recovers from illness, the appetite returns. (See the Frog Doctor page for more info on frog health) The first thing to do in this case is to make very sure that the frog's environment is clean and to administer any relevant treatments for obvious diseases. In the meantime, you may want to try coaxing your frog to eat by hand-feeding.
First of all, you'll need to kill a cricket or whatever your frog usually eats, and have it ready in blunt forceps. (the kind with round-ball tips so your frog wont hurt himself on the tips)
Case one, where things aren't totally desperate: try just wiggling the food right in front of the frog. If this doesn't work, you may want to force-feed it.
To do this, hold the frog in your non-writing hand, with your thumb on the back of its' head, and your forefinger wrapped around under its neck. (Obviously, you should be sure to wash your hands really well both before and after this procedure!) Press firmly, but not too hard as to squish it.
With the other hand, get a drivers license, or stiff card, and with the rounded corner gently open the frogs mouth. When the card is in the mouth, just press the frogs' mouth together so the card stays in there.
Pick up the cricket (or whatever) in the forceps and by maneuvering the card, tilt it downwards, opening the frogs' mouth. When it is open, just place the cricket in the frog's' mouth. When the card is pulled out of the mouth it will eat the cricket with no troubles at all.
Be careful not to be harsh on the poor thing though, because if it's so bad that you need to do this at all, don't forget that your froggy buddy ain't feeling too hot in the first place!
If this doesn't help, or you'd rather not risk it yourself, I'd suggest you bring in your froggie to visit your local vet for a checkup.

For those unable to get super tiny crickets to feed their tiny tree-frogs, one visitor wrote with this great advice:
"My daughter went camping and brought home a couple tiny tree frogs not more that 1/2 inch long each. We were unable to buy crickets in our area that were small enough for them to feed on.
What we found out (actually my wife thought about it) was that we had plenty of food for them in our own back yard. We found it on our rose bushes ... aphids! The frogs seem to love them, and they've been doing just fine!"
One added note though: it probably wouldn't hurt to dust the aphids with some vitamin powder every now and then!