Mostly, they are image file formats based on specific compression methods. They are the most popular and used.

jpg uses in compression several million colors (10 to 24) as well as the png-24 format

gif can use a palette of only 256 colors at a time (as well as png-8);

png - 24 has support for opacity and transparency, making it particularly useful in achieving different css graphic effects. Unfortunately, exit6 does not have support for this and the result will be filling the transparent space with the same color of the body tag. There are a number of techniques to successfully "implement" png images in IE6 but they are based on the filter property of Microsoft browsers so success is not 100% guaranteed

GIF is the only one capable of supporting animations (in the same 256 colors / frame conditions)
the JPG compression algorithm has the disadvantage of reducing the image quality with each new processing. (if I'm not mistaken and gif has the same problem). That is why the PNG algorithm does not have this problem and is suitable for intermediate saving of images before publication.
also the PNG algorithm is based on predictability making it particularly useful in the compression of linear gradients and derivative effects (the list could be continued).

What is the best format?

Basically the 3 formats are indispensable and a web interface should take them into account. Why?

JPG is best suited for photo compression. Its algorithm is recommended for this type of file by returning the smallest files.

PNG is useful for both transparency and opacity as well as compression of those images generated digitally (gradients and effects in photoshop, texts), in general those "sharp" images. A comparison between a jpg and a png that encodes the same image (with the above features) will show the following:

jpg image is pixelated around the graphic elements
the predictable algorithm for png will return the smallest file (observation: If you try to encode a photo for the result, the result will be exactly the opposite - because the jpg algorithm "approximates" the adjacent surfaces, while for png it will have problems in being sufficiently "predictable". )
Comparing png-8 and gif, the result is that the file weight will be less ugly for png, but let's not forget 6 it has problems with transparency and opacity of pngs and in addition the gif is able to "display" animations.