The blog of Zeq.

| Image Type | IE (all versions) | Edge | Chrome | Firefox | Safari | Opera | Netscape 4+ | Notes |
|---|---|---|---|---|---|---|---|---|
| ICO (favicon.ico) | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Most compatible; can store multiple sizes |
| PNG (favicon.png) | No (IE<7) | Yes | Yes | Yes | Yes | Yes | No | Modern; supports transparency |
| GIF (favicon.gif) | No | Yes | Yes | Yes | Yes | Yes | No | Static works; animation mostly ignored |
| SVG (favicon.svg) | No | Yes | Yes | Yes | Yes | Yes | No | Vector; modern high-DPI support only |
| Apple Touch Icon | No | No | No | No | Yes | No | No | For iOS home screen shortcuts |
| Microsoft Tile (mstile) | Yes (IE10+) | Yes | No | No | No | No | No | For Windows Start menu tiles |
| Web App Manifest Icon | No | Yes | Yes | No | Yes | Yes | No | For PWAs; modern browsers only |
Why is PNG/SVG considered the "supreme" icon format? Maybe because it's high resolution, but that's besides the point. I say .ico is supreme for web icons, it's supported by almost browser with icons. IE, Netscape, Chrome, and Edge all support it, but you can't say the same for PNG. Modern formats can’t handle the classics of browsing, and that means you shouldn't use the modern ones.
PNG's Flaws:
PNG favicons aren't support by old browsers, and, eh, their only missing in old browsers.
PNG's Upsides:
PNGs are high resolution, can be very optimized, and look good on HiDPI screens.
SVG's Flaws:
SVGs are even less supported in old browsers than PNG.
SVG's Upsides:
SVGs are fast, optimized, speedy, scale well, and look fantastic on HiDPI screens.
ICO, supreme ICO's Upsides:
ICOs are supported by every GUI browser not 50 years old, ICOs load fast, can have multiple sizes in one file, and have fast load times.
ICO's Flaws:
ICOs are low resolution, and look "blury" on HiDPI screens.