All Gameboy Advances play GB, GBC, and GBA. The best GBA is the SP model AG-101 which has a factory backlight screen. However, there are tons of modifications available for Gameboys out there. I say get a used GBA and go to https://www.retromodding.com/ and go to town on it. Customize it the way you want
For ROMs only, the absolute best way to play is to get a Nintendo 3DS or 2DS and install home brew software. And as a bonus you can play the entire DS and 3DS library as well.
FYI the Nintendo DS can play GBA games, but it does not support GB or GBC.
I know you said you wanted handheld, but for anyone else reading my absolute favorite way to play is with a Gamecube and a Gameboy Player! It actually plays all GB, GBC, and GBA games! Gameboy Players can be cheap, but the boot disc is expensive. To get around this there is home brew software for the Gameboy Player that is very good, although I haven't tried it myself. To use it you need an Action Replay and a cheap SD card to GC memory card adapter. The software will boot the Action Replay first, then boot whatever is on the SD card. (I bet there are ways to load roms too but I haven't looked). I play with a Raphnet SNES to Gamecube adapter.