It's a fair concern, and one the FAA and EASA are closely attending to with current electric aircraft certification.

Thermal runaway is more the problem concern, as it's a known issue with lithium ion batteries. But it's also a known issue with known solutions. It's mostly only poorly engineered lithium ion batteries which go rogue these days.

And as a note, battery electric cars catch fire an awful lot less frequently than internal combustion cars.

https://insideevs.com/news/561549/study-evs-smallest-fire-risk/

That's not all good news, however, as hybrid electric cars were most likely to catch fire, and hybrid electric aircraft are going to be common.

But aerospace is always more highly engineered and safety certified than the automotive space.