This is an issue when on wi-fi calling in general. Also if you have visual voicemail they often don’t come in until you have cellular service.
It would be nice if Apple would support RCS and MMS could just go the way of the dinosaurs. For now if non-iPhone users need to send media it’s better to use third-party messaging apps like Signal or a Matrix chat app.