Nope. The switch (i've only used cisco/arista so ymmv) will keep a map of resolved macs that have made IGMP subscriptions and only forward to hosts thus subscribed to ip:port groups. View on your subscribing host (netstat -g). This is done in switch hardware, however new subs/changing macs is done in software.
Secondly there is/was (i might be out of date) a bug/wontfix in mcast subs in Linux that no matter the group unique IP, if you subbed on a port it would be forwarded to any group with the same port number. So isolate all services using unique port numbers if you have multiple consumers on one host. (see `man 7 ip` setsockopt IP_ADD_MEMBERSHIP)