Ah, that's because sonarr doesn't allow different save paths per show, it will download every show to the same path. Since I only use rss feed for airing shows, they get uploaded in single .mkv files per torrent. So I set the save path for each rss filter to a different folder per show, like so:
/downloads/Anime/Kaguya-sama Love is War/Season 2/
or
/downloads/Anime/Plunderer/Season 1/
EDIT: I don't want to have duplicate copies of my files and I could never figure out how to get hardlinks to work, so I've given up on sonarr for my airing shows.