For me it was turning off "Autodetect subtitle files" in VLC.
Go to Tools > Preferences > Show All Settings > Scroll down to "Subtitles / OSD" under video (or just search for "autodetect subtitle") in the search bar > Then uncheck "Autodetect subtitle files"