I have an AutoDJ playlist of ~500 songs, and a couple of months ago Nightbot stopped shuffling it.
When requested songs are done playing, it switches back to AutoDJ playlist all right, but it
a) Always starts from the first song in that playlist
b) The order of tracks is always the same
Digging through the forum I’ve found similar reports of 2-3 months age, with devs responding that the fix is on the way etc, however the problem still exists.
I mean how hard can it be to implement naiive Math.floor(Math.random() * playlist.length); and requeue the song by its index? 
P.S. It’s a joke, but still, I’m kind of tired of listening first couple of songs from my playlist every stream)