I have just found through debugging that my extender was trying to stream media rather than playing it directly using UNC paths. The reason was that my extender users didn't have permission to access the UNC shares on another machine. I had do the following on the machine holding the media shares
1) Grant Everyone read access on the share permissions and underlying directory permissions
2) Turn off password protected sharing (this confused me for a while)
This makes sure that a user which doesn't exist locally is able to access the shares properly.
Without this Directory.Exists reports that the path doesn't exist and MediaBrowser.Library.Entities.Video.VideoFiles decides to create streaming urls meaning that you start to stream your media when you thought you were accessing it directly.
(I have also found some other stability issues with the extenders at the moment more on those later)