But the default behaviour of pretty much all browsers will still allow that to be downloaded via its right click context menu. Even if you can remove the default browser controls the URL to the video is plainly visible.
That said, if you're users aren't very "savvy", for example, they're not likely to know how to view the page source to get the video URL, then you might be able to disable the context menu for the video element with this custom JS:
Add that to the bottom of your xengallery_media_view template (or similar).
And that is pretty much all you can do. There may be other ways to do this. Obviously sites like YouTube and Vimeo use proprietary video players and on the surface it would seem that these seem to block the ability to directly download their contents, but, of course, there are may ways and many sites out there that will download any content you like from these sites in seconds. So, with that in mind the best you will do is slow people down rather than actually properly preventing this.