Edited from Microsoft's help article:
You can play most audio and video files, including mp4, Apple and QuickTime movie files directly from the OneDrive website.
Supported audio and video formats in OneDrive:
The following audio/video files can be played directly from your OneDrive in most browsers: .3g2, .3gp, .3gp2, .3gpp, .aac, .adts, .asf, .avi, .mp3, .m4a, .m4v, .mov, .mp4, .sami, .smi, .wav, .wma, .wmv
If a file doesn’t play, the format might be one that either OneDrive or your mobile device doesn't support.
Microsoft has additional helps for playing videos on iOS devices (like your iPad or iPhone) on the same site. You are encouraged to check there if you still cannot play videos on these devices from the OneDrive App!