We have had reports of intermittent video freezing when using wired internet connections. An investigation has isolated the root cause to a regression in UDP handling in the latest stable version of macOS.
On macOS Big Sur, UDP packets of a specific size are not transmitted properly. Apple is aware of the issue and has shipped a fix in the 11.3 beta release of macOS. We have verified that the problem is fixed in the latest 11.3 beta release, and we are confident that the fix will ship in the next stable release. Until the stable release is available, we recommend using a wireless connection as a workaround wherever possible.