Physical and magical holds can prevent you from performing actions. Protection from holds and free action can prevent this. There may be other ways to protect yourself.