Hi Alexandru,
I fully agree with Martin on this one, that all controls need an indication that they are focused, and that's why we have the (UI standard) dotted border. There is a nice article on that subject here for instance.
To remove the border, you will have to dig down to the individual control's CSS and even JavaScript, and you will need to override the default behaviour via a Theme. I am no expert on CSS, but just to give you an idea on what needs to be done, then in a new Theme you can navigate to the RadioButton node, switch the Theme designer to Stule Sheets and override the CSS on that node.
To know what you need to change, I find it very useful to switch Obscuring off in your application and then view the rendered Html to see what CSS styles you need to change. For this sample's purpose, you can locate the ".RadioButton-Control_Focus .RadioButton-Label" class and add a "border:none;" to that class. Whether it's the best CSS practice or not, I can not tell, but it works the way you asked for in IE9 at least.
Similar changes need to be tracked down and changed for the other controls you need this for.
Hope this helps,
Palli