You can also find out who's attacked you within the past 23 hours by clicking on the skull icon underneath the experience bar. If we're not talking about hundreds of attacks, I like to view them in my game history under the Main Menu. I can see the details of the attacks, whether I won or lost and by much, if anyone declanned me to attack me, etc.