I had a tiny mouse in my house for about 2 months. It used to come out and run in front of me in the end at night. I was petrified and my daughter in law eventually took me to hardware store and we spent $45 buying fancy mice traps. Ordinary mouse traps did not work, as food was always gone and mouse still alive.
I ended up avoiding certain rooms where my dogs "told" me it was hiding, but eventually the first day of the new fancy traps, he was gone.
To my horror, he was in the back yard (must have been going in and out through the dog door) and was strangled with a thing round his neck. I actually ran away and cried as I felt so bad for him -it was about 2" long at the most and I had to phone someone to get rid of it as cannot touch dead animals.
I would say - buy a cat - but I had a cat that just looked at the mouse. LOL I figured out that they had some sort of agreement. Outside of that, I would just catch them and put them outside. I do know - however - they do NOT eat cheese. Tried to catch them with that method and that failed hopelessly!
Baits are horrible. It's a really cruel death and you can end up with the mice crawling away and dying inside your walls where they will rot and stink. I think traps are the way to go. You can get the killing ones or the kind that trap the mouse alive (though you then have to work out what to do with it). Owning a cat or having a python around can help too. I once put bits of python poo around the corners of the pantry to repel mice. It seemed to work for a little while.
I used to live in the Scottish countryside where fieldmice were common. We also had a cat. A cat who used to like catching baby mice in spring, taking them inside and releasing them. She never bothered when the big wise mice were around. We bought humane traps which caught them alive and unharmed, save for some gorging on peanut butter- they love it! We were able to release them into the wild and have a strict word with the cat.