Well.. you might be, but it's ok because your cat doesn't think so.
If you have ever lived with more than one cat, you might have noticed they don't really meow at each other. Rather than meowing at each other, they communicate with body language.
Cats will eventually learn your habits and find you don't catch on to their little body language. They also hear the way you chat with others, so they learned to get your attention they need to be more vocal. Now some are more vocal than others but when they are meowing they are indeed meowing to you.
So don't be rude! Meow back!
Unfortunately, if your cat is meowing or talking to you they need/want something. They are not checking in to see how your day went. But it is still pretty cool that they are trying to communicate with you.
An experiment by J Comp Psychol examined humans and cats to see how they were communicating with each other and found that people were better at interpreting the meows from cats they lived with versus cats they didn't know.
So you and your cat basically have your own language!