Being frustrated because of what people told me might be the reason I was shy. I was afraid that what I said would annoy the person I was having a conversation with. And that used to be a problem of mine for quite some time.
Potentially you can talk about anything without being frustrated. From what I've observed, the core issue is that there is something that you and the other person should be talking about. It's ok if you want to lead the conversation towards topics that are more interesting to you, but it may be that the root of the problem is that you are judging the experience as negative in some way (you say it's irrelevant), and that is preventing you from just talking to another person. In other words it seems like you consider these topics as irrelevant and so they shouldn't exist
In reality, what happens in most people's everyday lives is not that "interesting". They just don't let that be an obstacle to socializing. Why would they? The two reasons I think of are fear, and the desire to be considerate
And finally, if you are interested in reframing these situations, there is an idea I got from the school of life that can be helpful. The gist of it is that small talk can be an opportunity, or an introduction to talking about deeper topics (if that's what you are concerned about).
Here is the link
https://www.theschooloflife.com/thebookoflife/what-to-do-at-parties-if-you-hate-small-talk/